EP2682925A1 - Vehicle monitoring method and vehicle management system - Google Patents

Vehicle monitoring method and vehicle management system Download PDF

Info

Publication number
EP2682925A1
EP2682925A1 EP12797315.4A EP12797315A EP2682925A1 EP 2682925 A1 EP2682925 A1 EP 2682925A1 EP 12797315 A EP12797315 A EP 12797315A EP 2682925 A1 EP2682925 A1 EP 2682925A1
Authority
EP
European Patent Office
Prior art keywords
information
rfid
collection point
vehicle
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP12797315.4A
Other languages
German (de)
French (fr)
Other versions
EP2682925B1 (en
EP2682925A4 (en
Inventor
Yuebao HOU
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of EP2682925A1 publication Critical patent/EP2682925A1/en
Publication of EP2682925A4 publication Critical patent/EP2682925A4/en
Application granted granted Critical
Publication of EP2682925B1 publication Critical patent/EP2682925B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08GTRAFFIC CONTROL SYSTEMS
    • G08G1/00Traffic control systems for road vehicles
    • G08G1/01Detecting movement of traffic to be counted or controlled
    • G08G1/017Detecting movement of traffic to be counted or controlled identifying vehicles

Definitions

  • the present invention relates to communication technologies, and in particular, to a vehicle surveillance method and a vehicle management system.
  • an urban vehicle is managed by using a radio frequency identification (Radio Frequency Identification, RFID), that is, an electronic identification.
  • RFID Radio Frequency Identification
  • An RFID collection point is arranged in critical areas, such as entrances of cities, urban arterial roads, and main entrances and exits of important buildings, to detect passing vehicles on a real-time basis and record the travel time, travel direction, travel speed, and RFID information of the vehicles.
  • the RFID collection point compares the RFID information with a blacklist of vehicles stored in the VMS, and generates an alarm if detecting a vehicle consistent with one in the blacklist.
  • NVS Network Video Surveillance
  • NVS cameras are installed in critical areas such as entrances of cities, urban arterial roads, and main entrances and exits of important buildings, to perform real-time video surveillance on the roads, and all video records are stored in the NVS.
  • the VMS because the VMS is separated from the NVS, to view video information corresponding to track information of a vehicle, the VMS needs to be switched to the NVS; a corresponding NVS camera identification is queried from the NVS system according to geographical position information of the RFID collection point; and then specific video information is obtained according to the NVS camera identification and time when an accident happens. Therefore, the query process is very complex and inconvenient.
  • Embodiments of the present invention provide a vehicle surveillance method, system, and apparatus to solve the problem in the prior art that due to the separation of a VMS from an NVS, it is very complex and inconvenient to view corresponding video records according to track information of a vehicle.
  • a vehicle surveillance method including: receiving information that is provided by a radio frequency identification RFID collection point and includes RFID information of a vehicle, determining that the RFID information of the vehicle meets a preset condition, and recording the information provided by the RFID collection point sending the RFID information of the vehicle, where the information provided by the RFID collection point includes geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle; determining a geographical query range according to the geographical position information of the RFID collection point; selecting a network video surveillance system NVS camera in the geographical query range according to the determined geographical query range; intercepting, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera; and combining the information provided by the RFID collection point and the intercepted video record.
  • a vehicle management system VMS including a vehicle information recording module, a geographical query range determining module, an information combining module, a camera selecting module, and a video record intercepting module.
  • the vehicle information recording module is configured to receive information that is provided by a radio frequency identification RFID collection point and includes RFID information of a vehicle, determine that the RFID information of the vehicle meets a preset condition, and record the information provided by the RFID collection point, where the information provided by the RFID collection point includes geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle;
  • the geographical query range determining module is configured to determine a geographical query range according to the geographical position information of the RFID collection point;
  • the camera selecting module is configured to select a network video surveillance NVS camera in the geographical query range according to the determined geographical query range;
  • the video record intercepting module is configured to intercept, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera;
  • the information combining module is configured to combine the information provided by the RFID collection point and the intercepted video record.
  • information provided by an RFID collection point is combined with an intercepted video record, so that a vehicle is presented by using a combination of the information of the RFID collection point and the video record. Therefore, a scene video record can be viewed visually when the information provided by the RFID collection point is viewed, thereby increasing the availability of a system. In addition, when the alarm of a vehicle whose RFID information meets a preset condition is processed, the scene situation can be viewed visually, which makes subsequent processing more convenient.
  • FIG. 3 A first embodiment of a vehicle surveillance method is shown in FIG. 3 .
  • Step 101 Receive information that is provided by a radio frequency identification RFID collection point and includes RFID information of a vehicle, determine that the RFID information of the vehicle meets a preset condition, and record the information provided by the RFID collection point sending the RFID information of the vehicle, where the information provided by the RFID collection point includes geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle.
  • an RFID collection point arranged in each main intersection and each key position of an urban detects RFID information of a passing vehicle.
  • the RFID collection point can send the detected RFID information of the passing vehicle to a VMS. Therefore, after receiving the RFID information of a vehicle sent by the RFID collection point, the VMS determines whether the RFID information of the vehicle meets a preset condition, and when the RFID information of the vehicle meets the preset condition, records the information provided by the RFID collection point sending the RFID information of the vehicle.
  • a blacklist of hit-and-run vehicles is stored in the VMS in advance; when the VMS determines that the RFID of a vehicle sent by the RFID collection point is consistent with the RFID information in the blacklist, the VMS records geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and vehicle speed information detected by the RFID collection point. Further, the VMS stores the geographical position information of the RFID collection point, the information about the time when the RFID collection point detects the RFID information of the vehicle, the vehicle speed information detected by the RFID collection point, and the RFID information of the vehicle correspondingly.
  • the geographical position information includes latitude and longitude information and a vector direction of the RFID collection point.
  • the RFID collection point arranged at intersection A to detect the RFID information of vehicles passing from east to west obtains the RFID information of the vehicle.
  • the latitude and longitude of the RFID collection point are (J1, W1) and its vector direction is from east to west. Therefore, the geographical position information of the RFID collection point recorded by the VMS includes the latitude and longitude information (J1, W1), and the vector direction of the RFID being from east to west, and the time when the RFID collection point detects the RFID information of the vehicle is recorded.
  • the information provided by the RFID collection point further includes vehicle speed information detected by the RFID collection point; in this case, the VMS also needs to record the vehicle speed information detected by the RFID collection point.
  • Step 102 Determine a geographical query range according to the geographical position information of the RFID collection point.
  • a latitude and longitude range is calculated by using the latitude and longitude of the RFID collection point as a center and using a set distance as a radius, and the latitude and longitude range is used as the geographical query range.
  • the set distance is determined according to the density of NVS cameras deployed in a city, for example, the set distance ranges from 200 meters to 500 meters.
  • each road passing by the RFID collection point is used as the geographical query range.
  • the RFID collection point is deployed at intersection A, there are seven straight roads to intersection A according to the geographical information of urban roads, and the seven roads are used as the geographical query range.
  • Step 103 Select an NVS camera in the geographical query range according to the determined geographical query range.
  • the VMS already stores information of each NVS camera, including latitude and longitude information (position information) of each NVS camera, and direction information of each NVS camera. Therefore, the VMS can obtain latitude and longitude information and a direction of each VMS camera in the geographical query range, and select, according to the geographical information of urban roads and the vector direction of the RFID collection point, an NVS camera that matches the RFID collection point.
  • the vector direction of the RFID collection point indicates the travel direction of a vehicle.
  • the vector direction of the RFID collection point may, on the one hand, indicate whether a vehicle travels from east to west or from west to east, and on the other hand, indicate whether a vehicle at the intersection travels at a straight lane, a left-turn lane, or a right-turn lane, so that the travel direction of the vehicle can be further learned.
  • the NVS camera that matches the RFID collection point means that the video records recorded by the NVS camera may include a video record of a vehicle when the vehicle passes by the RFID collection point or include a video record of a vehicle passing by the RFID collection point.
  • the NVS camera may be either an NVS camera that the vehicle passes by before the vehicle passes by the foregoing RFID collection point or an NVS camera that the vehicle passes by after the vehicle passes by the foregoing RFID collection point.
  • NVS camera For example, at an intersection, there are four roads that extend to east, south, west, and north directions, respectively, and eight NVS cameras are arranged to monitor both directions of the roads extending to east, south, west, and north directions, where the eight NVS cameras are in the geographical query range.
  • Which NVS camera monitors a direction of a road can be learned according to the latitude and longitude information and directions of the eight NVS cameras in combination with the geographical information of urban roads.
  • NVS cameras that monitor the eastbound and southbound roads and NVS cameras that monitor the westbound and northbound roads and whose directions are opposite to the travel direction of the vehicle are excluded, while NVS cameras that monitor the westbound and northbound roads and whose directions are the same as the travel direction of the vehicle are selected.
  • Step 104 According to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, intercept a video record corresponding to the RFID information in the video records recorded by the selected NVS camera.
  • the video record corresponding to the RFID information herein refers to a video record when the vehicle passes by the RFID collection point.
  • the VMS obtains video records, recorded by the NVS camera, from an NVS system according to the identification of the selected NVS camera, and then intercepts a video record corresponding to the track information of the vehicle from the video records.
  • the VMS obtains video records, recorded by the NVS camera, from an NVS system according to the identification of the selected NVS camera, and then intercepts a video record corresponding to the track information of the vehicle from the video records.
  • FIG. 4 For details about how to intercept a video record corresponding to the track information of the vehicle from the video records recorded by the NVS camera, reference may be made to the following embodiment illustrated in FIG. 4 .
  • Step 105 Combine the information provided by the RFID collection point and the intercepted video record to present the combination of the information provided by the RFID collection point and the video record.
  • a scene video record can be viewed visually when the information provided by the RFID collection point is viewed, thereby increasing the availability of the system.
  • a scene situation can be viewed visually, which makes subsequent processing more convenient.
  • FIG. 2 illustrates implementation steps of intercepting a video record corresponding to the RFID information.
  • Step 201 Obtain the identification and latitude and longitude information of a selected NVS camera, and calculate a line distance from the selected NVS camera to an RFID collection point according to the latitude and longitude information of the selected NVS camera and the RFID collection point and the geographical information of urban roads.
  • the RFID collection point is deployed at intersection A
  • the NVS camera is installed at intersection C
  • a vehicle travels from intersection A to intersection C
  • a route that the vehicle needs to travel from intersection A to intersection C is determined according to the geographical information of urban roads, and a line distance between intersection A and intersection C is calculated.
  • Step 202 Calculate travel time of the vehicle over the line distance according to the vehicle speed information detected by the RFID collection point and the line distance between the selected NVS camera and the RFID collection point.
  • the VMS has a speed detection apparatus at the RFID collection point; when a vehicle passes by the speed detection apparatus, the RFID collection point can obtain the speed of the vehicle. Therefore, after recording the vehicle speed information, the VMS can calculate the travel time of the vehicle over the line distance between the VMS camera and the RFID collection point.
  • Step 203 According to the information about the time when the RFID collection point detects the RFID information of the vehicle and the travel time, calculate time at which the vehicle passes by the NVS camera.
  • Step 204 Determine a start time and an end time of the video record according to the time at which the vehicle passes by the NVS camera and a video record duration set by the VMS, and intercept the video record in the video records recorded by the NVS camera.
  • Corresponding video record information is obtained by invoking an NVS video record positioning interface according to the identification of the NVS camera and the start time and the end time of the video record, and the video record information is combined with the track information of the vehicle.
  • Determining the start time and the end time of the video record according to the time of the video record and the video record duration set by the VMS is based on the prior art, and is not further described.
  • FIG. 3 A first embodiment of a vehicle management system VMS is shown in FIG. 3 .
  • the VMS includes a vehicle information recording module 301, a geographical query range determining module 302, an information combining module 303, a camera selecting module 304, and a video record intercepting module 305.
  • the vehicle information recording module 301 is configured to receive information that is provided by a radio frequency identification RFID collection point and includes RFID information of a vehicle, determine that the RFID information of the vehicle meets a preset condition, and record the information provided by the RFID collection point, where the information provided by the RFID collection point includes geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle.
  • the geographical query range determining module 302 is configured to determine a geographical query range according to geographical position information of the RFID collection point. The determining manner is described in the embodiment of the method illustrated in FIG. 1 , and is not further described.
  • the camera selecting module 304 is configured to select a network video surveillance system NVS camera in the geographical query range according to the determined geographical query range. The selection manner is described in the embodiment of the method illustrated in FIG. 1 , and is not further described.
  • the video record intercepting module 305 is configured to intercept, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera.
  • the information combining module 303 is configured to combine the information provided by the RFID collection point and the intercepted video record.
  • the geographical position information includes latitude and longitude information and a map vector direction.
  • the geographical query range determining module 302 is specifically configured to calculate a latitude and longitude range by using the latitude and longitude of the RFID collection point as a center and using a set distance as a radius, and use the calculated latitude and longitude range as the geographical query range.
  • the camera selecting module 304 is specifically configured to obtain latitude and longitude information and directions of each NVS camera in the geographical query range, and select, according to geographical information of urban roads and the vector direction of the RFID collection point, an NVS camera that matches the RFID collection point.
  • the information provided by the RFID collection point further includes vehicle speed information detected by the RFID collection point
  • the video record intercepting module 305 includes a line distance calculating submodule 3051, a time calculating submodule 3052, and an intercepting submodule 3053.
  • the line distance calculating submodule 3051 is configured to calculate a line distance between the selected NVS camera and the RFID collection point according to the latitude and longitude information of the selected NVS camera and the RFID collection point and geographical information of urban roads; the time calculating submodule 3052 is configured to calculate travel time of the vehicle over the line distance according to the vehicle speed information detected by the RFID collection point and the line distance between the selected NVS camera and the RFID collection point, and calculate, according to the information about the time when the RFID collection point detects the RFID information of the vehicle and the travel time, time at which the vehicle passes by the NVS camera; and the intercepting submodule 3053 is configured to determine a start time and an end time of the video record according to the time at which the vehicle passes by the NVS camera and a set video record duration, and intercept, according to the start time and the end time, the video record in the video records recorded by the NVS camera.
  • the VMS may be a hardware device such as a computer, while the method performing the foregoing functions may be a processor of the computer, and the foregoing functional modules may also be modules in the processor of the computer.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

Embodiments of the present invention disclose a vehicle surveillance method and a vehicle management system. The method includes: receiving information RFID information of a vehicle sent by an RFID collection point, determining that the RFID information of the vehicle meets a preset condition, and recording the information provided by the RFID collection point sending the RFID information of the vehicle; determining a geographical query range according to the geographical position information of the RFID collection point; selecting an NVS camera in the geographical query range according to the determined geographical query range; intercepting, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera; and combining the information provided by the RFID collection point and the intercepted video record.

Description

  • This application claims priority to Chinese Patent Application No. 201110225531.7 , filed with the Chinese Patent Office on August 5, 2011 and entitled "VEHICLE SURVEILLANCE METHOD AND VEHICLE MANAGEMENT SYSTEM", which is incorporated herein by reference in its entirety.
  • TECHNICAL FIELD
  • The present invention relates to communication technologies, and in particular, to a vehicle surveillance method and a vehicle management system.
  • BACKGROUND
  • In a vehicle management system (Vehicle Management System, VMS) of an urban security project, an urban vehicle is managed by using a radio frequency identification (Radio Frequency Identification, RFID), that is, an electronic identification. An RFID collection point is arranged in critical areas, such as entrances of cities, urban arterial roads, and main entrances and exits of important buildings, to detect passing vehicles on a real-time basis and record the travel time, travel direction, travel speed, and RFID information of the vehicles. In addition, the RFID collection point compares the RFID information with a blacklist of vehicles stored in the VMS, and generates an alarm if detecting a vehicle consistent with one in the blacklist.
  • A network video surveillance system (Network Video Surveillance, NVS) is also built in the urban security project, that is, NVS cameras are installed in critical areas such as entrances of cities, urban arterial roads, and main entrances and exits of important buildings, to perform real-time video surveillance on the roads, and all video records are stored in the NVS.
  • In the prior art, because the VMS is separated from the NVS, to view video information corresponding to track information of a vehicle, the VMS needs to be switched to the NVS; a corresponding NVS camera identification is queried from the NVS system according to geographical position information of the RFID collection point; and then specific video information is obtained according to the NVS camera identification and time when an accident happens. Therefore, the query process is very complex and inconvenient.
  • SUMMARY
  • Embodiments of the present invention provide a vehicle surveillance method, system, and apparatus to solve the problem in the prior art that due to the separation of a VMS from an NVS, it is very complex and inconvenient to view corresponding video records according to track information of a vehicle.
  • To solve the above technical problem, according to one aspect of the present invention, a vehicle surveillance method is provided, including: receiving information that is provided by a radio frequency identification RFID collection point and includes RFID information of a vehicle, determining that the RFID information of the vehicle meets a preset condition, and recording the information provided by the RFID collection point sending the RFID information of the vehicle, where the information provided by the RFID collection point includes geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle; determining a geographical query range according to the geographical position information of the RFID collection point; selecting a network video surveillance system NVS camera in the geographical query range according to the determined geographical query range; intercepting, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera; and combining the information provided by the RFID collection point and the intercepted video record.
  • According to another aspect of the present invention, a vehicle management system VMS is provided, including a vehicle information recording module, a geographical query range determining module, an information combining module, a camera selecting module, and a video record intercepting module.
  • The vehicle information recording module is configured to receive information that is provided by a radio frequency identification RFID collection point and includes RFID information of a vehicle, determine that the RFID information of the vehicle meets a preset condition, and record the information provided by the RFID collection point, where the information provided by the RFID collection point includes geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle; the geographical query range determining module is configured to determine a geographical query range according to the geographical position information of the RFID collection point; the camera selecting module is configured to select a network video surveillance NVS camera in the geographical query range according to the determined geographical query range; the video record intercepting module is configured to intercept, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera; and the information combining module is configured to combine the information provided by the RFID collection point and the intercepted video record.
  • In the embodiments of the present invention, information provided by an RFID collection point is combined with an intercepted video record, so that a vehicle is presented by using a combination of the information of the RFID collection point and the video record. Therefore, a scene video record can be viewed visually when the information provided by the RFID collection point is viewed, thereby increasing the availability of a system. In addition, when the alarm of a vehicle whose RFID information meets a preset condition is processed, the scene situation can be viewed visually, which makes subsequent processing more convenient.
  • BRIEF DESCRIPTION OF DRAWINGS
  • To describe the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings required for describing the embodiments of the present invention. Apparently, the accompanying drawings in the following description show merely some embodiments of the present invention, and persons of ordinary skill in the art may still derive other drawings from these accompanying drawings without creative efforts.
    • FIG. 1 is a flowchart of a vehicle surveillance method according to an embodiment of the present invention;
    • FIG. 2 is a flowchart of intercepting a video record corresponding to track information of a vehicle according to an embodiment of the present invention;
    • FIG. 3 is a schematic diagram of a vehicle management system according to an embodiment of the present invention; and
    • FIG. 4 is a schematic diagram of a video record intercepting module according to an embodiment of the present invention.
    DESCRIPTION OF EMBODIMENTS
  • The following describes a vehicle surveillance method, system, and apparatus in detail.
  • A first embodiment of a vehicle surveillance method is shown in FIG. 3.
  • Step 101: Receive information that is provided by a radio frequency identification RFID collection point and includes RFID information of a vehicle, determine that the RFID information of the vehicle meets a preset condition, and record the information provided by the RFID collection point sending the RFID information of the vehicle, where the information provided by the RFID collection point includes geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle.
  • In a VMS, an RFID collection point arranged in each main intersection and each key position of an urban detects RFID information of a passing vehicle. In the embodiment of the present invention, the RFID collection point can send the detected RFID information of the passing vehicle to a VMS. Therefore, after receiving the RFID information of a vehicle sent by the RFID collection point, the VMS determines whether the RFID information of the vehicle meets a preset condition, and when the RFID information of the vehicle meets the preset condition, records the information provided by the RFID collection point sending the RFID information of the vehicle. For example, a blacklist of hit-and-run vehicles is stored in the VMS in advance; when the VMS determines that the RFID of a vehicle sent by the RFID collection point is consistent with the RFID information in the blacklist, the VMS records geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and vehicle speed information detected by the RFID collection point. Further, the VMS stores the geographical position information of the RFID collection point, the information about the time when the RFID collection point detects the RFID information of the vehicle, the vehicle speed information detected by the RFID collection point, and the RFID information of the vehicle correspondingly. The geographical position information includes latitude and longitude information and a vector direction of the RFID collection point.
  • For example, a vehicle whose RFID information meets a preset condition passes through intersection A of an RFID collection point from east to west, the RFID collection point arranged at intersection A to detect the RFID information of vehicles passing from east to west obtains the RFID information of the vehicle. The latitude and longitude of the RFID collection point are (J1, W1) and its vector direction is from east to west. Therefore, the geographical position information of the RFID collection point recorded by the VMS includes the latitude and longitude information (J1, W1), and the vector direction of the RFID being from east to west, and the time when the RFID collection point detects the RFID information of the vehicle is recorded. In addition, the information provided by the RFID collection point further includes vehicle speed information detected by the RFID collection point; in this case, the VMS also needs to record the vehicle speed information detected by the RFID collection point.
  • Step 102: Determine a geographical query range according to the geographical position information of the RFID collection point.
  • There are multiple determining manners. For example, a latitude and longitude range is calculated by using the latitude and longitude of the RFID collection point as a center and using a set distance as a radius, and the latitude and longitude range is used as the geographical query range. The set distance is determined according to the density of NVS cameras deployed in a city, for example, the set distance ranges from 200 meters to 500 meters.
  • In another example, according to the geographical information of urban roads stored in an existing GIS of the VMS, each road passing by the RFID collection point is used as the geographical query range. For example, if the RFID collection point is deployed at intersection A, there are seven straight roads to intersection A according to the geographical information of urban roads, and the seven roads are used as the geographical query range. Step 103: Select an NVS camera in the geographical query range according to the determined geographical query range.
  • Specifically, the VMS already stores information of each NVS camera, including latitude and longitude information (position information) of each NVS camera, and direction information of each NVS camera. Therefore, the VMS can obtain latitude and longitude information and a direction of each VMS camera in the geographical query range, and select, according to the geographical information of urban roads and the vector direction of the RFID collection point, an NVS camera that matches the RFID collection point. The vector direction of the RFID collection point indicates the travel direction of a vehicle. For example, for a road extending from east to west, the vector direction of the RFID collection point may, on the one hand, indicate whether a vehicle travels from east to west or from west to east, and on the other hand, indicate whether a vehicle at the intersection travels at a straight lane, a left-turn lane, or a right-turn lane, so that the travel direction of the vehicle can be further learned. The NVS camera that matches the RFID collection point means that the video records recorded by the NVS camera may include a video record of a vehicle when the vehicle passes by the RFID collection point or include a video record of a vehicle passing by the RFID collection point. The NVS camera may be either an NVS camera that the vehicle passes by before the vehicle passes by the foregoing RFID collection point or an NVS camera that the vehicle passes by after the vehicle passes by the foregoing RFID collection point. For example, at an intersection, there are four roads that extend to east, south, west, and north directions, respectively, and eight NVS cameras are arranged to monitor both directions of the roads extending to east, south, west, and north directions, where the eight NVS cameras are in the geographical query range. Which NVS camera monitors a direction of a road can be learned according to the latitude and longitude information and directions of the eight NVS cameras in combination with the geographical information of urban roads. If a vehicle travels from the west and turns left to a northbound road, because it is detected, according to the vector direction of the RFID collection point, that the vehicle travels at the left-turn lane, it is determined that the vehicle travels at the northbound road. Therefore, according to the geographical information of urban roads, NVS cameras that monitor the eastbound and southbound roads and NVS cameras that monitor the westbound and northbound roads and whose directions are opposite to the travel direction of the vehicle are excluded, while NVS cameras that monitor the westbound and northbound roads and whose directions are the same as the travel direction of the vehicle are selected.
  • Generally, the NVS cameras are installed at higher positions which are not easily hidden by surrounding buildings. Therefore, video records recorded by an NVS camera located at the track line of a monitored vehicle and whose direction is the same as the travel direction of the vehicle may generally include images of the vehicle. Therefore, according to an implementation manner, the NVS camera which is located at the track line of the vehicle and whose direction is the same as the travel direction of the vehicle may be directly selected. Step 104: According to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, intercept a video record corresponding to the RFID information in the video records recorded by the selected NVS camera. The video record corresponding to the RFID information herein refers to a video record when the vehicle passes by the RFID collection point.
  • Specifically, the VMS obtains video records, recorded by the NVS camera, from an NVS system according to the identification of the selected NVS camera, and then intercepts a video record corresponding to the track information of the vehicle from the video records. For details about how to intercept a video record corresponding to the track information of the vehicle from the video records recorded by the NVS camera, reference may be made to the following embodiment illustrated in FIG. 4.
  • Step 105: Combine the information provided by the RFID collection point and the intercepted video record to present the combination of the information provided by the RFID collection point and the video record.
  • According to the foregoing embodiment, a scene video record can be viewed visually when the information provided by the RFID collection point is viewed, thereby increasing the availability of the system. In addition, when the alarm of a vehicle whose RFID information meets a preset condition is processed, a scene situation can be viewed visually, which makes subsequent processing more convenient.
  • FIG. 2 illustrates implementation steps of intercepting a video record corresponding to the RFID information.
  • Step 201: Obtain the identification and latitude and longitude information of a selected NVS camera, and calculate a line distance from the selected NVS camera to an RFID collection point according to the latitude and longitude information of the selected NVS camera and the RFID collection point and the geographical information of urban roads.
  • For example, the RFID collection point is deployed at intersection A, the NVS camera is installed at intersection C, and a vehicle travels from intersection A to intersection C; a route that the vehicle needs to travel from intersection A to intersection C is determined according to the geographical information of urban roads, and a line distance between intersection A and intersection C is calculated.
  • Step 202: Calculate travel time of the vehicle over the line distance according to the vehicle speed information detected by the RFID collection point and the line distance between the selected NVS camera and the RFID collection point.
  • The VMS has a speed detection apparatus at the RFID collection point; when a vehicle passes by the speed detection apparatus, the RFID collection point can obtain the speed of the vehicle. Therefore, after recording the vehicle speed information, the VMS can calculate the travel time of the vehicle over the line distance between the VMS camera and the RFID collection point.
  • For example, if the RFID collection point is deployed between intersection A and intersection C, the line distance from intersection A to intersection C is S, and the travel speed is V, then t = S/V.
  • Step 203: According to the information about the time when the RFID collection point detects the RFID information of the vehicle and the travel time, calculate time at which the vehicle passes by the NVS camera.
  • It is assumed that: the time at which the vehicle passes by the RFID collection point is T1, the travel time is t, and the time at which the vehicle passes by the NVS camera is T2. Then T2 = T1 + t.
  • Step 204: Determine a start time and an end time of the video record according to the time at which the vehicle passes by the NVS camera and a video record duration set by the VMS, and intercept the video record in the video records recorded by the NVS camera. Corresponding video record information is obtained by invoking an NVS video record positioning interface according to the identification of the NVS camera and the start time and the end time of the video record, and the video record information is combined with the track information of the vehicle.
  • Determining the start time and the end time of the video record according to the time of the video record and the video record duration set by the VMS is based on the prior art, and is not further described.
  • A first embodiment of a vehicle management system VMS is shown in FIG. 3.
  • The VMS includes a vehicle information recording module 301, a geographical query range determining module 302, an information combining module 303, a camera selecting module 304, and a video record intercepting module 305.
  • The vehicle information recording module 301 is configured to receive information that is provided by a radio frequency identification RFID collection point and includes RFID information of a vehicle, determine that the RFID information of the vehicle meets a preset condition, and record the information provided by the RFID collection point, where the information provided by the RFID collection point includes geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle.
  • The geographical query range determining module 302 is configured to determine a geographical query range according to geographical position information of the RFID collection point. The determining manner is described in the embodiment of the method illustrated in FIG. 1, and is not further described.
  • The camera selecting module 304 is configured to select a network video surveillance system NVS camera in the geographical query range according to the determined geographical query range. The selection manner is described in the embodiment of the method illustrated in FIG. 1, and is not further described.
  • The video record intercepting module 305 is configured to intercept, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera.
  • The information combining module 303 is configured to combine the information provided by the RFID collection point and the intercepted video record.
  • Further, the geographical position information includes latitude and longitude information and a map vector direction.
  • Further, the geographical query range determining module 302 is specifically configured to calculate a latitude and longitude range by using the latitude and longitude of the RFID collection point as a center and using a set distance as a radius, and use the calculated latitude and longitude range as the geographical query range.
  • Further, the camera selecting module 304 is specifically configured to obtain latitude and longitude information and directions of each NVS camera in the geographical query range, and select, according to geographical information of urban roads and the vector direction of the RFID collection point, an NVS camera that matches the RFID collection point.
  • Further, as shown in FIG. 4, the information provided by the RFID collection point further includes vehicle speed information detected by the RFID collection point, and the video record intercepting module 305 includes a line distance calculating submodule 3051, a time calculating submodule 3052, and an intercepting submodule 3053.
  • The line distance calculating submodule 3051 is configured to calculate a line distance between the selected NVS camera and the RFID collection point according to the latitude and longitude information of the selected NVS camera and the RFID collection point and geographical information of urban roads; the time calculating submodule 3052 is configured to calculate travel time of the vehicle over the line distance according to the vehicle speed information detected by the RFID collection point and the line distance between the selected NVS camera and the RFID collection point, and calculate, according to the information about the time when the RFID collection point detects the RFID information of the vehicle and the travel time, time at which the vehicle passes by the NVS camera; and the intercepting submodule 3053 is configured to determine a start time and an end time of the video record according to the time at which the vehicle passes by the NVS camera and a set video record duration, and intercept, according to the start time and the end time, the video record in the video records recorded by the NVS camera.
  • In the foregoing embodiments, the VMS may be a hardware device such as a computer, while the method performing the foregoing functions may be a processor of the computer, and the foregoing functional modules may also be modules in the processor of the computer.
  • Those skilled in the art should understand that the division of modules of the apparatus provided in the embodiments is division of functions and an actual structure may be the division or combination of the foregoing functional modules.
  • Persons of ordinary skill in the art may understand that all or a part of the steps of the methods in the embodiments may be implemented by a program instructing relevant hardware. The program may be stored in a computer readable storage medium.
  • The foregoing description is merely about the specific embodiments of the present invention, but is not intended to limit the protection scope of the present invention. Any variation or replacement readily figured out by persons skilled in the art within the technical scope disclosed in the present invention shall fall within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

  1. A vehicle surveillance method, comprising:
    receiving information that is provided by a radio frequency identification RFID collection point and comprises RFID information of a vehicle, determining that the RFID information of the vehicle meets a preset condition, and recording the information provided by the RFID collection point sending the RFID information of the vehicle, wherein the information provided by the RFID collection point comprises geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle;
    determining a geographical query range according to the geographical position information of the RFID collection point;
    selecting a network video surveillance NVS camera in the geographical query range according to the determined geographical query range;
    intercepting, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera; and
    combining the information provided by the RFID collection point and the intercepted video record.
  2. The method according to claim 1, wherein the geographical position information of the RFID collection point comprises latitude and longitude information of the RFID collection point and a vector direction of the RFID collection point.
  3. The method according to claim 2, wherein the determining a geographical query range according to the geographical position information of the RFID collection point comprises:
    calculating a latitude and longitude range by using the latitude and longitude of the RFID collection point as a center and using a set distance as a radius, and using the calculated latitude and longitude range as a geographical query range.
  4. The method according to claim 2 or 3, wherein the selecting an NVS camera in the geographical query range according to the determined geographical query range comprises:
    obtaining latitude and longitude information and directions of each NVS camera in the geographical query range, and selecting, according to geographical information of urban roads and the vector direction of the RFID collection point, an NVS camera that matches the RFID collection point.
  5. The method according to claim 2, wherein the information provided by the RFID collection point further comprises vehicle speed information detected by the RFID collection point and the intercepting, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera comprises:
    calculating a line distance between the selected NVS camera and the RFID collection point according to latitude and longitude information of the selected NVS camera and the RFID collection point and geographical information of urban roads;
    calculating travel time of the vehicle over the line distance according to the vehicle speed information detected by the RFID collection point and the line distance between the selected NVS camera the RFID collection point;
    calculating, according to the information about the time when the RFID collection point detects the RFID information of the vehicle and the travel time, time at which the vehicle passes by the NVS camera; and
    determining a start time segment and an end time segment of the video record according to the time at which the vehicle passes by the NVS camera and a video record duration, and intercepting, according to the start time segment and the end time segment, the video record in the video records recorded by the NVS camera.
  6. A vehicle management system VMS, comprising a vehicle information recording module, a geographical query range determining module, an information combining module, a camera selecting module, and a video record intercepting module, wherein:
    the vehicle information recording module is configured to receive information that is provided by a radio frequency identification RFID collection point and comprises RFID information of a vehicle, determine that the RFID information of the vehicle meets a preset condition, and record the information provided by the RFID collection point, wherein the information provided by the RFID collection point comprises geographical position information of the RFID collection point, information about time when the RFID collection point detects the RFID information of the vehicle, and the RFID information of the vehicle;
    the geographical query range determining module is configured to determine a geographical query range according to the geographical position information of the RFID collection point;
    the camera selecting module is configured to select a network video surveillance NVS camera in the geographical query range according to the determined geographical query range;
    the video record intercepting module is configured to intercept, according to the recorded information about the time when the RFID collection point detects the RFID information of the vehicle, a video record corresponding to the RFID information in video records recorded by the selected NVS camera; and
    the information combining module is configured to combine the information provided by the RFID collection point and the intercepted video record.
  7. The system according to claim 6, wherein the geographical position information of the RFID collection point comprises latitude and longitude information of the RFID collection point and a vector direction of the RFID collection point.
  8. The system according to claim 7, wherein the geographical query range determining module is specifically configured to calculate a latitude and longitude range by using the latitude and longitude of the RFID collection point as a center and using a set distance as a radius, and use the calculated latitude and longitude range as a geographical query range.
  9. The system according to claim 7 or 8, wherein the camera selecting module is specifically configured to obtain latitude and longitude information and directions of each NVS camera in the geographical query range, and select, according to geographical information of urban roads and the vector direction of the RFID collection point, an NVS camera that matches the RFID collection point.
  10. The system according to claim 7, wherein the information provided by the RFID collection point further comprises vehicle speed information detected by the RFID collection point and the video record intercepting module comprises a line distance calculating submodule, a time calculating submodule, and an intercepting submodule, wherein:
    the line distance calculating submodule is configured to calculate a line distance between the selected NVS camera and the RFID collection point according to latitude and longitude information of the selected NVS camera and the RFID collection point and geographical information of urban roads;
    the time calculating submodule is configured to calculate travel time of the vehicle over the line distance according to the vehicle speed information detected by the RFID collection point and the line distance between the selected NVS camera and the RFID collection point, and calculate, according to the information about the time when the RFID collection point detects the RFID information of the vehicle and the travel time, time at which the vehicle passes by the NVS camera; and
    the intercepting submodule is configured to determine a start time segment and an end time segment of the video record according to the time at which the vehicle passes by the NVS camera and a set video record duration, and intercept, according to the start time segment and the end time segment, the video record in the video records recorded by the NVS camera.
EP12797315.4A 2011-08-05 2012-05-29 Vehicle monitoring method and system Active EP2682925B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN2011102255317A CN102426782B (en) 2011-08-05 2011-08-05 Vehicle monitoring method and vehicle management system
PCT/CN2012/076184 WO2012167699A1 (en) 2011-08-05 2012-05-29 Vehicle monitoring method and vehicle management system

Publications (3)

Publication Number Publication Date
EP2682925A1 true EP2682925A1 (en) 2014-01-08
EP2682925A4 EP2682925A4 (en) 2014-06-18
EP2682925B1 EP2682925B1 (en) 2020-07-08

Family

ID=45960759

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12797315.4A Active EP2682925B1 (en) 2011-08-05 2012-05-29 Vehicle monitoring method and system

Country Status (3)

Country Link
EP (1) EP2682925B1 (en)
CN (1) CN102426782B (en)
WO (1) WO2012167699A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102426782B (en) * 2011-08-05 2013-10-02 华为技术有限公司 Vehicle monitoring method and vehicle management system
CN103902601B (en) * 2012-12-27 2017-12-26 上海易罗信息科技有限公司 Vehicle fixes search method, equipment and the navigation method of alarm point, equipment
CN104065934B (en) * 2014-07-02 2018-08-24 上海理工大学 A kind of monitoring device and its monitoring method of mobile object
WO2017015852A1 (en) * 2015-07-28 2017-02-02 深圳市润安科技发展有限公司 Method and device for managing foreign vehicle in special workplace
CN105260461B (en) * 2015-10-16 2018-05-04 杭州中奥科技有限公司 Realize the method that space big data is quickly handled and retrieved
CN105512700B (en) * 2015-12-01 2017-03-15 杨春光 Based on RFID and the analysis method and its analysis system of video technique
CN107527464B (en) * 2016-06-22 2019-11-22 杭州海康威视数字技术股份有限公司 A kind of determination method and device of motion profile
CN106530782B (en) * 2016-09-30 2019-11-12 广州大正新材料科技有限公司 A kind of road vehicle traffic alert method
CN106448262B (en) * 2016-09-30 2019-07-16 广州大正新材料科技有限公司 A kind of intelligent transportation alarm control method
CN107920230A (en) * 2017-11-20 2018-04-17 维坤智能科技(上海)有限公司 Video monitoring image secondary-confirmation system and method
CN111093157B (en) * 2019-12-16 2021-12-10 歌尔股份有限公司 Positioning method, management platform and management system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050219375A1 (en) * 2004-03-31 2005-10-06 Makoto Hasegawa Method of retrieving image data of a moving object, apparatus for photographing and detecting a moving object, and apparatus for retrieving image data of a moving object
US20070069921A1 (en) * 2004-05-21 2007-03-29 Sefton Alan K Automated site security, monitoring and access control system
US20100148937A1 (en) * 2008-12-17 2010-06-17 Barry Howard System and Method of Tracking Salvaged Vehicles and Parts Using RFID Tags

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100679495B1 (en) * 2005-06-28 2007-02-07 이경찬 Car Checking System and Method by RF-ID
CN101110160A (en) * 2007-08-31 2008-01-23 天津大学 Vehicle traffic information monitoring method and system combining RFID and machine vision
CN201374122Y (en) * 2009-03-31 2009-12-30 深圳市赛格导航科技股份有限公司 Traffic violation detector
CN101576926B (en) * 2009-06-04 2011-01-26 浙江大学 Monitor video searching method based on geographic information system
CN101977203A (en) * 2010-03-10 2011-02-16 杨波 Motor vehicle travelling automatically monitoring system
CN102426782B (en) * 2011-08-05 2013-10-02 华为技术有限公司 Vehicle monitoring method and vehicle management system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050219375A1 (en) * 2004-03-31 2005-10-06 Makoto Hasegawa Method of retrieving image data of a moving object, apparatus for photographing and detecting a moving object, and apparatus for retrieving image data of a moving object
US20070069921A1 (en) * 2004-05-21 2007-03-29 Sefton Alan K Automated site security, monitoring and access control system
US20100148937A1 (en) * 2008-12-17 2010-06-17 Barry Howard System and Method of Tracking Salvaged Vehicles and Parts Using RFID Tags

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO2012167699A1 *

Also Published As

Publication number Publication date
CN102426782A (en) 2012-04-25
EP2682925B1 (en) 2020-07-08
WO2012167699A1 (en) 2012-12-13
CN102426782B (en) 2013-10-02
EP2682925A4 (en) 2014-06-18

Similar Documents

Publication Publication Date Title
EP2682925B1 (en) Vehicle monitoring method and system
US10971007B2 (en) Road condition information sharing method
US10403138B2 (en) Traffic accident warning method and traffic accident warning apparatus
CN110880236B (en) Road condition information processing method, device and system
US20210014643A1 (en) Communication control device, communication control method, and computer program
US20180158329A1 (en) Methods and apparatus for monitoring traffic data
EP2876622B1 (en) Vehicle-surroundings monitoring device and vehicle-surroundings monitoring system
US11039384B2 (en) Wireless communication system, information acquiring terminal, computer program, method for determining whether to adopt provided information
CN103903465B (en) A kind of congestion in road reason real-time release method and system
US20230227057A1 (en) Reminding method and apparatus in assisted driving, reminding method and apparatus in map-assisted driving, and map
US20060224300A1 (en) System and method for providing information of moving objects' states of move, location data collection system, and car navigation system
CN106662454B (en) Warning notice system and warning notice method
US10996073B2 (en) Navigation system with abrupt maneuver monitoring mechanism and method of operation thereof
CN104730494A (en) Mobile Gunshot Detection
CN110271554B (en) Driving assistance system and method for vehicle
JP2015210713A (en) Driving recorder and cloud road-information operation system using the same
CN105761538A (en) Assistant station reporting method and system based on video recognition and vehicle-mounted terminal
JP2015153208A (en) Alarm system
JP2014044458A (en) On-vehicle device and danger notification method
CN111427063A (en) Method, device, equipment, system and medium for controlling passing of mobile device
CN115359671A (en) Intersection vehicle cooperative control method and related equipment
KR102187908B1 (en) Server, method and computer program for managing traffic
Haupt et al. smartLDW: A smartphone-based local danger warning system
JP2020101986A (en) Safe driving support device, terminal device, safe driving support system, safe driving support method, processing execution method, and computer program
CN104008652A (en) Vehicle monitoring system based on integrated navigation

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20131004

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

A4 Supplementary search report drawn up and despatched

Effective date: 20140520

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 7/18 20060101ALI20140514BHEP

Ipc: G08G 1/01 20060101AFI20140514BHEP

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20170516

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20191220

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1289306

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200715

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012071164

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1289306

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200708

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200708

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201008

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201009

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201008

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201109

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201108

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012071164

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

26N No opposition filed

Effective date: 20210409

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210531

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210531

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210529

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20210531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210529

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210531

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120529

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230524

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230411

Year of fee payment: 12

Ref country code: DE

Payment date: 20230404

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230406

Year of fee payment: 12

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200708