EP3572156A1 - Verfahren zur gepäcksortierung an einem flughafen - Google Patents

Verfahren zur gepäcksortierung an einem flughafen Download PDF

Info

Publication number
EP3572156A1
EP3572156A1 EP19171142.3A EP19171142A EP3572156A1 EP 3572156 A1 EP3572156 A1 EP 3572156A1 EP 19171142 A EP19171142 A EP 19171142A EP 3572156 A1 EP3572156 A1 EP 3572156A1
Authority
EP
European Patent Office
Prior art keywords
baggage
database
data
airport
luggage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP19171142.3A
Other languages
English (en)
French (fr)
Other versions
EP3572156B1 (de
Inventor
Alexa Floury
Hicham El Bernoussi
Emmanuel Piegay
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Solystic SAS
Original Assignee
Solystic SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=63442713&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=EP3572156(A1) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Solystic SAS filed Critical Solystic SAS
Publication of EP3572156A1 publication Critical patent/EP3572156A1/de
Application granted granted Critical
Publication of EP3572156B1 publication Critical patent/EP3572156B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C3/00—Sorting according to destination
    • B07C3/10—Apparatus characterised by the means used for detection ofthe destination
    • B07C3/14—Apparatus characterised by the means used for detection ofthe destination using light-responsive detecting means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C3/00—Sorting according to destination
    • B07C3/10—Apparatus characterised by the means used for detection ofthe destination
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64F—GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR
    • B64F1/00—Ground or aircraft-carrier-deck installations
    • B64F1/36—Other airport installations
    • B64F1/368—Arrangements or installations for routing, distributing or loading baggage
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G43/00—Control devices, e.g. for safety, warning or fault-correcting
    • B65G43/08—Control devices operated by article or material being fed, conveyed or discharged
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G47/00—Article or material-handling devices associated with conveyors; Methods employing such devices
    • B65G47/34—Devices for discharging articles or materials from conveyor 
    • B65G47/46—Devices for discharging articles or materials from conveyor  and distributing, e.g. automatically, to desired points
    • B65G47/48—Devices for discharging articles or materials from conveyor  and distributing, e.g. automatically, to desired points according to bodily destination marks on either articles or load-carriers
    • B65G47/49—Devices for discharging articles or materials from conveyor  and distributing, e.g. automatically, to desired points according to bodily destination marks on either articles or load-carriers without bodily contact between article or load carrier and automatic control device, e.g. the destination marks being electrically or electronically detected
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G06V30/153—Segmentation of character regions using recognition of characters or words
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C2301/00—Sorting according to destination
    • B07C2301/0066—Check for destination address in a database or list
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2201/00—Indexing codes relating to handling devices, e.g. conveyors, characterised by the type of product or load being conveyed or handled
    • B65G2201/02—Articles
    • B65G2201/0264—Luggage
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65G—TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS
    • B65G2811/00—Indexing codes relating to common features for more than one conveyor kind or type
    • B65G2811/06—Devices controlling the relative position of articles
    • B65G2811/0673—Control of conveying operations

Definitions

  • the invention relates to a baggage sorting process in an airport, in which the baggage is transported by a conveying system, and in which a digital image of a tag attached to a piece of luggage and edited to identify said baggage is acquired, and an aerial flight is automatically recognized from said image for controlling said conveying system and orienting the luggage in the airport.
  • luggage is provided with an identification label carrying lines of textual information and a barcode, the latter allowing each piece of luggage to be associated with a set of recorded data.
  • a luggage database built as and when baggage is checked at an airport of departure.
  • the recorded data can represent a passenger name, a boarding number, a flight number, airport codes of departure, arrival and, if applicable, transit, or a flight plan.
  • the reading of the barcode makes it possible to identify and orient a piece of luggage in an automated conveying system in the departure airport as in the airport of arrival or, where appropriate, at an airport of transit, the relevant information in the database being transmitted to the airports concerned.
  • the patent application WO 2017/137106 suggests basing baggage identification on textual information by means of OCR tag analysis including a comparison between the recognized textual elements and the data recorded in the baggage database, with the aim of avoid using a human operator, a source of loss of efficiency and additional cost.
  • this patent application does not answer the problems of sorting baggage due to a lack in the database, an interruption of access to it, or a delay in the transmission of its data.
  • the subject of the invention is a method of sorting baggage in an airport in which baggage is transported by a conveyor system, and in which at least one digital image is acquired of a tag which is attached to a baggage and which has been edited to uniquely identify said baggage, the tag including a legible identification code by a machine, this identification code being provided to correspond to a set of data of a first baggage database constructed as and when the baggage is checked, in which an attempt is made to identify the baggage by a reading of this identification code, and automatically recognizes from said image an aerial flight corresponding to said baggage to control said conveying system and guide the luggage in the airport, the method comprising the following steps: recovering from said digital image of the text information printed on the label, this textual information is compared to data stored in a second local database at the airport before the edition of the tag to identify said baggage, this second database being separate from said first baggage database, the data of the second database being representative of a list of flights nothings to or outbound from said airport for detecting a match and in response to a match detection, it
  • the sorting of an item of baggage in an airport can be based on the association of this baggage item with a given flight on the basis of information from a local database whose access and completeness be easily insured.
  • sorting the baggage can be based on textual information recognized on the label in comparison with data from a database of air flights prepared in advance, these recognized data can be substituted or combined with information stored in a luggage database constantly refreshed and possibly distant from the airport where the luggage is, and therefore whose exhaustiveness and access are likely to cause problems, which makes the identification of a piece of luggage and / or its automatic sorting.
  • FIG. 1A to 3D An airport baggage sorting method according to the invention is illustrated by the Figures 1A to 3D .
  • each piece of luggage transported 110 is identified by means of a label 100 published at the time of the registration of the corresponding luggage and comprising an identification code such as a barcode 120 and textual information of different types that may be useful in tracking this baggage to its final destination.
  • the identification code can be a machine identification code, such as QR codes or RFID chips, without being restricted to barcodes only.
  • the information indicated on the label comprises at least one airport IATA code generally composed of a trigram 122 corresponding to an airport and a flight number 126 composed of a coded name 127 of an airline and a number flight 128 internal to the company, but may also include, for example, a date 124 or a city 130.
  • a luggage database 200 is dynamic, created as and when the luggage 110 is stored so as to store this information, and possibly additional information, in associated with the corresponding barcodes, and is shared between several airports by means of a computer network.
  • a luggage database 200 may consist of sets B (j) of data Dk (j) of different types k (owner, IATA code, flight number, date, etc.), each set corresponding to one piece of luggage. (j) as illustrated by figure 3A , and its function is the identification, tracking and sorting of luggage throughout their journey.
  • a database of air flights 210 is constituted for each airport, listing the flights to or from the corresponding airport during a given activity period, for example one or more consecutive days of activity of the airport. airport, including in particular the day of treatment of a luggage according to the invention.
  • These air travel databases 210 consist of sets E (i) of data each corresponding to an air flight (i), these data comprising at least one airport trigram (destination airport IATA code) T ( (i), and an aerial flight number consisting of an airline company code name C (i) and an airline internal flight number I (i), as illustrated by the figure 3B .
  • Airline databases may be prepared well in advance, prior to the commencement of a given airport activity period and prior to check-in of baggage on board the listed flights, and not intended to be modified during this period; these databases can therefore be considered static at least during a given period of activity, in particular in relation to dynamic baggage databases, refreshed as baggage is checked in or at the end of baggage check for a given flight, including during the busy period of the airport.
  • flight databases can be located in and specific to the airport in question rather than being distant and imposing long-distance connections.
  • flight databases involve greater stability and can be known far more in advance than a baggage database, and their use therefore allows for much greater operational reliability than the use of the database. luggage data.
  • a conveyor system 301 in which an automatic sorting unit 332 controls a conveyor unit 302, such as a set of conveyor belts, which carries the baggage to an automatic identification system 300 comprising a reading device 125 of the bar codes 120 carried by the labels 100 for sorting according to the data Dk (j) associated with this barcode in the database of luggage 200, the reading device 125 being connected to the baggage database 200.
  • an automatic sorting unit 332 controls a conveyor unit 302, such as a set of conveyor belts, which carries the baggage to an automatic identification system 300 comprising a reading device 125 of the bar codes 120 carried by the labels 100 for sorting according to the data Dk (j) associated with this barcode in the database of luggage 200, the reading device 125 being connected to the baggage database 200.
  • the automatic identification system 300 further comprises a gantry 310 equipped with several cameras 320 arranged to acquire digital images of the luggage from different angles of view, in order to maximize the chances of having exploitable images of labels 100.
  • a baggage is passed under the gantry, so that at least one digital image of the tag attached to the baggage is taken during a preliminary step OCR-0 of an optical character recognition software label, called OCR analysis.
  • the automatic identification system 300 includes a computer processing unit 330 which is connected to the automatic sorting unit 332, the reading device 125 and the airline data base 210 of the given airport, and which submits this digital image to the OCR analysis so as to locate therein textual information or lines of texts, present in the form of lines 130 and 131 of alphanumeric characters and to automatically compare them with the data T (i), C (i) and I (i) data sets E (i) of said airline database 210, for detecting a match between the textual information and that data.
  • a computer processing unit 330 which is connected to the automatic sorting unit 332, the reading device 125 and the airline data base 210 of the given airport, and which submits this digital image to the OCR analysis so as to locate therein textual information or lines of texts, present in the form of lines 130 and 131 of alphanumeric characters and to automatically compare them with the data T (i), C (i) and I (i) data sets E (i) of said airline database 210, for detecting a match between the text
  • the identification of relevant lines of characters, that is potentially corresponding to the data of the flight database, advantageously uses the fact that, by convention, the characters of the airport trigram 122 are the largest of a luggage tag 100.
  • the OCR recognition of these lines of characters can include, in this order and as illustrated by the Figure 2A , a single or multiple OCR-1 binarization of the digital image, an OCR-2 determination of aggregations 131 A of pixels of the binarized image having the same color and an expected size of characters of an airport trigram 122 which corresponds to a line of the largest characters present on the image of the label, an OCR-3 calculation of a neighborhood graph of said aggregations 131 A , an OCR-4 determination that a line 131 of alphanumeric characters forms a triplet of neighboring aggregations from said neighborhood graph, this triplet of aggregations plausibly corresponding to an airport trigram 122, a determination and an extraction OCR-5 of a zone of interest 132 of the image substantially centered on said triplet of aggregations 131, an OCR-6 rectification and binarization of said area of interest of the image, an OCR-7 location of lines of text 130 in the vicinity of said aggregation triplet in said area of interest of the rectif
  • a first mode M1 illustrated by the Figure 2B in response to a detection of a Y OCR match of the OCR analysis assumptions with at least one of the data T (i), C (i) and I (i) of one of the sets E (i ) of the database during a step D 0 of detection of concordance or of non-concordance of the hypotheses, the computer processing unit performs a transmission TR-1 of a data representative of an air flight corresponding to the luggage, such as a corresponding flight number 126, to the automatic sorting unit 332 which then command, during a step COM-1, the conveying unit 302 on the basis of the information obtained by said OCR analysis of the steps OCR-0 to OCR-8 and / or data of said set of data E (i ) in accordance with the textual information of the tag retrieved from the digital images, so as to sort by routing the baggage to a collection point PC.
  • the sorting performed by the automatic sorting unit 332 may for example be carried out by means of an assignment table 220 assigning a collection point CP (m) of baggage to each flight (m) and therefore to each set E (m) data from the airline database 210 as illustrated by the figure 3C .
  • the collection points may be for example a conveyor entrance, a boarding gate, or a luggage distribution carousel.
  • the sorting is based entirely on the information obtained by means of the OCR analysis, and thus on the data of air flights stored in an airline database, and the OCR analysis can represent an independent solution on which to base a sorting of luggage.
  • a second mode M2, illustrated by the Figure 2C differs from the first mode M1 in that the OCR analysis is used to recognize data of an air flight corresponding to the baggage and operate the transmission TR-1 to the conveying system in case of a failure N 1 of the decryption M2-1 of the barcode detected during a detection step D 1 of success or failure of decryption of the barcode, for example in the case of a damaged and illegible barcode, or in the case of failure N 2 association M2-2 code decrypted to a set of data B (j) of the luggage database 200 during a step D 2 of detecting success or failure of the association, for example in case of impossibility of connection to the luggage database or lack of relevant data in it.
  • a transmission TR- is carried out.
  • 2 data representative of the baggage automatic sorting system 332 which then controls during a step COM-2 the conveyor unit 302 according to the information of the data set of the baggage database associated with the baggage by its label, to sort the luggage.
  • a third mode M3, illustrated by the 2D figure an M3-1 reading of a barcode of the baggage tag and the OCR analysis described above of the same tag is performed, and then a comparison M3-2 is made between the data obtained by both methods, so as to cross the data obtained through these two methods and thus increase the reliability of sorting baggage.
  • a TR-4 transmission of data representative of non-reading information is carried out on the basis of automatic sorting 332 which then controls the conveyor unit 302 in response to a particular treatment of the baggage during a step E by placing it away from the automatic sorting system, for example by taking it to an output pre-determined and / or by applying a videocoding operation or a fully manual sorting, in order to increase the reliability of sorting.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Quality & Reliability (AREA)
  • Multimedia (AREA)
  • Discharge Of Articles From Conveyors (AREA)
  • Sorting Of Articles (AREA)
EP19171142.3A 2018-05-24 2019-04-25 Verfahren zur gepäcksortierung an einem flughafen Active EP3572156B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
FR1870600A FR3081354B1 (fr) 2018-05-24 2018-05-24 Procede de tri de bagages dans un aeroport

Publications (2)

Publication Number Publication Date
EP3572156A1 true EP3572156A1 (de) 2019-11-27
EP3572156B1 EP3572156B1 (de) 2020-07-22

Family

ID=63442713

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19171142.3A Active EP3572156B1 (de) 2018-05-24 2019-04-25 Verfahren zur gepäcksortierung an einem flughafen

Country Status (5)

Country Link
US (1) US11505411B2 (de)
EP (1) EP3572156B1 (de)
ES (1) ES2823559T3 (de)
FR (1) FR3081354B1 (de)
PT (1) PT3572156T (de)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112193716A (zh) * 2020-10-14 2021-01-08 安徽尚佰智能家居有限公司 一种油漆板材的质检及补标系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3081355B1 (fr) * 2018-05-24 2020-06-12 Solystic Procede de tri de bagages en aeroport avec videocodage optimise
CN114308690B (zh) * 2021-12-31 2024-02-27 厦门民航凯亚有限公司 一种民航旅客行李分拣方法
WO2023209234A1 (en) * 2022-04-29 2023-11-02 Sita B.V. Article processing apparatus, system and method therefor

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017137106A1 (de) * 2016-02-11 2017-08-17 Siemens Aktiengesellschaft Verfahren zur sortierung von gepäckstücken und gepäcksortiersystem
EP3301624A1 (de) 2016-09-30 2018-04-04 Siemens Aktiengesellschaft Baggage-handling-system zum treffen einer sortierentscheidung

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4058217A (en) * 1973-05-01 1977-11-15 Unisearch Limited Automatic article sorting system
US5225990A (en) * 1991-03-26 1993-07-06 Brals Limited Apparatus and methods for baggage reconciliation and location
JP3267489B2 (ja) 1995-11-09 2002-03-18 富士通株式会社 乗り継ぎ情報管理方法及びシステム
US5770841A (en) 1995-09-29 1998-06-23 United Parcel Service Of America, Inc. System and method for reading package information
CA2752641C (en) 2002-05-16 2016-02-09 United Parcel Service Of America, Inc. Systems and methods for package sortation and delivery using radio frequency identification technology
GB0822732D0 (en) 2008-12-12 2009-01-21 Direct Radio Response Ltd Baggage routing system
WO2014177361A1 (de) 2013-04-29 2014-11-06 Siemens Aktiengesellschaft Gepäckabfertigungssystem für einen flughafen
DE202015006102U1 (de) 2015-08-26 2015-11-13 Siemens Aktiengesellschaft Fernauslesung von Gepäckstücken
US10421559B2 (en) * 2015-08-30 2019-09-24 Farzad Soudbakhsh Robotic handling system (RHS)
EP3214590A1 (de) 2016-03-02 2017-09-06 Siemens Aktiengesellschaft Verfahren zur erstellung einer beschreibung eines gepäckstücks und gepäckbeschreibungssystem

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017137106A1 (de) * 2016-02-11 2017-08-17 Siemens Aktiengesellschaft Verfahren zur sortierung von gepäckstücken und gepäcksortiersystem
EP3301624A1 (de) 2016-09-30 2018-04-04 Siemens Aktiengesellschaft Baggage-handling-system zum treffen einer sortierentscheidung

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112193716A (zh) * 2020-10-14 2021-01-08 安徽尚佰智能家居有限公司 一种油漆板材的质检及补标系统

Also Published As

Publication number Publication date
FR3081354A1 (fr) 2019-11-29
ES2823559T3 (es) 2021-05-07
PT3572156T (pt) 2020-10-21
FR3081354B1 (fr) 2021-06-11
US11505411B2 (en) 2022-11-22
EP3572156B1 (de) 2020-07-22
US20190359429A1 (en) 2019-11-28

Similar Documents

Publication Publication Date Title
EP3572156B1 (de) Verfahren zur gepäcksortierung an einem flughafen
US20210042936A1 (en) Authentication-based tracking
EP3572157B1 (de) Gepäcksortierverfahren am flughafen mit optimierter videokodierung
US20180111699A1 (en) Tagless baggage tracking system and method
CA3016217C (en) Method for making a description of a piece of luggage and luggage description system
EP3786836A1 (de) Identifizierung und verfolgung von artikeln
US7839286B2 (en) Method for carrying out IT-assisted customer-oriented airline baggage management
EP3557289B1 (de) Sicherheitskontrollsystem und -verfahren
CN103597434A (zh) 个人信息显示系统和相关方法
US20120263350A1 (en) Method for identifying lost or unassignable luggage
CN108961082A (zh) 一种基于ai图像识别的车险定损系统及方法
CN107644238A (zh) 一种机场行李提取智能指引系统及控制方法
CA3014096C (en) Method for sorting pieces of luggage and luggage sorting system
CN117636527A (zh) 一种智能一体化安检处置台
WO2014060227A1 (fr) Appairage d'un objet et d'un detenteur de cet objet
CN110866570A (zh) 中转行李转码系统
WO2010067132A2 (en) Baggage tracking system
CN112149475B (zh) 行李箱验证方法及装置、系统、存储介质
EP3713218B1 (de) Set zur identifizierung von gepäck
US11501396B2 (en) System and method for collecting and returning luggage at public transportation systems
Li et al. Digitizing and inventorying traffic control infrastructures: A review of practices
FR3094116A1 (fr) Procede suivi bagage
US20230316212A1 (en) Package identification using multiple signals
Koottappilakkal et al. A differentiated model for inbound baggage handling systems at airports
Kirubakaran et al. Driving into the Future: Artificial Intelligence based Traffic Sign Recognition using Learning Assisted OCR Principle

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20191127

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

TPAC Observations filed by third parties

Free format text: ORIGINAL CODE: EPIDOSNTIPA

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20200416

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602019000305

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1292925

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200815

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: FI

Ref legal event code: FGE

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Ref document number: 3572156

Country of ref document: PT

Date of ref document: 20201021

Kind code of ref document: T

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20201014

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: NO

Ref legal event code: T2

Effective date: 20200722

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1292925

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200722

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201023

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201022

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201122

REG Reference to a national code

Ref country code: DE

Ref legal event code: R026

Ref document number: 602019000305

Country of ref document: DE

PLBI Opposition filed

Free format text: ORIGINAL CODE: 0009260

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2823559

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20210507

PLAX Notice of opposition and request to file observation + time limit sent

Free format text: ORIGINAL CODE: EPIDOSNOBS2

REG Reference to a national code

Ref country code: FI

Ref legal event code: MDE

Opponent name: SIEMENS AKTIENGESELLSCHAFT

Ref country code: FI

Ref legal event code: MDE

Opponent name: BEUMER GROUP A/S

26 Opposition filed

Opponent name: SIEMENS AKTIENGESELLSCHAFT

Effective date: 20210416

Opponent name: BEUMER GROUP A/S

Effective date: 20210421

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

PLBB Reply of patent proprietor to notice(s) of opposition received

Free format text: ORIGINAL CODE: EPIDOSNOBS3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210425

PLAY Examination report in opposition despatched + time limit

Free format text: ORIGINAL CODE: EPIDOSNORE2

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210425

PLBC Reply to examination report in opposition received

Free format text: ORIGINAL CODE: EPIDOSNORE3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201122

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220430

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220430

PLAB Opposition data, opponent's data or that of the opponent's representative modified

Free format text: ORIGINAL CODE: 0009299OPPO

R26 Opposition filed (corrected)

Opponent name: SIEMENS AKTIENGESELLSCHAFT

Effective date: 20210416

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230526

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20190425

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PT

Payment date: 20230413

Year of fee payment: 5

Ref country code: NO

Payment date: 20230421

Year of fee payment: 5

Ref country code: FR

Payment date: 20230424

Year of fee payment: 5

Ref country code: ES

Payment date: 20230627

Year of fee payment: 5

Ref country code: DE

Payment date: 20230420

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20230420

Year of fee payment: 5

Ref country code: FI

Payment date: 20230419

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20230419

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230419

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200722

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240418

Year of fee payment: 6