CN114455405B - Off-line elevator control system and method based on face recognition - Google Patents
Off-line elevator control system and method based on face recognition Download PDFInfo
- Publication number
- CN114455405B CN114455405B CN202111624529.7A CN202111624529A CN114455405B CN 114455405 B CN114455405 B CN 114455405B CN 202111624529 A CN202111624529 A CN 202111624529A CN 114455405 B CN114455405 B CN 114455405B
- Authority
- CN
- China
- Prior art keywords
- card
- elevator control
- face
- writer
- control device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012217 deletion Methods 0.000 claims description 17
- 230000037430 deletion Effects 0.000 claims description 17
- 230000006855 networking Effects 0.000 abstract description 14
- 238000012545 processing Methods 0.000 abstract description 9
- 230000003993 interaction Effects 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 14
- 230000005540 biological transmission Effects 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000005286 illumination Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000012546 transfer Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000013078 crystal Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B1/00—Control systems of elevators in general
- B66B1/02—Control systems without regulation, i.e. without retroactive action
- B66B1/06—Control systems without regulation, i.e. without retroactive action electric
- B66B1/14—Control systems without regulation, i.e. without retroactive action electric with devices, e.g. push-buttons, for indirect control of movements
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B1/00—Control systems of elevators in general
- B66B1/34—Details, e.g. call counting devices, data transmission from car to control system, devices giving information to the control system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B1/00—Control systems of elevators in general
- B66B1/34—Details, e.g. call counting devices, data transmission from car to control system, devices giving information to the control system
- B66B1/46—Adaptations of switches or switchgear
- B66B1/468—Call registering systems
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B2201/00—Aspects of control systems of elevators
- B66B2201/40—Details of the change of control mode
- B66B2201/46—Switches or switchgear
- B66B2201/4607—Call registering systems
- B66B2201/4676—Call registering systems for checking authorization of the passengers
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Computer Networks & Wireless Communication (AREA)
- Elevator Control (AREA)
- Indicating And Signalling Devices For Elevators (AREA)
Abstract
The invention discloses an off-line elevator control system and method based on face recognition, wherein a CPU card is used as a carrier of face information data, the face information data required by face recognition can be input into the elevator control system through the CPU card in an off-line environment, the networking of elevator wiring is not needed, and the technical problem of high networking cost of elevator wiring is solved. Meanwhile, because the face information data comprise the face photo data and the data volume is larger, the invention adopts the CPU card with larger storage capacity instead of the IC card as the carrier, thereby ensuring that the invention can be suitable for face recognition schemes with different recognition rates when the face information data are input by using a card mode. In addition, the CPU card contains a microprocessor, the function of the CPU card is equivalent to that of a microcomputer, the CPU card has data processing capability, and the CPU card can realize information interaction with an elevator control device when the face information data is input through a card reader/writer, so that finer data processing in an off-line environment is realized.
Description
Technical Field
The invention relates to the technical field of elevator control, in particular to an off-line elevator control system and method based on face recognition.
Background
Along with the continuous progress of scientific technology, people also put forward higher and higher requirements on the automation level of work and life, and the elevator is increasingly widely applied in buildings, so that the elevator becomes a main riding tool for middle and high-rise owners to go home and go up and down. Most of the elevators in use in the whole country at present have not too many measures in safety management and intelligence, and there is a gap in management of personnel access and elevator use permission. In fact, a substantial part of the building, a part of the floors of the building or the building is not expected to be accessible by anyone at will via the elevator during certain periods of time.
The conventional elevator control system generally controls the floor permission by swiping a card, but the conventional elevator control system carries the card with him or her or occasionally forgets to take the card, so that the embarrassing situation that the elevator can not be controlled to reach a specified floor can not be controlled. With the continuous development of the biometric technology, the cost is reduced, people rely on the biometric technology, and the identity of the people is proved by using characteristics which cannot be replaced by others, so that the biometric technology represented by face recognition is generated and developed rapidly. The face recognition technology is applied to the elevator, so that the elevator control system is convenient, safe and high in recognition efficiency, and meets the requirements of modern people.
For networking type elevator control, face recognition schemes are easy to deploy for the elevator control, networking type elevator control is also available on the market, but most of the elevator control needs to be networking for elevator wiring, and wiring and labor and equipment cost are increased. Particularly, some elevators of old buildings (cells or buildings, etc.) cannot change their wiring structures, which causes inconvenience in deploying face recognition schemes for the elevators of the old buildings (cells or buildings, etc.).
Disclosure of Invention
The invention aims to at least solve one of the technical problems in the prior art, provides an off-line elevator control system and method based on face recognition, can input face information data required by the face recognition into an elevator control system through a CPU card in an off-line environment, and deploy a face recognition scheme for an elevator in the off-line environment without wiring and networking for the elevator, thereby solving the technical problem of high cost of wiring and networking for the elevator.
In a first aspect, the invention provides an off-line elevator control method based on face recognition, which is applied to an off-line elevator control system based on face recognition, wherein the elevator control system comprises an elevator control device, a card reader/reader and a CPU card, and the elevator control device is electrically connected with the card reader/reader;
The elevator control method comprises the following steps:
the elevator control device receives the face information data sent by the card reader/writer, and the face information data is read from the CPU card by the card reader/writer.
In a second aspect, the invention provides an off-line elevator control system based on face recognition, which comprises an elevator control device, a card reader/writer and a CPU card, wherein the elevator control device is electrically connected with the card reader/writer;
the elevator control device is used for receiving the face information data sent by the card reader/writer, and the face information data is read from the CPU card by the card reader/writer.
Compared with the prior art that face information data is required to be input into an elevator control system in a networking mode, the off-line elevator control system and the off-line elevator control method based on face recognition provided by the invention have the advantages that the CPU card is used as a carrier of the face information data, the face information data required by face recognition can be input into the elevator control system through the CPU card in an off-line environment, the networking of elevator wiring is not required, and the technical problem of high cost of the elevator wiring networking is solved. Meanwhile, because the face information data comprises the face photo data and the data volume is larger, the invention adopts the CPU card with larger storage capacity instead of the IC card as the carrier, thereby ensuring that the invention can be suitable for face recognition schemes with different recognition rates when the face information data is input by using a card mode (the face recognition models with different recognition rates have different requirements on the face information data, and the finer and more complex face information data means the larger data volume). In addition, the CPU card contains a microprocessor, the function of the CPU card is equivalent to that of a microcomputer, the CPU card has data processing capability, and the CPU card can realize information interaction with an elevator control device when the face information data is input through a card reader/writer, so that finer data processing in an off-line environment is realized.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The invention is further described below with reference to the drawings and examples;
fig. 1 is a block diagram of an elevator control system in one embodiment.
Fig. 2 is a block diagram of an elevator control system in one embodiment.
FIG. 3 is a diagram of a hardware architecture of a control motherboard in one embodiment.
Fig. 4 is a flow chart of an elevator control method in one embodiment.
Fig. 5 is a flow chart of an elevator control method in one embodiment.
FIG. 6 is a flow chart of reading CPU card data by the card reader in one embodiment.
Fig. 7 is a flow chart of a management operation performed by the elevator control in one embodiment.
Fig. 8 is a schematic diagram of a Haar classifier algorithm.
Fig. 9 is a schematic diagram of a doubly linked list circulation structure.
Reference numerals:
110. a control main board; 121. a card reader; 122. a reader-writer; 130. a voice output module; 140. a face machine; 150. and a CPU card.
Detailed Description
Reference will now be made in detail to the present embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the accompanying drawings are used to supplement the description of the written description so that one can intuitively and intuitively understand each technical feature and overall technical scheme of the present invention, but not to limit the scope of the present invention.
As shown in fig. 1 and 2, the present invention provides two off-line elevator control systems based on face recognition, which include an elevator control device electrically connected to the card reader 121 or the reader/writer 122, a card reader 121/writer 122, and a CPU card 150. In the elevator control system, a face machine 140 is also arranged, the face machine 140 is a terminal device integrated with a camera and a face recognition module, and an elevator control device of the face machine 140 is electrically connected. The human face recognition module formed in the embodiment is a program module implemented by a programming technology, and how to implement the human face recognition module by the programming technology belongs to the prior art, which is not an invention point of the present invention and is not repeated here. In this embodiment, the elevator control device, the card reader 121, and the reader/writer 122 are computer devices capable of data transmission and processing.
Specifically, as shown in fig. 3, the elevator control device in this embodiment is a main chip in the control main board 110, and the control main board 110 provides multiple interfaces connected with the main chip, so as to conveniently extend corresponding functions for the elevator control system.
Specifically, the main chip is used as an elevator control device and is used as all data processing control and analysis centers, and the elevator control device comprises:
(1) The card reader 121 interfaces with the read head to receive and transmit card related data.
(2) The two-dimension code interface is connected with the two-dimension code module and receives information sent by the module.
(3) The fire control and manual detection, the outside connects the switch, just opens all floors after the trigger switch, does not manage the elevator button, can remove the fire control state through manual.
(4) Crystal oscillator and clock source.
(5) RTC, the timing of date, not influenced by the outage, ensure time accuracy.
(6) FLASH storage, personnel information storage, equipment application parameters, card swiping records, loss cards and the like.
(7) ETH ethernet communication.
(8) 485 communication, convenient and quick connection, low cost, capability of receiving an external remote control floor instruction by the interface and capability of upgrading a firmware program.
(9) And outputting voice to prompt the operation state of the card swiping or the face information or prompting the operation state by mistake.
(10) And floor output, namely sending the floor data to a floor control chip through 485, and executing relay action.
(11) Wiegand input, when the face machine 140 or other wiegand card reader 121 outputs a wiegand signal, the relevant card number is searched for and floor information is output for the card number binding personnel information.
As shown in fig. 6, the working principle of the card reading is as follows:
(a) The flow chart of card reading is shown in the right chart
(b) The card is interrogated and if read, the collision avoidance mechanism is next stepped on and the card ID is read.
(c) When the card is read, whether the card is an IC card or a CPU card 150 is further analyzed, an RTAS instruction is sent first to see whether the card has a response, if the card has the response which is the CPU card 150, no response is the IC card, and the information of the card can be queried from the returned information, including the information of a transmission mode, a numbering mode, a manufacturer code number, a COS version number and the like. If the IC card is to be withdrawn, the IC card is unable to store a large amount of face data.
(d) Reading card data, because the CPU card 150 performs three kinds of authentications including external authentication, internal authentication and PIN code sequentially, each authentication has different security levels, the sequential authentication sequence exists, only one authentication is failed, the card can be completely withdrawn, the number of times of wrong authentication is counted, and when the number of times of wrong authentication exceeds a set value, the card can be locked. If the card passes the authentication, the DF file can be opened, and the basic information of the card personnel can be read.
(d) And decrypting the data, wherein the data content adopts a company-independent encryption algorithm because the absolute safety of the data in the card is ensured. At this point decryption is required to restore the data.
(e) Judging the correctness of the data, and prompting an alarm and returning if the illegal card or the data is tampered.
(f) If the data is correct, the card data is packed and sent to the host.
In one embodiment, an off-line elevator control method based on face recognition is also provided, which is applied to the off-line elevator control system based on face recognition. As shown in fig. 4, the elevator control method includes:
step 200: the elevator control device receives face information data sent by the card reader 121/writer 122, and the face information data is read from the CPU card 150 by the card reader 121/writer 122.
It will be appreciated that when the method is applied to the elevator control system shown in fig. 1, the card reader 121 is used to read face information data from the CPU card 150, and when the method is applied to the elevator control system shown in fig. 2, the reader/writer 122 is used to read face information data from the CPU card.
Compared with the prior art that the face information data is required to be input into the elevator control system in a networking mode, the off-line elevator control system and the off-line elevator control method based on the face recognition provided by the invention have the advantages that the CPU card 150 is used as a carrier of the face information data, the face information data required by the face recognition can be input into the elevator control system through the CPU card 150 in an off-line environment, the networking of elevator wiring is not required, and the technical problem of high cost of the elevator wiring networking is solved. Meanwhile, because the face information data comprises the face photo data and the data volume is larger, the CPU card 150 with larger storage capacity is adopted as a carrier instead of an IC card, so that the face recognition scheme with different recognition rates can be applied when the face information data is input by using a card mode (the face recognition models with different recognition rates have different requirements on the face information data, and the finer and more complex face information data means the larger data volume). In addition, the CPU card 150 contains a microprocessor, and its function is equivalent to a microcomputer, and has data processing capability, so that information interaction with the elevator control device can be realized when face information data is input through the card reader 121/reader/writer 122, and finer data processing in an off-line environment can be realized.
As shown in fig. 5, in one embodiment, the face information data includes n+1 packets of face photo data, where N is a positive integer;
the elevator control device receives face information data sent by the card reader 121/writer 122, and includes:
step 210: after receiving the photo reading instruction, the card reader 121/reader 122 reads the 1 st pack of face photo data from the CPU card 150 into the local cache of the card reader 121/reader 122;
step 220: after the card reader 121/reader/writer 122 saves the 1 st packet of face photo data to the local cache of the card reader 121/reader/writer 122, continuing to read the 2 nd packet of face photo data from the CPU card 150 to the local cache of the card reader 121/reader/writer 122;
step 230: while the card reader 121/reader/writer 122 reads the 2 nd pack of face photo data from the CPU card 150 into the local buffer, the elevator control apparatus transmits the 1 st pack of face photo data from the local buffer of the card reader 121/reader/writer 122 into the local buffer of the elevator control apparatus using DMA;
step 240: by analogy, while the card reader 121/writer 122 reads the n+1th packet of face photo data from the CPU card 150 into the local buffer memory of the card reader 121/writer 122, the elevator control device transmits the N packet of face photo data from the local buffer memory of the card reader 121/writer 122 into the local buffer memory of the elevator control device using DMA; until the elevator control device transmits the (n+1) th pack of face photo data to the local buffer memory of the elevator control device from the local buffer memory of the card reader 121/the reader/writer 122 by utilizing DMA, the elevator control device receives the face information data sent by the card reader 121/the reader/writer 122.
Because the capacity occupied by the face information data is relatively large, the face photo data is transmitted to the face recognition module on the elevator control device or the face recognition module in the face machine 140 in a card reading mode, and the transmission process needs to occupy a long time, so that the waiting time process of a card swiping user is caused, and the use experience is influenced. The present inventors have found that the reason for this problem is that, in the prior art, after the card reader 121/reader/writer 122 needs to read the complete face information data to the local cache of the card reader 121/reader/writer 122, the card reader 121/reader/writer 122 sends the complete face information data in the local cache of the card reader 121/reader/writer 122 to the elevator control device, and then the elevator control device sends the complete face information data to the face recognition model, and only after the face recognition model calls the camera to successfully recognize the face of the user who swipes the card, the process of entering the face information data by the card swiping mode is considered to be finished. Therefore, when the card reading failure (such as data loss in the data transmission process) occurs and the face recognition error occurs, the card reader 121/reader/writer 122 is required to re-read the face information data in the CPU card 150, which results in excessively long time for entering the face information data. For example, in the prior art, the size of the complete face information data is 10K, the total time spent for reading the complete face information data by the card reader 121/reader 122 is 3 seconds, the total time spent for transmitting the complete face information data from the local cache of the card reader 121/reader 122 to the local cache of the elevator control device through DMA is 2s, the total time spent for independently transmitting the complete face information data to the face recognition module by the elevator control device is 2s, the face recognition module calls the camera to acquire the photo of the user who swipes the card and recognizes the photo 3s, and the total time spent is 10s.
In this embodiment, the time for inputting the face information data can be greatly shortened. Specifically, in this embodiment, the size of the complete face information data is 10K, including 10 packets of face photo data, the total time spent by the card reader 121/reader 122 for reading the complete face information data is still 3 seconds, but the time spent by the card reader 121/reader 122 for reading each packet of face photo data is 0.3 seconds; the total consumption of transmitting the complete face information data from the local buffer memory of the card reader 121/reader 122 to the local buffer memory of the elevator control device through the DMA is still 2s, but when the card reader 121/reader 122 reads the 2 nd pack of face photo data, the DMA can start transmitting the 1 st pack of face photo data, and the time for transmitting the 1 st pack of face photo data is 0.2s; the total time spent for the elevator control device to independently transmit the complete face information data to the face recognition module is still 2s, but after the elevator control device receives the 1 st pack of face photo data, the elevator control device can transmit the 1 st pack of face photo data to the face recognition module, and the time spent for transmitting the 1 st pack of face photo data is 0.2s; because the decoupling is realized in the process of transmitting data in the three processes, the total time consumption of the first three processes is 3s+0.2s+0.2s, and the time consumption of the last process for the face recognition module to call the camera to acquire the picture of the user with the card and recognize is 3s, in this embodiment, the total time consumption of inputting the face information data is 6.4s, 3.6s is saved, and the time approaching to 1/3 is saved. In some examples, the process of entering face information data may not include invoking a camera to recognize the face of the card swiping user. At this time, the time consumption of the prior art is 7s, the time consumption of the embodiment is 3.4s, the time is saved by 3.6s, and more than half of the time is saved. Thus, the speed of transferring photos is greatly improved through pre-reading and DMA transmission, delay is reduced, and the maximization of transfer rate is realized.
Further, in one embodiment, it is considered that, because the CPU card 150 is in contact with the card reader 121/writer 122 in a wireless manner, the CPU card 150 is far away from the card reader 121/writer 122 due to the fact that the hand holding the CPU card 150 during the swiping process has too large an action amplitude, compared with the wired data transmission, thereby resulting in interruption of the data transmission. In this embodiment, the card reader 121/writer 122 transmits a packet of data to the elevator control device every time it reads the packet of data, so even if the transmission is interrupted during the card swiping process of the CPU card 150, the content of the previous packet is already transmitted to the memory of the elevator control device, so that when the card swiping is resumed, only the unread packet of data (i.e. the face photo data) can be read, and the data reading time is further shortened. In one example, the size of the complete face information data is 10K, including 10 packets of face photo data, when the card reader 121/writer 122 reads the 5 th packet of face photo data, and the card reader 121/writer 122 interrupts reading the 5 th packet of face photo data from the CPU card 150 due to the excessive action amplitude, the elevator control device already stores the 4 th packet of face photo data in the local cache. When the card reader 121/writer 122 determines that the 5 th packet of face photo data is interrupted, the serial number of the incomplete face photo data read from the CPU card 150 during interruption and the identifier of the CPU card 150 are obtained, it can be understood that when the face photo data is written into the CPU card 150 in a packet-by-packet manner, each packet of face photo data carries a serial number, and when the card is swiped, the card reader 121/writer 122 will present each packet of face photo data from the CPU card 150 in the sequence of the serial numbers and record the serial number corresponding to the current face photo data packet. Therefore, when the user corrects the position of the interrupted CPU card 150 again so that the card reader 121/writer 122 can read the CPU card 150, if the card reader 121/writer 122 does not read another CPU card 150 after interruption, the card reader 121/writer 122 will determine whether the CPU card 150 is the interrupted CPU card 150 according to the identifier of the CPU card 150, if yes, determine the sequence number of the face photo data packet that is not completely transmitted when interruption, and resume reading from the 5 th packet of face photo data corresponding to the sequence number, thereby realizing a resume function and further accelerating the data reading time. It will be appreciated that, assuming that after the CPU card 150 identified as a interrupts the reading of the 5 th packet of face photo data, when the CPU card 150 identified as B approaches the card reader 121/writer 122 and wants to record the face photo data, the card reader 121/writer 122 determines that the CPU card 150 is not an interrupted card, so the CPU card 150 will start reading from the face photo data packet with the forefront sequence number and send a verification object change signal to the elevator control device, and after the elevator control device receives the verification object change signal, the address of the DMA transfer data will be reset, that is, the DMA is controlled to transfer the newly received face photo data of the CPU card 150 identified as B into a new local buffer, so as to avoid confusion with the face photo data of the CPU card 150 identified as a. After the CPU card 150 identified as a is interrupted, if the card reader 121/writer 122 reads the CPU card 150 identified as B, if the card reader 121/writer 122 reads the CPU card 150 identified as a again within a set time (for example, within 24 hours) after the interruption, an interruption resume signal is sent to the elevator control device, and after the elevator control device receives the interruption resume signal, the address of the DMA transfer data is reset, that is, the DMA is controlled to transfer the newly received 5 th packet of face photo data of the CPU card 150 identified as a to the local buffer corresponding to the 1 st packet to 4 th packet of face photo data used by the elevator control device to save the CPU card 150 identified as a. If the card reader 121/writer 122 does not read the CPU card 150 identified as a again within a set time (for example, within 24 hours) after the interruption, a delete signal is sent to the elevator control device, where the delete signal is used to instruct the elevator control device to delete the saved face photo data of the CPU card 150 identified as a, and release the corresponding memory, so as to avoid excessive memory occupation.
In one embodiment, before the elevator control device receives the face information data sent by the card reader 121/writer 122, the elevator control method further includes:
the elevator control device receives user management information sent by the card reader 121/writer 122, wherein the user management information is read from the CPU card 150 by the card reader 121/writer 122; the user management information comprises a card number, elevator riding authority, a deletion mark, a face information input mark and a validity period mark of the CPU card 150; wherein, the riding rights comprise a time period for allowing riding, a floor for getting up and down, a building for allowing riding, and the like.
The elevator control device executes corresponding management operation according to the user management information.
The management operation includes:
the elevator control device detects the face information input mark, and when detecting that the face information input mark identifies the current user as a newly added person, the elevator control device executes the step of receiving the face information data sent by the card reader 121/writer 122 (i.e. steps 210 to 240).
The elevator control device detects the deletion flag, and deletes the user management information from the database when it detects that the deletion flag identifies that the current CPU card 150 is in a deleted state.
In this embodiment, the deletion mark may be written into the CPU card 150 by the reader/writer of the management center, and after the user last uses the elevator through the CPU card 150, the elevator control device deletes the user management information corresponding to the user from the database and deletes the corresponding face information data from the face recognition model after the last card swiping, so as to update the data in time in an offline environment, ensure safety, and save storage space.
The elevator control device detects the validity period mark, and when detecting that the validity period mark identifies that the current CPU card 150 is in a state exceeding the validity period, the elevator taking authority corresponding to the current CPU card 150 is cancelled. The property can limit the property charge paid by the owner, and the expired person does not have the elevator taking authority of the elevator.
The elevator control device detects the card number of the CPU, and when detecting that the card number of the CPU card 150 belongs to the loss registration card number, cancels the elevator taking authority corresponding to the current CPU card 150.
In one example, the loss registration is written to the common storage area of the CPU card 150 by the reader/writer of the management center when the user collects face information data to the management center, and the elevator control device reads the data of the common storage area of each CPU card 150 by the reader/writer 122 when the user swipes the card at the elevator.
Further, after the elevator control device completes deleting the user management information corresponding to the user, or when the elevator control device records that the loss report card is at the time point of the present elevator card swiping, the data are also recorded as log data, and when the next time the user swipes the card, the log data are written into the public storage area of the CPU card 150. Preferably, some CPU cards 150 may be defined as preferred cards, for example, a CPU card 150 of a user who needs to re-acquire a face or a CPU card 150 whose validity period has elapsed is defined as a preferred card, which can largely ensure that these data written thereto are read by the management center reader/writer. Specifically, the elevator control device deletes from the external storage device (i.e., FLASH storage in fig. 3) of the elevator control device only after writing these log data into the common storage area in the preferred card; in order to further ensure that the log data can be sent to the management center, it is also possible to limit the corresponding log data to be deleted after writing three preferred cards.
Specifically, as shown in fig. 7, in one embodiment, there are provided a plurality of specific management operation items of the elevator control apparatus, including the following:
(a) The card data is received, and the type of the card (i.e. the CPU card 150) is judged, mainly including a user card and some function cards, and mainly described below as a user card.
(b) Inquiring whether the card personnel information exists, if the personnel ID exists in the ladder control device, marking whether the card personnel is deleted, and if the card personnel is deleted, returning. The function is to delete the owner personnel when the manager needs to delete the card, and the card can be automatically deleted without recovering the card, so that the difficulty of card recovery is solved.
(c) Inquiring whether the card has loss, and when the owner loses the card, preventing others from picking up and using the card, so that the lost card is unlocked by adopting a card loss reporting mode until the loss reporting is released.
(d) Inquiring whether the card has passed the validity period, thereby limiting the property charge paid by the owner, and the expired person has no ladder layer opening authority.
(e) If the item ID is correct, each cell item has an independent item ID, and different items cannot pass through.
(f) If the building ID is correct, each building in the cell also has an independent building ID, and only the personnel who congest the authority of the building can access and pass, otherwise, the access is not allowed.
(g) Judging whether a photo is needed for the newly added person, if so, sending an instruction to the card reader 121 to read photo information (namely face information data), and immediately packaging and sending the photo data to the face machine 140 until the face machine 140 receives the photo. Face information (e.g., name, person ID, expiration date, card number, floor, etc.) is then saved for subsequent rights opening by wiegand code. If the above steps fail somewhere, the card reading function is exited, and the error is prompted by voice.
(h) If not, then judging whether there is photo update, effective date update or other information update, if there is photo update, the process is as above (g), if there is effective date or other information update, the personnel information will be stored directly.
(i) Acquiring floor information, searching the ID of the building from the card content, acquiring the floor information from the building matched with the ID, and simultaneously acquiring the open public floor information.
(j) Inquiring the expiration remaining days of the user card, and if the expiration remaining days are lower than the number of days of the equipment, sending the reminding expiration language.
(k) Outputting the floor information, outputting the floor information to a floor control module, and executing manual or automatic floor actions.
(l) Save the swipe record for later extraction of the swipe information via networking and 485.
In some embodiments, after the elevator control device receives the face information data sent by the card reader 121/writer 122, the elevator control method further includes:
the elevator control device receives user management information sent by the card reader 121/writer 122, wherein the user management information is read from the CPU card 150 by the card reader 121/writer 122; the user management information comprises a face information input mark;
the elevator control device executes corresponding management operation according to the user management information;
the management operation includes:
when the face information input mark is detected, and the face information input mark is detected to identify that the current user is in a face information updating state, the elevator control device receives the face information data sent by the card reader 121/the reader-writer 122 and updates the previous face information data of the current user by utilizing the face information data.
Specifically, the elevator control system further comprises an illumination sensor which is arranged in the elevator and is electrically connected with the elevator control device, under the condition that face recognition fails in the process of inputting face information data, the illumination intensity in the current elevator is collected through the illumination sensor, the illumination intensity data in the current elevator is written into a public storage area of the CPU card 150 through the reader 122, the voice output module 130 is controlled to play voice, and the user is reminded of the face information data input failure, and the user needs to go to a management center to collect the face information data again. When the user re-collects the face information data from the management center, the reader-writer of the management center obtains the illumination intensity parameter and other parameters in the elevator when the user fails to acquire the face from the public storage area of the CPU card 150 of the user, so that the illumination intensity parameter of the collection area can be set to be consistent with the elevator when the management center collects the face information data of the user, and the success rate of face recognition is improved. According to the embodiment, the illumination intensity data in the current elevator are written into the public storage area of the CPU card 150, so that the reason of the face recognition failure can be timely and accurately fed back under the off-line environment of the elevator.
The user management information needs to be reliably stored, and the user management information is changed frequently, so that a search, addition, repair and deletion operation method is needed, how to quickly and accurately find the user management information is very important, and if the control is not good, the user management information is disordered or the data is lost. As shown in fig. 9, in one embodiment, the elevator control method further includes: the elevator control device constructs a doubly linked list circulation structure, each user management information is respectively stored on a corresponding node in the doubly linked list circulation structure, and the doubly linked list circulation structure is stored in a database by the elevator control device so as to facilitate subsequent management.
The data structure of the user management information adopts a double linked list circulation structure, each data node has two address directions, and the two addresses are respectively forward and backward, so that the double linked list can conveniently access the forward and backward nodes at any node, and the node (namely the user management information) is very convenient to find, insert and delete by adopting the idea of a dichotomy, because the accurate positioning of the data can be realized, the ordering disorder can not be caused, and the safety and the accuracy of the data are ensured.
The photo capacity stored by the modern shooting equipment is large, often at least more than 100KB and several MB, and the card storage space is limited, so that only pictures with the size of less than 10KB can be stored, and large-ratio picture compression processing is needed. The compressed photo firstly needs to accurately intercept the face information, if no picture with the face information is detected, the picture cannot be used as identity input, an Adaboost algorithm built in open source Opencv is used, two features (namely two methods) are mainly used for face detection, haar features and LBP features, and the Haar features are described in detail below. As shown in fig. 8, the gist of the Haar classifier algorithm is as follows:
(1) Detection was performed using Haar-like features.
(2) Haar-like feature evaluation is accelerated using an Integral Image (Integral Image).
(3) The AdaBoost algorithm is used to train a strong classifier that distinguishes faces from non-faces.
(4) And strong classifiers are cascaded together by using screening type cascading, so that the accuracy is improved.
Each feature consists of 2-3 rectangles, and in these wavelet diagrams, the light areas represent "accumulated data" and the dark areas represent "data minus the area". Boundary, line, center features are detected separately, and these features can be expressed as:
where wi is the weight of the rectangle, rectsum (ri) is the gray integral of the image enclosed by the rectangle ri, and N is the number of rectangles constituting the featurej.
Adaboost is a statistical-based learning algorithm, and in the learning process, the weight of the feature is continuously adjusted according to the effect of the features of each positive example and each negative example which are defined in advance, and finally a judgment criterion is given according to the performance of the feature.
After face information is detected, effective face is taken, picture compression is carried out, and the compression ratio is up to the specified pixel size. The opencv implementation code is as follows:
new_image＝cv.resize(image,None,fx,fy)
where fx, fy represents the compression ratio of the picture in the width and height directions.
In some embodiments, an off-line face recognition-based elevator control system is provided, the elevator control system comprising an elevator control device, a card reader 121/reader/writer 122, and a CPU card 150, the elevator control device being electrically connected to the card reader 121/reader/writer 122;
the elevator control device is configured to receive face information data sent by the card reader 121/writer 122, where the face information data is read by the card reader 121/writer 122 from the CPU card 150.
In one embodiment, the face information data includes n+1 packets of face photo data, where N is a positive integer;
the card reader 121/writer 122 is configured to:
after receiving the photo reading instruction, reading the 1 st pack of face photo data from the CPU card 150 into a local cache of the card reader 121/writer 122;
after the card reader 121/reader/writer 122 saves the 1 st packet of face photo data to the local cache of the card reader 121/reader/writer 122, continuing to read the 2 nd packet of face photo data from the CPU card 150 to the local cache of the card reader 121/reader/writer 122;
the elevator control device is also used for:
while the card reader 121/reader/writer 122 reads the 2 nd pack of face photo data from the CPU card 150 into the local buffer, the 1 st pack of face photo data is transferred from the local buffer of the card reader 121/reader/writer 122 into the local buffer of the elevator control device by using DMA;
By analogy, while the card reader 121/writer 122 reads the n+1th packet of face photo data from the CPU card 150 into the local buffer memory of the card reader 121/writer 122, the elevator control device transmits the N packet of face photo data from the local buffer memory of the card reader 121/writer 122 into the local buffer memory of the elevator control device using DMA; until the elevator control device transmits the (n+1) th pack of face photo data to the local buffer memory of the elevator control device from the local buffer memory of the card reader 121/the reader/writer 122 by utilizing DMA, the elevator control device receives the face information data sent by the card reader 121/the reader/writer 122.
In one embodiment, the elevator control device is further configured to:
receiving user management information sent by the card reader 121/writer 122, wherein the user management information is read from the CPU card 150 by the card reader 121/writer 122; the user management information comprises a card number, elevator riding authority, a deletion mark, a face information input mark and a validity period mark of the CPU card 150;
executing corresponding management operation according to the user management information;
the management operation includes:
detecting the face information input mark, and executing the step that the elevator control device receives the face information data sent by the card reader 121/reader-writer 122 when detecting that the face information input mark identifies the current user as a newly added person;
Detecting the deletion mark, and deleting the user management information from the database when the deletion mark is detected to identify that the current CPU card 150 is in a deletion state;
detecting the validity period mark, and canceling the elevator taking authority corresponding to the current CPU card 150 when detecting that the validity period mark marks that the current CPU card 150 is in a state exceeding the validity period;
and detecting the card number of the CPU, and canceling the elevator taking authority corresponding to the current CPU card 150 when detecting that the card number of the CPU card 150 belongs to the loss registration card number.
In one embodiment, the elevator control device is further configured to:
receiving user management information sent by the card reader 121/writer 122, wherein the user management information is read from the CPU card 150 by the card reader 121/writer 122; the user management information comprises a face information input mark;
executing corresponding management operation according to the user management information;
the management operation includes:
when the face information input mark is detected, and the face information input mark is detected to identify that the current user is in a face information updating state, the elevator control device receives the face information data sent by the card reader 121/the reader-writer 122 and updates the previous face information data of the current user by utilizing the face information data.
In one embodiment, the elevator control device is further configured to construct a doubly linked list circulation structure, and each user management information is stored on a corresponding node in the doubly linked list circulation structure.
It is to be understood that the system embodiment and the method embodiment of the present application are based on the same inventive concept, and are not described herein.
Those skilled in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a non-volatile computer readable storage medium, and where the program, when executed, may include processes in the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRA), memory bus direct RAM (RDRA), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
Claims (8)
1. The off-line elevator control method based on the face recognition is applied to an off-line elevator control system based on the face recognition and is characterized by comprising an elevator control device, a card reader/writer and a CPU card, wherein the elevator control device is electrically connected with the card reader/writer;
the elevator control method comprises the following steps:
the elevator control device receives face information data sent by the card reader/writer, and the face information data is read from the CPU card by the card reader/writer;
after the elevator control device receives the face information data sent by the card reader/writer, the elevator control method further comprises the following steps:
the elevator control device receives user management information sent by the card reader/writer, and the user management information is read from the CPU card by the card reader/writer; the user management information comprises a face information input mark;
The elevator control device executes corresponding management operation according to the user management information;
the management operation includes:
when the face information input mark is detected, and the face information input mark is detected to identify that the current user is in a face information updating state, the elevator control device receives face information data sent by the card reader/writer and updates the previous face information data of the current user by utilizing the face information data.
2. The off-line elevator control method based on face recognition according to claim 1, wherein the face information data includes n+1 packets of face photo data, N being a positive integer;
the elevator control device receives the face information data sent by the card reader/writer, and comprises:
after receiving a photo reading instruction, the card reader/reader reads the 1 st pack of face photo data from the CPU card to a local cache of the card reader/reader;
after the card reader/reader saves the 1 st pack of face photo data to the local cache of the card reader/reader, continuing to read the 2 nd pack of face photo data from the CPU card to the local cache of the card reader/reader;
while the card reader/reader reads the 2 nd pack of face photo data from the CPU card into the local cache, the elevator control device utilizes DMA to transmit the 1 st pack of face photo data from the local cache of the card reader/reader into the local cache of the elevator control device;
By analogy, while the card reader/writer reads the (n+1) th packet of face photo data from the CPU card into the local cache of the card reader/writer, the elevator control device transmits the (N) th packet of face photo data from the local cache of the card reader/writer into the local cache of the elevator control device by using DMA (direct memory access); and transmitting the (n+1) th pack of face photo data to the local buffer memory of the elevator control device by using the DMA (direct memory access) from the local buffer memory of the card reader/writer until the elevator control device finishes receiving the face information data sent by the card reader/writer.
3. The off-line elevator control method based on face recognition according to claim 1, wherein before the elevator control device receives the face information data transmitted from the card reader/writer, the elevator control method further comprises:
the elevator control device receives user management information sent by the card reader/writer, and the user management information is read from the CPU card by the card reader/writer; the user management information comprises a card number of a CPU card, elevator riding authority, a deletion mark, a face information input mark and a validity period mark;
The elevator control device executes corresponding management operation according to the user management information;
the management operation includes:
detecting the face information input mark, and executing the step that the elevator control device receives the face information data sent by the card reader/writer when detecting that the face information input mark identifies the current user as a newly added person;
detecting the deletion mark, and deleting the user management information from a database when the deletion mark is detected to identify that the current CPU card is in a deletion state;
detecting the validity period mark, and canceling the elevator taking authority corresponding to the current CPU card when the validity period mark is detected to mark that the current CPU card is in a state exceeding the validity period;
and detecting the card number of the CPU, and canceling the elevator taking authority corresponding to the current CPU card when detecting that the card number of the CPU card belongs to the loss card number.
4. An off-line elevator control method based on face recognition according to claim 1 or 3, characterized in that the elevator control method further comprises:
the elevator control device constructs a doubly linked list circulation structure, and each user management information is respectively stored on a corresponding node in the doubly linked list circulation structure.
5. An off-line elevator control system based on face recognition is characterized by comprising an elevator control device, a card reader/reader and a CPU card, wherein the elevator control device is electrically connected with the card reader/reader;
the elevator control device is used for receiving face information data sent by the card reader/writer, and the face information data is read from the CPU card by the card reader/writer;
the elevator control device is also used for:
receiving user management information sent by the card reader/writer, wherein the user management information is read from the CPU card by the card reader/writer; the user management information comprises a face information input mark;
executing corresponding management operation according to the user management information;
the management operation includes:
when the face information input mark is detected, and the face information input mark is detected to identify that the current user is in a face information updating state, the elevator control device receives face information data sent by the card reader/writer and updates the previous face information data of the current user by utilizing the face information data.
6. The off-line elevator control system based on face recognition according to claim 5, wherein the face information data includes n+1 packets of face photo data, N being a positive integer;
the reader/writer is used for:
after receiving a photo reading instruction, reading the 1 st pack of face photo data from the CPU card to a local cache of a card reader/writer;
after the card reader/reader saves the 1 st pack of face photo data to the local cache of the card reader/reader, continuing to read the 2 nd pack of face photo data from the CPU card to the local cache of the card reader/reader;
the elevator control device is also used for:
the card reader/reader reads the 2 nd pack of face photo data from the CPU card into a local cache, and simultaneously, the DMA is utilized to transmit the 1 st pack of face photo data from the local cache of the card reader/reader to the local cache of the elevator control device;
by analogy, while the card reader/writer reads the (n+1) th packet of face photo data from the CPU card into the local cache of the card reader/writer, the elevator control device transmits the (N) th packet of face photo data from the local cache of the card reader/writer into the local cache of the elevator control device by using DMA (direct memory access); and transmitting the (n+1) th pack of face photo data to the local buffer memory of the elevator control device by using the DMA (direct memory access) from the local buffer memory of the card reader/writer until the elevator control device finishes receiving the face information data sent by the card reader/writer.
7. An off-line elevator control system based on face recognition as defined in claim 5, wherein:
the elevator control device is also used for:
receiving user management information sent by the card reader/writer, wherein the user management information is read from the CPU card by the card reader/writer; the user management information comprises a card number of a CPU card, elevator riding authority, a deletion mark, a face information input mark and a validity period mark;
executing corresponding management operation according to the user management information;
the management operation includes:
detecting the face information input mark, and executing the step that the elevator control device receives the face information data sent by the card reader/writer when detecting that the face information input mark identifies the current user as a newly added person;
detecting the deletion mark, and deleting the user management information from a database when the deletion mark is detected to identify that the current CPU card is in a deletion state;
detecting the validity period mark, and canceling the elevator taking authority corresponding to the current CPU card when the validity period mark is detected to mark that the current CPU card is in a state exceeding the validity period;
And detecting the card number of the CPU, and canceling the elevator taking authority corresponding to the current CPU card when detecting that the card number of the CPU card belongs to the loss card number.
8. An off-line elevator control system based on face recognition according to claim 5 or 7, wherein the elevator control device is further configured to construct a doubly linked list loop structure, and each user management information is stored on a corresponding node in the doubly linked list loop structure.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111461989 | 2021-12-02 | ||
| CN2021114619892 | 2021-12-02 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114455405A CN114455405A (en) | 2022-05-10 |
| CN114455405B true CN114455405B (en) | 2023-11-24 |
Family
ID=81407252
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111624529.7A Active CN114455405B (en) | 2021-12-02 | 2021-12-23 | Off-line elevator control system and method based on face recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114455405B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2853728Y (en) * | 2005-11-25 | 2007-01-03 | 上海新时达电气有限公司 | Elevator intelligent management device using IC card |
| CN102799821A (en) * | 2012-07-11 | 2012-11-28 | 深圳市飞瑞斯科技有限公司 | Method for checking intelligent card and identity of card holder, and face identification identity checking device |
| CN106698118A (en) * | 2016-12-09 | 2017-05-24 | 成都焦点合安物联网有限公司 | Elevator control method and system based on face recognition |
| CN107934704A (en) * | 2017-11-30 | 2018-04-20 | 常熟安智生物识别技术有限公司 | A kind of terraced control system using recognition of face |
| CN108074303A (en) * | 2016-11-16 | 2018-05-25 | 常熟安智生物识别技术有限公司 | A kind of scheme that can expand recognition of face number of users |
| CN109118621A (en) * | 2018-07-24 | 2019-01-01 | 石数字技术成都有限公司 | The face registration system of recognition of face gate inhibition a kind of and application in access control |
| CN213338863U (en) * | 2020-11-17 | 2021-06-01 | 广州科升信息科技有限公司 | Offline authorization system of dynamic face access control |
| WO2021184203A1 (en) * | 2020-03-17 | 2021-09-23 | 江苏威尔曼科技有限公司 | Facial recognition-based smart elevator calling system |
-
2021
- 2021-12-23 CN CN202111624529.7A patent/CN114455405B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN2853728Y (en) * | 2005-11-25 | 2007-01-03 | 上海新时达电气有限公司 | Elevator intelligent management device using IC card |
| CN102799821A (en) * | 2012-07-11 | 2012-11-28 | 深圳市飞瑞斯科技有限公司 | Method for checking intelligent card and identity of card holder, and face identification identity checking device |
| CN108074303A (en) * | 2016-11-16 | 2018-05-25 | 常熟安智生物识别技术有限公司 | A kind of scheme that can expand recognition of face number of users |
| CN106698118A (en) * | 2016-12-09 | 2017-05-24 | 成都焦点合安物联网有限公司 | Elevator control method and system based on face recognition |
| CN107934704A (en) * | 2017-11-30 | 2018-04-20 | 常熟安智生物识别技术有限公司 | A kind of terraced control system using recognition of face |
| CN109118621A (en) * | 2018-07-24 | 2019-01-01 | 石数字技术成都有限公司 | The face registration system of recognition of face gate inhibition a kind of and application in access control |
| WO2021184203A1 (en) * | 2020-03-17 | 2021-09-23 | 江苏威尔曼科技有限公司 | Facial recognition-based smart elevator calling system |
| CN213338863U (en) * | 2020-11-17 | 2021-06-01 | 广州科升信息科技有限公司 | Offline authorization system of dynamic face access control |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114455405A (en) | 2022-05-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111243109B (en) | Riding ticket checking method, device and system based on face recognition and computer equipment | |
| CN103917727B (en) | Utilize the locking device adding strong security of iris image | |
| EP1953676B1 (en) | Personal authentication device | |
| EP3267365A1 (en) | Fingerprint recognition method and system capable of improving fingerprint recognition rate | |
| US20090258667A1 (en) | Function unlocking system, function unlocking method, and function unlocking program | |
| CN105261105A (en) | Safety access control method | |
| WO2020093612A1 (en) | Clock-in/out method and system based on smart wristband, and computer device | |
| JP2009527804A (en) | Distributed standoff ID verification compatible with multiple face recognition systems (FRS) | |
| CN108734092A (en) | Personage's authentication device | |
| US20190206171A1 (en) | Entry/exit management system and entry/exit management method | |
| JP2012067458A (en) | Passage management system and passage management method | |
| JP2006309490A (en) | Biological authentication system | |
| CN111368721B (en) | Identity recognition processing method and device, electronic equipment and storage medium | |
| JP4460265B2 (en) | Entrance / exit management device | |
| CN114455405B (en) | Off-line elevator control system and method based on face recognition | |
| CN207764866U (en) | intelligent access control system | |
| JP2003331323A (en) | Automatic gate system | |
| CN107301696A (en) | Dynamic access control system and dynamic access control method | |
| CN105427480A (en) | Teller machine based on image analysis | |
| CN204303055U (en) | Embedded intelligence door-lock controller | |
| JP4085198B2 (en) | Personal authentication device and personal authentication system | |
| JP2006099687A (en) | User authentication device | |
| JP2003242464A (en) | Ic card and management device using it | |
| WO2016208121A1 (en) | Entry/exit management device and entry/exit management method | |
| CN207725235U (en) | A kind of charging pile with fingerprint recognition and payment system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |