WO2016110866A1

WO2016110866A1 - Intelligent elevator management system using image processing

Info

Publication number: WO2016110866A1
Application number: PCT/IN2015/000433
Authority: WO
Inventors: Sai Narayanan Hari; Karunamurthy Vignesh; Barath Kumar Ramaswami
Original assignee: Sai Narayanan Hari
Priority date: 2015-01-07
Filing date: 2015-11-23
Publication date: 2016-07-14
Also published as: WO2016110866A8

Abstract

The present invention discloses an intelligent elevator management system using image processing. The system involves (a) providing plurality of call button means in the landing area of each floor,(b) maintaining a data structure in a data ware house that representing plurality of comparison tables and proximity modules, (c) providing at least one image acquisition means at a strategic position inside the elevator and on the landing area of each floor (d) providing image processing means in the data ware house to processing the images received from image acquisition means. (e) comparing the data received from the image processing means with comparison table and proximity module in the data ware house (f) operating the proximate and suitable elevator to move to the respective floor with the instruction from the data ware house thereby minimizing the waiting time and power usage.

Description

TITLE: INTELLIGENT ELEVATOR MANAGEMENT SYSTEM

USING IMAGE PROCESSI G

APPLICANT NAME: HARI SAI NARAYANAN

PRIORITY APPLICATION NUMBER: 126/C HE/2015 DATED 07/01 /2015

FIELD OF THE INVENTION:

The present invention relates to an elevator management system. More particularly the present invention relates to a method to facilitate intelligent elevator management system using image processing.

BACK GROUND OF THE INVENTION AND PRIOR ART:

The dramatic increase in the number of multi storey buildings have led to an increasing need for the development of fast and efficient elevator systems that can cater to industrial as well as commercial needs. In this modern world, time and power are the two precious parameters. Most of the elevator systems that are in use today focus on minimizing waiting time and maximize the speed of operation. Certain elevator systems used in industrial complexes have authorization modules and have a focus on security. The elevator management systems that are currently in use today function and operate based on proximity and their primary focus is on speed of operation. In a scenario where there are multiple elevators servicing a floor, the elevator with the closest proximity is brought down or up to service^ the user in a particular floor. This method hence is centred onthe speed of operation and hence is aimed at bringing down the waiting time for an elevator. Other elevator management systems include the use of RFID tags to authorize persons for respective floors. Hence the focus of these elevator systems are security and the present day trend is in reducing the travel time of elevators. Considering a scenario wherein the security needs can be negated, most of these elevator management systems do not take into account the number of persons waiting to be serviced and hence even though they monitor the weight limits, they do not make proper use of them.

The following are the types of elevator systems that are currently in use.

RFID tag based system for intelligent control in elevators.

Claire Swedberg has already proposed that RFID raises elevator efficiency. In the RFID tag based system every person who is able to operate the elevator is given a unique RFID tag. The purpose of this tag is to offer security and authorize that respective person using that RFID tag. For example, consider a multi storey building having 50 plus floors and more than 15 companies working in that building. The RFID tag restricts the person who is using it to certain areas in the building like the canteen and his company's office but restricts access to other floors of other companies. Thus this method's primary focus is on security and authorization. Although it has certain power saving aspects like proximity sensing and operation based on it, the system does not focus on weight management. Although all the elevator manufacturers include weight measuring and sensing as a part of the elevator system, these elevators do not have proactive overload sensing systems. The disadvantage of the above system can be brought to light by a suitable example.

Consider an elevator that is present at floor x having y persons in it. If we consider the maximum permissible weight of that particular elevator to be z kilograms, then the number of persons who can be accommodated in that elevator within the overload limit is z-w kilograms where w is taken as the average weight of y persons. Thus in a case where the elevator has to come down from floor x to another floor and if the average weight of the persons waiting for the elevator in another floor exceed z-w, then there is no point in bringing down the elevator to that floor because the elevator will be issuing an overload warning the moment the persons waiting get inside. This not only wastes time but also wastes considerable energy and causes irritation among the persons who wait. The problem is that not everyone will be serviced and a few people may be left out and will have to wait for a different elevator to come and pick them up. This problem does not have a solution as of now and this is one of the focus points of this invention.

REFERENCES: [1] Claire Swedberg "RFID raises elevator efficiency" RFID journal, Oct-05 201 1.

[2] Yu Li. Research and Development of Elevator Remote Monitor System. Tianjin:

Tianjin University. 2007.

[3] S. Lim and A. El Gamal, "Integration of image capture and processing: beyond single chip digital camera," in Sensors and Camera Systems for Scientific,

Industrial, and Digital Photography Applications II, vol. 4306 of of SPIE, pp. 219-

226, Jan. 2001 .

[4] "Digital Image Processing"; R. C. Gonzalez and R. E. Woods; Addison. Wesley; 1992.

[5] Patent by Wilmer G.Radke on "Automatic elevator control system", Dec-08

1981 , Publication number-US4304319A

[6] Hong Sha, Liu Jiachen, Fan Linyu, Luo Yi, "Intelligent Monitoring and Energy

Saving Analysis of Elevator,"

[7] Journal of Computer Applications, 2010, vol. 30, pp. 278-280.

OBJECT OF THE INVENTION:

The object of the present invention is to develop an intelligent elevator management system.

Another object of the present invention is to develop a method to facilitate intelligent elevator management system which primarily focusing on weight limit aspect of the elevator and reduction in waiting time.

Yet another object of the present invention is to utilize image processing technique in the intelligent elevator management system.

Further object of the present invention is to construct an intelligent elevator management system that can effectively govern the movement of elevator to minimize waiting time and power usage.

BRIEF DESCRIPTION OF DRAWINGS:

Figure 1 depicts the flowchart picturing the flow of operation in the proposed technique. Figure 2 depicts (a) rgb image (b): black and white image (c): Resultant image using canny operator. SUMMARY OF THE INVENTION:

The present invention discloses an intelligent elevator management system using image processing. The system involves (a) providing plurality of call button means in the landing area of each floor,(b) maintaining a data structure in a data ware house that representing plurality of comparison tables and proximity modules, (c) providing atleast one image acquisition means at a strategic position inside the elevator and on the landing area of each floor (d) providing image processing means in the data ware house to processing the images received from image acquisition means. (e) comparing the data received from the image processing means with comparison table and proximity module in the data ware house (f) operating the proximate and suitable elevator to move to the respective floor with the instruction from the data ware house thereby minimizing the waiting time and power usage.

DETAILED DESCRIPTION OF THE INVENTION:

The present invention discloses an intelligent elevator management system using image processing. The technique in this invention primarily focuses on the weight limit aspect of an elevator and is aimed at avoiding the wastage time that is spent in cases where the weight limit is reached and the passengers have to wait an extra period before they are serviced. To accomplish the above mentioned parameters an automated technique is needed to effectively control and regulate the elevator system. Hence there have been several attempts to design automated elevator system that can regularise the elevator lines. An intelligent elevator management system should acquire real time images and process it so that it can effectively govern the movement of elevator to minimize waiting time and power usage.

The flow of operation in the proposed technique is shown in Figurel . The details of each operation are given below.

Call button Event

The intelligent elevator system continuously waits for a call button event and monitors the call button switch on different floors.. The idea of having separate image processing modules for each floor is ruled out because of the price of installing the modules. However-when a-single-module is used for a^_b^itdi^'n^"g^"hav^'rn^"g^~mc eniumt^r^" of floors or levels, the cost of wiring that runs from each floor will be high and will introduce delay problems. Hence a suitable trade-off is to be made when considering this issue and it is entirely dependent on the user.

The call button event follows a first come first served basis priority system that can be implemented by a FIFO queue. The call button event that happens first is put in the queue first and the person who has activated that call button is served first. Considering the fact that elevators are not used in emergencies like fire, seismic activity etc... , there may be several instances of high priority usage. In these cases a separate emergency high priority button can be provided for select floors. These high priority elevators can be managed by the same hardware but a separate queue can be used that is assigned the highest priority.

Image Acquisition

A video camera feed continuously records the activities that happen in the elevator corridor. This camera is placed in a strategic position that not only monitors the call button but also a part of the elevator corridor. The main reason for this placement is to cover area where people are most likely to stand and wait for an elevator. Snapshots can be taken from this live feed when a call button is pressed.

Since the event manager follows a FIFO system priority is given to the first call button press and all other events are placed after the first event. However, when the whole process is delayed the accuracy of the images captured may be advantageous. Consider the case where two events are. detected at two different floors. The second event that happens is forced to wait in the FIFO queue. If the whole process is made to wait then the window the FIFO queue forces the event to wait can be used to advantage. More people may come and wait and sometimes people leave. Thus it is better to start the process after a suitable clear signal is sent from the event handler.

The camera used in the corridors for image acquisition can be simple VGA cameras that can take snaps at 640x480 resolutions and capture video at 30 frames per second. The reason for the lesser resolution is reflected on the simplicity of hardware required to process the images and also the lesser time required for executionnDHhes^imagesT

Image Processing

There will be a video camera running covering the corridor of the elevator, every time the elevator call button is pressed the video camera captures the image of corridor thus the people waiting for the elevator. There are cases wherein after the call button is pressed certain people arrive late and hence the accuracy of the camera in capturing the number of persons in the image. Thus to overcome this drawback, the camera is made to capture multiple images at fixed delays and makes use of the latest image, i.e. the last known image. By using edge detection method in image processing we can edge out people waiting in the corridor for the elevator thus finding out number people before the elevator. With the help of the comparison table and proximity module we can compare the load of the elevator and capacity and bring in nearest elevator thus saving time and reduce power consumption.

Comparison

The final output obtained from the image processing module is a value that corresponds to the number of white pixels that are present in the image. This is a direct measure of the number of edge pixels that correlate to the number of persons in the image. This is done by creating a table with approximate values that are decided based on field calibration and is dependent on the area where the module is deployed.

Elevator operation

The elevator operation can be explained based on the proximity system or module that takes into account the proximity of the user operating the elevator and also weight constraints.

The elevator system is assumed to be consisting of 3 elevators namely L1 , L2 and L3. These elevators are made to cover 4 floors which are named as F0, F1 , F2 and F3. Each elevator has a maximum capacity of 10 persons. The various modules of the algorithm contain the occupancy of the elevators along with its respective position. It also comprises the detail of the floor where the call button is pushed and the number of people in that floor. Keywords

Lift 1 - L1

Lift 2- L2

Lift 3- L3

Ground Floor- FO

First Floor- F1

Second Floor- F2

Third Floor- F3

CF- Floor where the elevator call button is pushed

Expansions

Case 1 :

L1 : 7(F2) L2:4(F1 ) L3:10(F3)

CF: 2(F0)

Lift 1 is at second floor with 7 persons. Lift 2 is at first floor with 4 persons. Lift 3 is at third floor with 10 persons. The elevator call button is pressed at ground floor and it consists of 2 persons.

The elevators L1 and L3 are far from ground floor so they aren't called. In this case calling lift 2 is the most efficient because first floor is the nearest to ground floor and it also has the capacity to take in 2 more people.

Inference: L2 is called.

Case 2:

L1 : 5(F0) L2:7(F2) L3:9(F3)

CF: 4(F1 )

Lift 1 is at ground floor with 5 persons. Lift 2 is at second floor with 7 persons. Lift 3 is at third floor with 9 persons. The elevator call button is pressed at first floor and it consists of 4 persons.

The elevator L2 being at second floor is nearest from first floor but since it already consists of 7 persons it can't take in 4 more people because it will exceed the maximum capacity of 10. So L2 isn't called. L3 is far from first floor so it isn't called. In this case calling lift 1 is the most efficient because first floor is the nearest to ground floor and it also has the capacity to take in 4 more people.

Inference: L1 is called.

Case 3:

L1 : 10(F1 ) L2:8(F2) L3:6(F3)

CF: 1 (F0)

Lift 1 is at first floor with 10 persons. Lift 2 is at second floor with 8 persons. Lift 3 is at third floor with 6 persons.

The elevator call button is pressed at ground floor and it consists of 1 person.

The elevator L1 being at first floor is nearest to ground floor but since it already consists of 10 persons it can't take in 1 more person because it will exceed the maximum capacity of 10. So L1 isn't called. L3 is far from ground floor so it isn't called. In this case calling lift 2 is the most efficient because it has the capacity to take in 1 more person.

Inference: L2 is called.

Case 4:

L1 : 4(F0) L2:6(F1 ) L3:2(F2)

CF: 5(F3)

Lift 1 is at ground floor with 4 persons. Lift 2^'is at first floor with 6 persons. Lift 3 is at second floor with 2 persons. The elevator call button is pressed at third floor and it consists of 5 persons.

The elevator L1 being at ground floor is farthest to third floor. So L1 isn't called. L2 is also far from third floor so it isn't called. In this case calling lift 3 is the most efficient because it is the nearest to third floor and also has the capacity to fit in 5 more people.

Inference: L3 is called.

Case 5:

L1 : 7(F2) L2:8(F1) L3:6(F1 )

CF: 3(F3) Lift 1 is at second floor with 7 persons. Lift 2 is at first floor with 8 persons. Lift 3 is also at first floor with 6 persons. The elevator call button is pressed at third floor and it consists of 3 persons.

The elevators L2 and L3 are at first floor it is farther to third floor. So L2 and L3 aren't called. In this case calling lift 1 is the most efficient because it is the nearest to third floor and also has the capacity to fit in 3 more people.

Inference: L1 is called.

Case 6:

L1 : 2(F3) L2:8(F1 ) L3:6(F0)

CF: 6(F2)

Lift 1 is at third floor with 2 persons. Lift 2 is at first floor with 8 persons. Lift 3 is at ground floor with 6 persons. The elevator call button is pressed at second floor and it consists of 6 persons.

The elevator L3 being at ground floor is farthest to second floor. So L3 isn't called. L2 is near second floor but it can't take in 6 persons because it will exceed its capacity limit of 10 so it isn't called. In this case calling lift 1 is the most efficient because it is the nearest to third floor and also has the capacity to fit in 6 more people.

Inference: L1 is called.

Case 7:

L1 : 6(F2) L2:7(F1) L3:5(F0)

CF: 5(F3)

Lift 1 is at second floor with 6 persons. Lift 2 is at first floor with 7 persons. Lift 3 is at ground floor with 5 persons. The elevator call button is pressed at third floor and it consists of 5 persons.

The elevator L2 being at first floor is farthest to third floor. So L2 isn't called. L2 is near to third floor when compared to L1 but it will exceed its maximum capacity if it takes in 5 more people. In this case calling lift 3 is the most efficient because it has the capacity to fit in 5 more people.

Inference: L3 is called. Case 8:

L1 : 9(F1) L2:7(F0) L3:4(F3)

CF: 2(F2)

Lift 1 is at first floor with 9 persons. Lift 2 is at ground floor with 7 persons. Lift 3 is at third floor with 4 persons. The elevator call button is pressed at second floor and it consists of 2 persons.

The elevators L1 being at first floor is near to second floor but it can't fit in 2 more people because of exceeding maximum capacity. So L1 isn't called. L2 is far from second floor so it isn't called. In this case calling lift 3 is the most efficient because it is the nearest to second floor and also has the capacity to fit in 2 more people.

Inference: L3 is called.

Case 9:

L1 : 10(F0) L2:8(F2) L3:9(F3)

CF: 2(F1 )

Lift 1 is at ground floor with 10 persons. Lift 2 is at second floor with 8 persons. Lift 3 is at third floor with 9 persons. The elevator call button is pressed at first floor and it consists of 2 persons.

The elevator L1 being near to first floor can't fit in 2 more people because of exceeding maximum capacity .So L1 isn't called. L3 is also far from first floor so it isn't called.

In this case calling lift 2 is the most efficient because it is the nearest to first floor and also has the capacity to fit in 2 more people.

Inference: L2 is called.

Case 10:

L1 : 6(F2) L2:6(F1 ) L3:3(F0)

CF: 4(F3)

Lift 1 is at second floor with 6 persons. Lift 2 is at first floor with 6 persons. Lift 3 is at ground floor with 3 persons. The elevator call button is pressed at third floor and it consists of 4 persons. The elevator L2 and L3 are farthest to third floor. So both aren't called. In this case calling lift 1 is the most efficient because it is the nearest to third floor and also has the capacity to fit in 4 more people.

Inference: L1 is called.

REAL TIME ANALYSIS

The image processing module can be explained step by step by taking the help of the images captured. Below steps depicts how exactly this intelligent elevator system will function. The last stage in the elevator management system is image processing and using the processed image to identify the elevator to be brought down. Image processing here refers to the identification of the number of persons in the waiting area. This is done using a straight forward approach. Edge detection technique is used to find the number of persons waiting for an elevator. The rgb image taken from the VGA camera is first converted into grayscale format. The grayscale image is fed as input to the canny operator. The canny operator is an inbuilt function in MATLAB that traces out the edges. The number of edge pixels that are traced can be calculated and an approximation can be made to find out the number of persons waiting. This however requires precise calibration as real time scenarios are always different. The calibration step is done at regular intervals and a table is created for reference. The table is also updated at regular intervals to reduce the errors in identification process.

The resultant images at various stages are shown in Figure 2. The first image Figure 2(a) rgb image shows the acquired image from the camera placed in the elevator waiting area. The second image Figure 2(b) black and white image shows the grayscale image that is used for edge detection. The final image Figure 2(c) is the resultant image that is got after the canny operation.

The number of pixels that do not correspond to black values are counted. This value is normalised and tabulated. An approximation table can be formed as below.

Table 1 APPROXIMATION TABLE USED FOR PERSON COUNT: Number of non-zero Normalised count value Approximated number of pixels in the resultant persons

image

430135 0.0568 1-2

520728 0.0687 2-3

532433 0.0708 3-4

Table 1 : Table containing a list of approximated values based on the edge pixel count. This is a calibration step and varies from place to place.

Table 1 .shows the number of non-zero pixels in the final canny image, the normalised count value and the approximated number of persons in the image. The above table gives an approximate evaluation of the number of persons waiting for an elevator. The number of non-zero pixels in the table depends on the number of persons waiting and also the number of people in the background. Hence proper calibration of the camera system is vital for efficient functioning of the system. This can be accomplished by strategic placement of the camera that is used for image acquisition. Also the number of pixels is dependent on the size of the image and hence we use the normalised values instead of the original pixel values to avoid confusion and maintain an operating standard.

Thus an improved elevator management system has been introduced, that offers more flexibility and takes additional parameters like weight and proximity to decide on the optimum elevator to initialise and service the required persons waiting in different floors. The algorithm can be extended to adapt to various scenarios like malls and other industrial cases too. This technique can also be implemented using low cost hardware that makes it a suitable economic option to be employed for real time usage.

In one of the preferred embodiment the present invention shall discloses an intelligent, economical and real time method to facilitate elevator management system of a building having plurality of elevators, and to effectively govern the movement Of elevator by focusing weight limit aspect and power usage thereby eliminating the prolonged waiting time and power wastage. The method -involves (a) providing plurality of call button means in the landing area of each floor for signaling the elevator to reach the respective floor,(b) maintaining a data structure in a data ware house that representing plurality of comparison tables and proximity modules to calculate the load and capacity of the elevator, to calculate the quantity of persons waiting in the landing area and to direct the proximate elevator with suitable capacity to reach the respective floor where the call button is activated, (c) providing atleast one image acquisition means at a strategic position inside the elevator and on the landing area of each floor to capture the images inside the elevator and on the landing area at delayed intervals after receiving signal from call button (d) providing image processing means in the data ware house to processing the images received from image acquisition means by making use of last known image to detect the accurate quantity of persons inside the elevator and waiting for the elevator in the landing area of the respective floor from where the call button is activated. (e) comparing the data received from the image processing means with comparison table and proximity module in the data ware house to calculate the load capacity of all the elevators, quantity of waiting person and directing the proximate elevator having suitable capacity to occupy all the waiting person, to reach the respective floor from where the call button signal is received (f) operating the proximate and suitable elevator to move to the respective floor with the instruction from the data ware house thereby minimizing the waiting time and power usage.

As per the invention the call button means follows a first come first serve basis priority system.

According to the invention the call button means also includes a separate emergency high priority button for use in top priority situations.

In accordance with the invention image acquisition means includes simple VGA cameras that can take snaps at 640X480 resolution and capture video at 30 frames per second thereby making the method cost effective.

As per the invention the image processing means employs edge detection technique to process the image According to the invention the data structure is calibrated at regular intervals to reduce the errors in identification process.

From the foregoing it will be appreciated that, although specific embodiments of the invention have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention. Accordingly, the invention is not limited except as by the appended claims.

Claims

I claim,

1. An intelligent, economical and ^' real time method that facilitate elevator management system of a building having plurality of elevators, to effectively govern the movement of elevator by focusing weight limit aspect and power usage thereby eliminating the prolonged waiting time and power wastage, the said method comprises of

a. Providing plurality of call button means in the landing area of each floor for signaling the elevator to reach the respective floor

b. maintaining a data structure in a data ware house that representing plurality of comparison tables and proximity modules to calculate the load and capacity of the elevator, to calculate the quantity of persons waiting in the landing area and to direct the proximate elevator with suitable capacity to reach the respective floor where the call button is activated c. Providing atleast one image acquisition means at a strategic position inside the elevator and on the landing area of each floor to capture the images inside the elevator and on the landing area at delayed intervals after receiving signal from call button.

d. Providing image processing means in the data ware house to processing the images received from image acquisition means by making use of last known image to detect the accurate quantity of persons inside the elevator and waiting for the elevator in the landing area of the respective floor from where the call button is activated.

e. Comparing the data received from the image processing means with comparison table and proximity module in the data ware house to calculate the load capacity of all the elevators, quantity of waiting person and directing the proximate elevator having suitable capacity to occupy all the waiting person, to reach the respective floor from where the call button signal is received f. Operating the proximate and suitable elevator to move to the respective floor with the instruction from the data ware house thereby minimizing the waiting time and power usage.

2. The method as claimed in claim 1 wherein the said call button means follows a first come first serve basis priority system.

3. The method as claimed in claim 1 wherein the call button means also includes a separate emergency high priority button for use in top priority situations.

4. The method as claimed in claim 1 where in the said image acquisition means includes simple VGA cameras that can take snaps at 640X480 resolution and capture video at 30 frames per second thereby making the method cost effective.

5. The method as claimed in claim 1 wherein the said image processing means employs edge detection technique to process the image

6. The method as claimed in claim 1 wherein the said data structure is calibrated at regular intervals to reduce the errors in identification process