TWI790059B - Elevator interior space detection system - Google Patents

Abstract

An elevator interior space detection system includes a storage device, an image capturing device, and a processor. The storage device is configured to store a closed door-empty carriage image of an elevator and to store a door coordinate and a floor coordinate of the closed door-empty carriage image. The image capturing device is configured to capture a real-time image of the elevator. The processor is configured to obtain a plurality of closed door features and a plurality of real-time door features according to the door coordinate, the closed door-empty carriage image, and the real-time image; determine whether the elevator is in a half-closed state/a closed state or not according to the closed door features and the real-time door features; obtain a plurality of empty floor features and a plurality of real-time floor features according to the floor coordinate, the closed door-empty carriage image, and the real-time image; obtain a remaining capacity according to the empty floor features and the real-time floor features while the elevator is in the half-closed state/the closed state; the full capacity mode of the elevator is enabled when the remaining capacity is less than a full capacity threshold, so that the elevator is brought to the specified storey.

Description

Elevator loading space detection system

The present invention relates to an elevator technology, in particular to an elevator loading space detection system.

Today's buildings are mostly high-rise buildings and are all equipped with elevators. The elevators are used to carry people up and down floors. During peak hours (such as commuting time, commuting time or department store anniversary time), the elevator will be filled with a large number of passengers at one time. In this case, in order to save the time of the passengers in the elevator, the elevator can be set to go directly to the floor specified by the passengers in the elevator (eg, the full load mode of the elevator). Generally, the current load capacity of the elevator is used to judge whether to directly reach the designated floor (that is, to judge whether to trigger the full load mode of the elevator). However, in some situations, there may already be a large number of passengers in the elevator but the full load mode of the elevator cannot be triggered yet. For example, the weight of the passengers in the elevator is relatively light, so that the full load mode of the elevator cannot be triggered.

In view of the above, the present invention provides an elevator passenger space detection system. The elevator passenger space detection system includes a storage device, a camera device and a processor. The storage device is used for storing a door-closed and empty car image of an elevator and a door coordinate and a floor coordinate of the door-closed and empty car image. The camera device is used for shooting a real-time image of the elevator. The processor is used to obtain a plurality of door closing features according to the door coordinates and the image of the closed door and the empty car, wherein the door closing features are related to an elevator door of the elevator; according to the door coordinates and the real-time image, obtain a plurality of real-time door features, wherein the real-time door The feature is associated with the elevator door; according to these door-closing features and these real-time door features, it is judged whether the elevator is in a half-closed state or a door-closed state; according to the floor coordinates and the image of the empty car with the door closed, multiple empty floor features are obtained, among which The empty floor feature is associated with a floor of the elevator; according to the floor coordinates and real-time images, multiple real-time floor features are obtained, wherein these real-time floor features are associated with the floor; The floor features and the real-time floor features are used to obtain a remaining number of people; and when the remaining number of people is less than a full-load threshold, a full-load mode of the elevator is activated, so that the elevator directly reaches the floor designated by the user.

To sum up, according to some embodiments, the present invention can determine whether to trigger the full-load mode of the elevator through the image recognition technology. In this way, it can be ensured that when there are a large number of passengers in the elevator, the full-load mode of the elevator can be triggered and started.

Referring to FIG. 1 , it is a schematic block diagram of an elevator passenger space detection system 10 according to some embodiments of the present invention. The elevator passenger space detection system 10 includes a storage device 11 , a processor 13 and a camera device 15 . The processor 13 is electrically connected to the storage device 11 and the camera device 15 . The photographing device 15 is used to photograph an elevator to generate an image. The photographing device 15 can be arranged at the top corner of the wall opposite the elevator door in the elevator. The photographing device 15 photographs towards the elevator door. The lens of the photographing device 15 may be a wide-angle lens, for example, the viewing angle of the lens of the photographing device 15 may be 170 degrees. In this way, the images generated by the photographing device 15 can cover all areas inside the elevator. When the photographing device 15 photographs the elevator in real time, it generates a real-time image of the elevator. For example, when the elevator doors of the elevator are currently half-opened, closed, or the elevator has passengers, the real-time image shows half-open elevator doors, closed elevator doors, or human figures. The photography device 15 may be a camera, a video camera, or a device with a photography function. The storage device 11 can be a non-transitory storage medium (such as a traditional hard disk, a solid state disk, a flash memory, or an optical disk, etc.) or a transient storage medium (such as a random access memory). The processor 13 may be a computing circuit such as a central processing unit, a microprocessor, an application-specific integrated circuit (Application-specific Integrated Circuit, ASIC), or a system on a chip (System on a Chip, SOC).

The storage device 11 is used for storing a door-closed and empty car image 30 of the elevator and a door coordinate and a floor coordinate of the door-closed and empty car image 30 . For example, in the case that the elevator is an empty car (i.e. does not have any passengers) and the elevator door is closed, the designer can input a preprocessing command to the processor 13, and the processor 13 controls the camera 15 in response to the preprocessing command The elevator is photographed to generate an image 30 of the elevator with its doors closed and empty. That is to say, the door-closed empty car image 30 is an image in which the elevator is an empty car and the elevator doors are closed. The processor 13 performs image recognition on the closed door and empty car image 30 to obtain the door coordinates of the relevant elevator door and the floor coordinates of the relevant elevator floor (hereinafter referred to as the floor) of the closed door and empty car image 30 . For example, the door coordinates may include the coordinate points of the four corners of the elevator door, and the floor coordinates may include the coordinate points of the four corners of the floor. Then, the processor 13 stores the image 30 of the empty compartment with the door closed, the coordinates of the door and the coordinates of the floor into the storage device 11 .

In some embodiments, the closed door and empty car image 30 is obtained by performing dynamic average calculation on a plurality of closed door and empty car original images. For example, the processor 13 can control the photographing device 15 to take a plurality of original images of closed doors and empty cars of the elevator (that is, the photographing device 15 shoots an image of the elevator being an empty car and the elevator door is closed, and the image has not been subjected to any image processing), and The original images of the closed doors and empty carriages are subjected to dynamic average calculation to generate the closed doors and empty carriage images 30 . An application example is used for illustration. The processor 13 generates the image 30 of the closed and empty car after performing dynamic average calculation on the 25 original images of the closed door and empty car. In this way, the noise problem of the closed-door and empty-car image 30 can be reduced (for example, the influence of noise on the color quality of the closed-door and empty-car image 30 can be reduced), and the accuracy of the door coordinates and floor coordinates can be improved.

Referring to FIG. 2 , it is a schematic flowchart of an elevator passenger space detection method according to some embodiments of the present invention. The elevator passenger space detection method is suitable to be executed by the processor 13 . Firstly, the processor 13 obtains a plurality of door closing features according to the door coordinates and the images of the closed and empty cars (step S201 ), and obtains a plurality of real-time door features according to the door coordinates and the real-time images (step S203 ). Since the door-closed empty car image and the real-time image can be obtained by the same photographing device 15 shooting inside the elevator at a fixed position, the elevator door is presented at the same position in the door-closed empty car image and the real-time image. That is to say, the door-closed and empty compartment image and the real-time image have the same door coordinates. The door closing features and the real-time door features are associated with elevator doors of the elevator. For example, each door-closing feature can be the color, contour, and/or material of different areas of the elevator door in the closed door and empty car image, and each real-time door feature can be related to different areas of the elevator door in the real-time image. Color, outline, and/or texture.

After obtaining these door closing features and these real-time door features, the processor 13 judges whether the elevator is in a half-closed state (also called a half-open state) or a closed door state ( Step S205). For example, when the door-closing features are consistent with the real-time door features (for example, the door-closing features under the same elevator door area are all consistent with the real-time door features), the processor 13 determines that the elevator is in the door-closing state. When the door-closing features do not match the real-time door features (for example, the door-closing features under the same elevator door area do not match the real-time door features), the processor 13 judges that the elevator is in a fully open state. When at least one of the door-closing features does not match at least one of the real-time door features (for example, at least one door-closing feature under the same elevator door area does not match the real-time door feature), the processor 13 judges that the elevator is in half off state. When the processor 13 judges that the elevator is not in a half-closed state or a door-closed state (for example, when the processor 13 judges that the elevator is in a fully-opened state), it controls the photography device 15 to re-shoot a new real-time image, and returns to step S203 to obtain a new real-time image. The real-time gate feature of the image, and continue the subsequent steps. When the processor 13 judges that the elevator is in a half-closed state or a door-closed state, it executes steps S207-S215. Since the door-closing rate of the elevator is generally fast, steps S207-S215 are executed when the elevator is in the half-closed state or the door-closed state to ensure that the full-load mode can be triggered when the full-load mode is required. Furthermore, since the elevator is in the fully open state, it means that passengers are getting in and out of the elevator without triggering the full load mode, so by executing steps S207-S215 only when the elevator is in the half-closed state or the door-closed state, the computing resources of the processor 13 can be saved.

In step S207, the processor 13 obtains a plurality of empty floor features according to the floor coordinates and the image of the empty compartment with the door closed. In step S209, the processor 13 obtains a plurality of real-time floor features according to the floor coordinates and the real-time image. Since the image of the closed and empty car and the real-time image can be obtained by the same photographing device 15 shooting inside the elevator at a fixed position, the floor is presented at the same position in the image of the closed and empty car and the real-time image. That is to say, the image of the closed and empty compartment and the real-time image have the same floor coordinates. The empty floor features and the real-time floor features are associated with the floor of the elevator. For example, each empty floor feature may be the color, outline, and/or material of different areas of the floor in the closed-door and empty car image, and each real-time floor feature may be the color, contour, and/or material of different areas of the floor in the real-time image. profile, and/or texture.

When the empty floor features and the real-time floor features are obtained and the elevator is in a half-closed state or a door-closed state, the processor 13 obtains a remaining capacity according to the empty floor features and the real-time floor features (step S211) . For example, the processor 13 obtains the remaining capacity according to the number of the vacant floor characteristics matching the real-time floor characteristics. Specifically, the processor 13 calculates the remaining number of occupants according to the number of vacant floor features under the same floor area that match the real-time floor features. In some embodiments, the processor 13 may subtract the number of empty floor features that do not match the real-time floor features from the number of empty floor features or the number of real-time floor features to calculate the remaining capacity number of people. Wherein, the number of the empty floor features that do not match the real-time floor features may be the number of empty floor features under the same floor area that does not match the real-time floor features. In some embodiments, the processor 13 can match the empty floor features to the number of the real-time floor features (the number of the empty floor features or the number of the real-time floor features minus the number of the empty floor features The number corresponding to these real-time floor features) is divided by the number of these empty floor features or the number of these real-time floor features to calculate the remaining capacity.

In step S213, the processor 13 determines whether the remaining capacity is less than a full load threshold. The full load threshold can be pre-stored in the storage device 11 . When the remaining number of people is not less than the full load threshold, it means that the elevator does not carry a large number of passengers or the number of passengers in the elevator has not reached the upper limit, then the processor 13 controls the camera 15 to re-shoot a new real-time image, and returns to the execution step S203 Obtain the real-time gate feature of the new real-time image, and continue the subsequent steps. When the remaining capacity is less than the full-load threshold, it means that the elevator carries a large number of passengers or the number of passengers in the elevator has reached the upper limit, then the processor 13 starts the full-load mode of the elevator (step S215). In the full load mode, the processor 13 controls the elevator to go directly to the floor designated by the passengers (users) in the elevator.

Therefore, by performing image recognition (such as door feature extraction and floor feature extraction) on the closed door empty car image and the real-time image, the door feature and floor feature of the closed door empty car image and the real-time image are compared, and the remaining capacity is compared. The number of people and the full load threshold can ensure that the full load mode of the elevator can be triggered when the full load mode is required. That is to say, the problem that the full load mode of the elevator cannot be triggered when there are a large number of passengers in the elevator can be avoided.

In some embodiments, the order of step S201 and step S203 can be reversed. In some embodiments, the order of step S207 and step S209 can be reversed.

Refer to FIG. 3 and FIG. 4 . FIG. 3 is a schematic diagram of an image 30 of an empty car with a closed door according to some embodiments of the present invention. FIG. 4 is a schematic diagram of the first door image 31 according to some embodiments of the present invention. In some embodiments of step S201 , the processor 13 extracts a door image (hereinafter referred to as the first door image 31 ) related to the elevator door from the closed door and empty car image 30 according to the door coordinates. The processor 13 divides the first door image 31 into a plurality of first door blocks 33A˜33D. For convenience of illustration, only four first door blocks are drawn in FIG. 4 , but the present invention is not limited thereto. In some embodiments, the processor 13 divides the first door image 31 into a plurality of first door blocks 33A˜ 33D respectively including a plurality of pixels by a plurality of boundaries. The number of pixels included in each of the first door blocks 33A- 33D may be consistent, and each of the first door blocks 33A- 33D is defined in a different position in the first door image 31 and does not overlap with each other. In other words, a plurality of adjacent pixels are combined to form a first gate block. In some embodiments, each of the first gate blocks 33A- 33D includes a pixel matrix composed of a plurality of pixels (for example, a 3*3 pixel matrix, that is, a pixel matrix composed of 9 pixels).

Next, the processor 13 executes the door feature program on each of the first door blocks 33A-33D to obtain the door-closing features respectively corresponding to the first door blocks 33A-33D. In some embodiments, the door feature programs include a door color program, a door outline program, and a door material program. Each door closing feature includes a first door color feature, a first door outline feature and a first door material feature. Wherein, the first door color feature is the color presented by the corresponding first door block in the closed door empty car image 30, and the first door outline feature is presented by the corresponding first door block in the closed door empty car image 30 The contour, first door material feature is the material presented in the closed door empty car image 30 with respect to the corresponding first door block. Specifically, by executing the door color program on the first door blocks 33A~33D, the processor 13 can obtain the first door color feature; by executing the door outline program on the first door blocks 33A~33D, the processor 13 can obtain The first door outline feature; and by executing the door material program on the first door blocks 33A-33D, the processor 13 can obtain the first door material feature.

Refer to FIG. 5 and FIG. 6 . FIG. 5 is a schematic diagram of a real-time image 50 according to some embodiments of the present invention. FIG. 6 is a schematic diagram of a second door image 51 according to some embodiments of the present invention. Similar to step S201 , in some embodiments of step S203 , the processor 13 retrieves a door image (hereinafter referred to as a second door image 51 ) related to the elevator door from the real-time image 50 according to the door coordinates. The processor 13 divides the second door image 51 into a plurality of second door blocks 53A˜53D. For convenience of illustration, FIG. 6 only draws four second door blocks, but the present invention is not limited thereto. In some embodiments, the processor 13 divides the second door image 51 into a plurality of second door blocks 53A˜ 53D respectively including a plurality of pixels by a plurality of boundaries. The number of pixels included in each of the second gate blocks 53A-53D may be consistent, and each of the second gate blocks 53A-53D are defined at different positions in the second gate image 51 and do not overlap with each other. In other words, a plurality of adjacent pixels are combined to form a second gate block. In some embodiments, each of the second gate blocks 53A- 53D includes a pixel matrix composed of a plurality of pixels (for example, a 3*3 pixel matrix, that is, a pixel matrix composed of 9 pixels). In some embodiments, as shown in FIG. 4 and FIG. 6 , since the size of the first door image 31 and the second door image 51 can be the same, and distinguish the first door blocks 33A~33D from the second door blocks The manners of 53A-53D may be the same, so the positions of the first door blocks 33A-33D in the first door image 31 correspond to the positions of the second door blocks 53A-53D in the second door image 51 .

Next, the processor 13 executes the gate feature program on each of the second gate blocks 53A-53D to obtain the real-time gate features respectively corresponding to the second gate blocks 53A-53D. In some embodiments, each real-time door feature includes a second door color feature, a second door outline feature, and a second door material feature. Wherein, the second door color feature is the color presented in the real-time image 50 with respect to the corresponding second door block, the second door outline feature is the outline presented in the real-time image 50 with respect to the corresponding second door block, and the second The door material feature is the material presented in the real-time image 50 with respect to the corresponding second door block. Specifically, by executing the door color program on the second door blocks 53A~53D, the processor 13 can obtain the second door color feature; by executing the door outline program on the second door blocks 53A~53D, the processor 13 can obtain The second door profile feature; and by executing the door material program on the second door blocks 53A-53D, the processor 13 can obtain the second door material feature.

The aforementioned door color program can be that the processor 13 calculates the average brightness of each first door block 33A~33D as a feature of each first door color, and calculates the average brightness of each second door block 53A~53D Take as each second door color feature. The average brightness of each of the first gate blocks 33A-33D can be obtained by averaging the three primary color light values (RGB values) of the pixels in each of the first gate blocks 33A-33D. The average brightness of each second gate block 53A-53D can be obtained by averaging the RGB values of the pixels in each second gate block 53A-53D. In some embodiments, the processor 13 may perform blurring processing on the closed-door and empty-car image 30 and the real-time image 50 , so that the RGB value difference between adjacent pixels is small. In this way, the single first gate blocks 33A-33D and the single second gate blocks 53A-53D will not be affected by the different RGB values of the pixels therein and have large changes. That is to say, it is possible to improve the accuracy of comparing whether the door closing feature (such as the first door color feature) matches the real-time door feature (such as the second door color feature).

The aforementioned door contour program and door material program can be realized by neural network. For example, the designer can input a plurality of sample images of elevator doors to the processor 13 through an input/output interface (such as a keyboard, a mouse, an image transmission interface, etc.) (not shown in the figure). The processor 13 performs machine learning training for contour image recognition according to the sample images to determine a first judgment logic. Similarly, the processor 13 performs machine learning training for material image recognition according to the sample image to determine a second judgment logic. Wherein, the machine learning training for contour image recognition and the machine learning training for material image recognition can be known or self-developed models, the details of which are omitted here. The processor 13 can obtain the first door profile feature according to the first judgment logic and the first door blocks 33A~33D; the processor 13 can obtain the second door profile feature according to the first judgment logic and the second door blocks 53A~53D. Door outline feature. The processor 13 can obtain the material characteristics of the first door according to the second judgment logic and the first door blocks 33A~33D; the processor 13 can obtain the second door material characteristics according to the second judgment logic and the second door blocks 53A~53D. Door material characteristics.

As shown in FIG. 4 and FIG. 6, in some embodiments of step S205, the processor 13 bases the positions of the first door blocks 33A~33D on the first door image 31 and the positions of the second door blocks 53A~33D 53D at the position of the second door image 51, comparing the door closing features corresponding to each of the first door blocks 33A~33D and the real-time door features corresponding to each of the second door blocks 53A~53D at the same position If they match, calculate the pixel size of the second door block corresponding to each real-time door feature that does not match to generate a door gap area, and judge whether the elevator is in a half-closed state or a door-closed state based on the door gap area.

The second door image 51 presents a half-open elevator door as an example for illustration. Area 34 and area 35 in FIG. 4 are both images inside the elevator door. The area 54 in FIG. 6 is the outside image of the elevator door, and the area 55 is the inside image of the elevator door. The first door blocks 33A- 33C correspond to the same positions as the second door blocks 53A- 53C respectively, and the first door blocks 33A- 33C and the second door blocks 53A- 53C are respectively located in the area 35 and the area 55 . The first door block 33D and the second door block 53D correspond to the same position, and the first door block 33D and the second door block 53D are respectively located in the area 34 and the area 54 . It can be seen from Fig. 4 and Fig. 6 that since area 54 is an image outside the door and area 34, area 35, and area 55 are all images inside the door, the real-time door feature corresponding to each second door block in area 54 does not match the area. Each of the first door blocks in 34 corresponds to the door closing feature (these second door blocks are referred to as inconsistent second door blocks hereinafter). For example, the real-time door feature of the second door block 53D does not match the door-close feature of the first door block 33D. That is to say, due to the influence of the color, material and outline of the door, the real-time door feature values of the second door block 53D (for example, the value of the second door color feature, the second door outline feature and the second door material feature) value) is different from the value of the door closing feature of the first door block 33D (for example, the values of the first door color feature, the first door outline feature and the first door material feature). The real-time door feature corresponding to each second door block in the area 55 is consistent with the closing door feature corresponding to each first door block in the area 35 (hereinafter these second door blocks are referred to as matching second doors block). For example, the real-time door features of the second door blocks 53A-53C are consistent with the door-closing features of the first door blocks 33A-33C. That is to say, the real-time door feature values (for example, the second door color feature, the second door outline feature, and the second door material feature) of the second door blocks 53A-53C are not affected by the color, material, and outline of the door. The value of the feature) is the same as the value of the door closing feature of the first door blocks 33A-33C (for example, the values of the first door color feature, the first door outline feature and the first door material feature).

The area 54 formed by the inconsistent second door block indicates the size of the door gap opened by the elevator door, that is, the area 54 is the area of the door gap. Therefore, the processor 13 can obtain the door gap area by summing up the pixel sizes of each inconsistent second door block. When the area of the door gap is less than a door gap threshold, it means that the elevator door is half closed or closed (that is, the door gap opened by the elevator door is small or completely closed), and the processor 13 judges that the elevator is in a half closed state or a closed state . When the area of the door gap is not less than the threshold value of the door gap, it means that the elevator door is fully open (that is, the elevator door is opened with a large gap or fully opened), and the processor 13 judges that the elevator is in a fully open state. Wherein, the door gap threshold may be pre-stored in the storage device 11 .

In some embodiments, the processor 13 obtains a total door area according to the pixel size of the second door image 51 . For example, the processor 13 adds the size of each pixel of the second door image 51 to obtain the total area of the door. The processor 13 can obtain the remaining gate area (for example, the area 55 shown in FIG. 6 ) by summing the pixel size of each corresponding second gate block. The door gap area can be calculated by the processor 13 by subtracting the remaining door area from the total door area. In some embodiments, the processor 13 obtains a door state value according to the door gap area and the total door area. For example, the processor 13 divides the door gap area by the total door area to calculate the door state value. When the door state value is less than the door state threshold (for example, when the door state threshold is less than 0.8, it means that the elevator door is half-closed or closed), the processor 13 determines that the elevator is in a half-closed state or closed state. When the door state value is not less than the door state threshold (for example, when the door state threshold is not less than 0.8, it means that the elevator door is fully open), the processor 13 judges that the elevator is in the fully open state. The door state threshold can be pre-stored in the storage device 11 .

Referring to FIG. 7 , it is a schematic diagram of a second door image 51 according to some embodiments of the present invention. In some embodiments, the processor 13 determines the one-stop pedestrian area A1 of the second door image 51 . Because there may be passengers standing in front of the elevator door and blocking the image presented by part of the elevator door, in order to avoid affecting the judgment of the switch state of the elevator door when there are passengers standing in front of the elevator door (for example, to avoid the judgment of step S205 being affected), The processor 13 may perform the related actions of step S205 based on the area outside the standing area A1 (referred to as the remaining area A2 hereinafter). That is to say, step S205 may be performed only on the second door block located in the remaining area A2 and its corresponding first door block by excluding the second door block located in the pedestrian area A1 .

As shown in FIG. 7 , in some embodiments, the standing area A1 is an area between a standing human edge L2 and the bottom edge L1 of the image defined by an inward displacement of a distance from the bottom edge L1 of the second door image 51 . The bottom edge L1 of the image may correspond to the floor. For example, the bottom edge L1 of the image is displaced a distance from the floor toward the top of the elevator (eg, the exit panel of the elevator) (eg, the direction D1, ie, the bottom-to-top direction of the second door image 51 ) to define the standing edge L2. In some embodiments, the inward displacement distance of the bottom edge L1 of the image can be set according to the height of the passenger. In some embodiments, the processor 13 sequentially executes the step of obtaining the standing and occupant marking block along the first dimension (for example, the direction D2 , ie, the direction from left to right of the second door image 51 ). The step of obtaining the marked block of standing people may be that the processor 13 determines whether the door closing feature corresponding to the first door block matches the real-time door feature corresponding to the second door block along the second dimension (for example, direction D1). , and the first matching second door block judged along the second dimension in the second door image 51 is used as the standing person marking block. The processor 13 concatenates each stop person marking block sequentially obtained on the first dimension to form the outbound person edge L2. In some embodiments, in the station area A1, the real-time door feature corresponding to each second door block at the same position does not match the door closing feature corresponding to each first door block. For example, as shown in FIG. 4 and FIG. 7 , the second door blocks 53A-53C are located in the station area A1. Since the elevator doors are blocked by passengers in the station A1, the real-time door features of the first door blocks 33A-33C do not match the door-closing features of the second door blocks 53A-53C.

In some embodiments, the calculation of the door gap area is based on the second door block ( That is, it does not conform to the pixel size of the second gate block, such as the second gate block 53E in the outer gate area 541 in the remaining area A2 shown in FIG. 7 ). For example, the processor 13 calculates the door gap area (for example, the door outside area 541 ) by summing up the pixel sizes of each of the non-matching second door blocks in the remaining area A2 . In some embodiments, the total door area is calculated according to the pixel size of each second door block located outside the pedestrian area A1 (ie the remaining area A2 ) in the second door image 51 . For example, the processor 13 calculates the total gate area by summing the pixel size of each second gate block in the remaining area A2. The processor 13 may be to add the second gate block corresponding to each matching real-time door feature in the remaining area A2 (that is, the second gate block that matches the second gate block, for example, the door area in the remaining area A2 as shown in FIG. 7 The remaining door area (such as the door inner area 551 ) is calculated based on the pixel size of the second door block 53F in 551 , and the door gap area is calculated by subtracting the remaining door area from the total door area.

In some embodiments, the storage device 11 stores the compartment volume parameters. The car volume parameters include the upper limit of the elevator capacity and field parameters. The upper limit of the number of people allowed is the maximum number of people that the elevator can accommodate at one time within the safety specifications of the elevator. The field parameter is the area corresponding to the decorations in the elevator and the floor. For example, when decorations are placed on an area of the floor, the area may not be available for passengers to stand on, so the area of the area may be defined as a field parameter. In some embodiments of step S211, the processor 13 obtains the remaining number of occupants according to the empty floor features, the real-time floor features and the car volume parameters. That is to say, in addition to considering the empty floor features and the real-time floor features, the processor 13 further considers the compartment volume parameters. In this way, it is possible to ensure that the remaining number of people is accurate. For example, the remaining number of people is the number of passengers remaining in the elevator under the condition that all passengers in the elevator can successfully enter and stand successfully.

Refer to FIG. 3 and FIG. 8 . FIG. 8 is a schematic diagram of a first floor image 81 according to some embodiments of the present invention. In some embodiments of step S207 , the processor 13 retrieves a floor image (hereinafter referred to as the first floor image 81 ) related to the floor from the closed door and empty compartment image 30 according to the floor coordinates. The processor 13 divides the first floor image 81 into a plurality of first floor blocks 83A˜83D. For convenience of illustration, only four first floor blocks are drawn in FIG. 8 , but the present invention is not limited thereto. In some embodiments, the processor 13 divides the first floor image 81 into a plurality of first floor blocks 83A˜ 83D respectively including a plurality of pixels by a plurality of boundaries. The number of pixels included in each of the first floor blocks 83A-83D may be consistent, and each of the first floor blocks 83A-83D is defined at a different position in the first floor image 81 and does not overlap with each other. In other words, a plurality of adjacent pixels are combined to form a first floor block. In some embodiments, each of the first floor blocks 83A- 83D includes a pixel matrix composed of a plurality of pixels (for example, a 3*3 pixel matrix, that is, a pixel matrix composed of 9 pixels).

Next, the processor 13 executes the floor feature program on each of the first floor blocks 83A-83D, so as to obtain the empty floor features respectively corresponding to the first floor blocks 83A-83D. In some embodiments, the floor feature programs include a floor color program, a floor profile program, and a floor material program. Each empty floor feature includes a first floor color feature, a first floor profile feature and a first floor material feature. Wherein, the first floor color feature is the color presented in the closed door empty car image 30 with respect to the corresponding first floor block, and the first floor contour feature is presented in the closed door empty car image 30 with respect to the corresponding first floor block The profile, first floor material feature is the material present in the closed door empty car image 30 with respect to the corresponding first floor segment. Specifically, by executing the floor color program on the first floor blocks 83A~83D, the processor 13 can obtain the first floor color feature; by executing the floor contour program on the first floor blocks 83A~83D, the processor 13 can obtain The first floor profile feature; and by executing the floor material program on the first floor blocks 83A-83D, the processor 13 can obtain the first floor material feature.

Refer to FIG. 9 and FIG. 10 . FIG. 9 is a schematic diagram of a real-time image 90 according to some embodiments of the present invention. FIG. 10 is a schematic diagram of a second floor image 91 according to some embodiments of the present invention. Similar to step S207 , in some embodiments of step S209 , the processor 13 extracts a floor image (hereinafter referred to as a second floor image 91 ) related to the floor from the real-time image 90 according to the floor coordinates. The processor 13 divides the second floor image 91 into a plurality of second floor blocks 93A˜93D. For convenience of illustration, FIG. 10 only draws four second floor blocks, but the present invention is not limited thereto. In some embodiments, the processor 13 divides the second floor image 91 into a plurality of second floor blocks 93A˜ 93D respectively including a plurality of pixels by a plurality of boundaries. The number of pixels included in each of the second floor blocks 93A-93D may be consistent, and each of the second floor blocks 93A-93D is defined at a different position in the second floor image 91 and does not overlap with each other. In other words, a plurality of adjacent pixels are combined to form a second floor block. In some embodiments, each of the second floor blocks 93A˜ 93D includes a pixel matrix composed of a plurality of pixels (for example, a 3*3 pixel matrix, that is, a pixel matrix composed of 9 pixels). In some embodiments, as shown in FIG. 8 and FIG. 10 , since the size of the first floor image 81 and the second floor image 91 can be the same, and distinguish the first floor blocks 83A~83D from the second floor blocks The manners of 93A-93D may be the same, so the positions of the first floor blocks 83A-83D in the first floor image 81 correspond to the positions of the second floor blocks 93A-93D in the second floor image 91 .

Next, the processor 13 executes the floor feature program on each of the second floor blocks 93A-93D, so as to obtain the real-time floor features respectively corresponding to the second floor blocks 93A-93D. In some embodiments, each real-time floor feature includes a second floor color feature, a second floor profile feature, and a second floor material feature. Wherein, the second floor color feature is the color presented in the real-time image 90 with respect to the corresponding second floor block, the second floor outline feature is the contour presented in the real-time image 90 with respect to the corresponding second floor block, and the second The floor material feature is the material presented in the real-time image 90 with respect to the corresponding second floor segment. Specifically, by executing the floor color program on the second floor blocks 93A~93D, the processor 13 can obtain the second floor color feature; by executing the floor contour program on the second floor blocks 93A~93D, the processor 13 can obtain The second floor profile feature; and by executing the floor material program on the second floor blocks 93A-93D, the processor 13 can obtain the second floor material feature.

Similar to the door color program, the aforementioned floor color program can be that the processor 13 calculates the average brightness of each first floor block 83A~83D as the feature of each first floor color, and calculates the brightness value of each second floor block The brightness average value of 93A~93D is used as the characteristic of each second floor color. The average brightness of each first floor block 83A-83D can be obtained by averaging the RGB values of the pixels in each first floor block 83A-83D. The average brightness of each second floor block 93A-93D can be obtained by averaging the RGB values of the pixels in each second floor block 93A-93D. In some embodiments, after the image 30 of the closed and empty car and the real-time image 50 are blurred, the single first floor blocks 83A~83D and the single second floor blocks 93A~93D will not be affected by the pixels therein. There are large changes due to the influence of different RGB values.

Similar to the door profile program and the door material program, the floor profile program and the floor material program can be implemented by a neural network. For example, the processor 13 determines the third judgment logic related to the floor outline and the fourth judgment logic related to the floor material according to a plurality of sample images of the floor. The processor 13 can obtain the first floor contour feature and the second floor contour feature according to the third judgment logic, the first floor blocks 83A-83D and the second floor blocks 93A-93D. The processor 13 can obtain the first floor material feature and the second floor material feature according to the fourth judgment logic, the first floor blocks 83A-83D and the second floor blocks 93A-93D.

As shown in FIG. 8 and FIG. 10 , in some embodiments of step S211, the processor 13 bases the positions of the first floor blocks 83A~83D on the first floor image 81 and the positions of the second floor blocks 93A~83D 93D at the position of the second floor image 91, comparing the empty floor features corresponding to each of the first floor blocks 83A~83D and the real-time floor features corresponding to each of the second floor blocks 93A~93D at the same position Whether it matches or not, the pixel size of the second floor block corresponding to each matching real-time floor feature is calculated to generate a remaining floor area, and the remaining number of people can be obtained according to the remaining floor area and the car volume parameters.

The second floor image 91 presents a human figure as an example for illustration. As shown in FIG. 8 , the area 85 is an area outside the area 84 in the first floor image 81 , and neither the area 84 nor the area 85 has human figures. As shown in FIG. 10 , the area 95 is an area outside the area 94 in the second floor image 91 . Area 94 has a human figure and area 95 has no human figure. The first floor block 83D and the second floor block 93D correspond to the same position, and the first floor block 83D and the second floor block 93D are respectively located in the area 85 and the area 95 . The first floor blocks 83A-83C respectively correspond to the same positions as the second floor blocks 93A-93C, and the first floor blocks 83A-83C and the second floor blocks 93A-93C are located in the area 84 and the area 94 respectively.

It can be seen from FIG. 8 and FIG. 10 that since the area 94 has human figures and the areas 84, 85 and 95 do not have human figures, the real-time floor features corresponding to each second floor block in the area 94 do not match those in the area 84. The empty floor features corresponding to each of the first floor blocks (these second floor blocks are referred to as inconsistent second floor blocks hereinafter). For example, the real-time floor features of the second floor blocks 93A-93C do not match the empty floor features of the first floor blocks 83A-83C. That is to say, due to the influence of the color, material and outline of the silhouette, the values of the real-time floor features of the second floor blocks 93A-93C (for example, the second floor color feature, the second floor outline feature and the second floor material The value of the characteristic) is different from the value of the empty floor characteristic of the first floor blocks 83A-83C (eg, the values of the first floor color characteristic, the first floor contour characteristic and the first floor material characteristic). The real-time floor features corresponding to each second floor block in the area 95 are consistent with the empty floor features corresponding to each first floor block in the area 85 (these second floor blocks are referred to as matching second floor blocks hereinafter). floor block). For example, the real-time floor characteristics of the second floor block 93D match the empty floor characteristics of the first floor block 83D. That is to say, because it is not affected by the color, material and outline of the human figure, the real-time floor feature values of the second floor block 93D (for example, values of the second floor color feature, the second floor outline feature, and the second floor material feature) value) is the same as the value of the empty floor characteristic of the first floor block 83D (eg, the values of the first floor color characteristic, the first floor outline characteristic and the first floor material characteristic). The area 95 formed by matching the second floor block represents the area of the floor where no people stand on it, that is, the area 95 is the remaining floor area. Therefore, the processor 13 can obtain the remaining floor area by summing up the pixel size of each matching second floor block. Next, the processor 13 may multiply the remaining floor area by the compartment volume parameter to calculate the remaining number of people.

In some embodiments, the processor 13 obtains a total floor area according to the pixel size of the second floor image 91 . For example, the processor 13 adds the size of each pixel of the second floor door image to obtain the total floor area. The processor 13 can obtain the standing area (for example, the area 94 shown in FIG. 10 ) by summing the pixel size of each inconsistent second floor block. The remaining floor area can be calculated by the processor 13 by subtracting the standing area from the total floor area.

As shown in FIG. 9 , in some embodiments of step S211 , the processor 13 determines an important floor area 941 of the second floor image 91 and generates an area weight corresponding to the important floor area 941 . The critical floor area 941 may be a location where passengers are less inclined to stand on the floor. In other words, when there are many passengers in the elevator, the key floor area 941 is easier for passengers to stand on. For example, the key floor area 941 is at least a corner of the second floor image 91 and/or an area in the second floor image 91 close to the second door image 51 of the real-time image 90 . The area weight can be a headcount drop parameter to reduce the value of the remaining floor area. For example, the headcount drop parameter can be a small value. The processor 13 compares the real-time floor features and empty floor features corresponding to the important floor area 941 . When in the key floor area 941, the real-time floor feature corresponding to each second floor block under the same position does not match the empty floor feature corresponding to each second floor block, it means that the key floor area 941 has stood If there are passengers, the processor 13 combines the area weight with the remaining floor area to generate a remaining floor weighted area, and obtains the remaining number of people according to the remaining floor weighted area and the compartment volume parameters. In other words, the processor 13 further considers the area weights in addition to the parameters of the remaining floor area and the compartment volume. Since there are passengers standing in the key floor area 941, it means that there may be a large number of passengers in the elevator, so by considering the remaining floor area, the car volume parameters and the area weight together, it can be ensured that the full load mode can be triggered when the full load mode is required . In some embodiments, the processor 13 may calculate the remaining floor weighted area by multiplying the area weight by the remaining floor area, and multiply the remaining floor weighted area by the compartment volume parameter to calculate the remaining number of occupants.

In some embodiments, before judging whether the elevator is in a half-closed state or a door-closed state (step S205), the processor 13 judges whether a degree of difference between the door-closed and empty car image 30 and the real-time images 50, 90 reaches an abnormal threshold . The abnormal threshold can be stored in the storage device 11 . When the degree of difference does not reach the abnormal threshold, the processor 13 executes step S205 and its subsequent steps. When the degree of difference reaches the abnormal threshold value, it means that the real-time images 50 and 90 have abnormalities (for example, the real-time images have poor imaging results due to the unstable light source in the elevator), and the processor 13 controls the photographing device 15 to take new real-time images again. , return to step S203 to obtain the real-time gate features of the new real-time images 50, 90, and continue with the subsequent steps. In this way, subsequent steps (such as steps S205 - S215 ) are prevented from being affected by real-time images in abnormal states. In some embodiments, the degree of difference may be the image area outside the first door image 31 and the first floor image 81 in the closed door and empty compartment image and the image outside the second door image 51 and the second floor image 91 in the real-time image The degree of color difference, the degree of contour difference, and/or the degree of material difference between regions.

To sum up, according to some embodiments, the present invention can determine whether to trigger the full-load mode of the elevator through the image recognition technology. In this way, it can be ensured that when there are a large number of passengers in the elevator, the full-load mode of the elevator can be triggered and started.

10: Elevator loading space detection system 11: storage device 13: Processor 15: Photographic device 30: Image of closed door and empty carriage 31: First Door Image 33A~33D: The first gate block 34: area 35: area 50: Live video 51:Second door image 53A~53F: The second door block 54: area 541: Area outside the door 55: area 551: Area inside the door A1: Standing area A2: remaining area L1: Bottom edge of image L2: standing on the edge D1: Direction D2: Direction 81: The first floor image 83A~83D: The first floor block 84: area 85: area 90: Live video 91:Second floor image 93A~93D: The second floor block 94: area 941: key floor area 95: area S201~S215: steps

[ FIG. 1 ] is a schematic block diagram of an elevator passenger space detection system according to some embodiments of the present invention. [ FIG. 2 ] is a schematic flowchart of a method for detecting an elevator passenger space according to some embodiments of the present invention. [ FIG. 3 ] is a schematic diagram of an image of an empty car with the door closed according to some embodiments of the present invention. [ FIG. 4 ] is a schematic diagram of the first gate image according to some embodiments of the present invention. [ FIG. 5 ] is a schematic diagram of a real-time image according to some embodiments of the present invention. [ FIG. 6 ] is a schematic diagram of a second door image according to some embodiments of the present invention. [ FIG. 7 ] is a schematic diagram of a second door image according to some embodiments of the present invention. [ FIG. 8 ] is a schematic diagram of a first floor image according to some embodiments of the present invention. [ FIG. 9 ] is a schematic diagram of real-time images according to some embodiments of the present invention. [ FIG. 10 ] is a schematic diagram of a second floor image according to some embodiments of the present invention.

S201~S215: steps

Claims (10)

A detection system for an elevator loading space, comprising: a storage device for storing an image of an elevator with a door-closed and empty car and a door coordinate and a floor coordinate of the image of the door-closed and empty car; a photographing device for taking pictures of the elevator A real-time image; and a processor, used to: obtain a plurality of door closing features according to the door coordinates and the closed door empty car image, wherein each of the door closing features is associated with a different area of an elevator door of the elevator in the door closing The color, outline, or material presented in the empty car image; according to the door coordinates and the real-time image, multiple real-time door features are obtained, wherein each of the real-time door features is associated with a different area of the elevator door in the real-time image The presented color, outline, or material; according to the door closing features and the real-time door features, it is judged whether the elevator is in a half-closed state or a closed door state; according to the floor coordinates and the image of the closed empty car, multiple empty Floor features, wherein each of the empty floor features is related to the color, outline, or material of different areas of a floor of the elevator in the closed door empty car image; according to the floor coordinates and the real-time image, multiple real-time Floor features, wherein each of the real-time floor features is associated with the color, outline, or material of different areas of the floor in the real-time image; when the elevator is in the half-closed state or the door-closed state, according to these Empty floor features and the real-time floor features to obtain a remaining capacity; and When the remaining capacity is less than a full-load threshold, a full-load mode of the elevator is activated, so that the elevator directly reaches the floor designated by the user. The elevator passenger space detection system as described in Claim 1, wherein the processor extracts a first door image from the door-closed and empty car image according to the door coordinates, and divides the first door image into a plurality of first door images a door block, and generate the door closing features corresponding to the first door blocks; wherein, the processor extracts a second door image from the real-time image according to the door coordinates, and the second door image Divide into a plurality of second door blocks, and generate the real-time door features corresponding to the second door blocks. The elevator passenger space detection system as described in claim 2, wherein the processor is based on the positions of the first door blocks on the first door image and the second door blocks on the second door image position, compare whether the door closing feature corresponding to each of the first door blocks in the same position is consistent with the real-time door feature corresponding to each of the second door blocks, and each real-time door feature that does not match The pixel size of the second door block corresponding to the door feature is calculated to generate a door gap area, and it is judged whether the elevator is in the half-closed state or the door-closed state according to the door gap area. The elevator passenger space detection system as described in claim 3, wherein the processor obtains a total door area according to the pixel size of the second door image, and obtains a door state according to the door gap area and the total door area Value, when the door state value is less than a door state threshold, the processor judges that the elevator is in the half-closed state or the closed door state. The elevator passenger space detection system as described in claim 4, wherein, the processor determines the passenger area of a station of the second door image, and the calculation of the area of the door gap is based on the area located at the station in the second door image The second door block corresponding to each inconsistent real-time door feature outside the human zone pixel size, the total door area is calculated based on the pixel size of each second door block outside the standing area in the second door image, and the standing area is the second door image An area between the bottom edge of a standing person and the bottom edge of the image defined by an inward displacement of a distance from the bottom edge of an image, and in the standing person area, each of the second door blocks corresponding to the same position The real-time door feature does not match the door closing feature corresponding to each of the first door blocks. The elevator passenger space detection system as described in claim 1, wherein, the storage device stores a car volume parameter, the car volume parameter includes an upper limit of the number of people that can be accommodated and a field parameter of the elevator, and the processor is based on these The remaining number of occupants is obtained based on the empty floor features, the real-time floor features and the compartment volume parameters. The elevator passenger space detection system according to claim 6, wherein the processor extracts a first floor image from the closed door and empty car image according to the floor coordinates, and divides the first floor image into a plurality of first floor images a floor block, and generate the empty floor features corresponding to the first floor blocks; wherein, the processor extracts a second floor image from the real-time image according to the floor coordinates, and the second floor The image area is divided into a plurality of second floor blocks, and the real-time floor features corresponding to the second floor blocks are generated. The elevator passenger space detection system according to claim 7, wherein the processor is based on the positions of the first floor blocks on the first floor image and the second floor blocks on the second floor image position, compare whether the empty floor feature corresponding to each of the first floor blocks under the same position is consistent with the real-time floor feature corresponding to each of the second floor blocks, and match each of the The pixel size of the second floor block corresponding to the real-time floor feature is calculated to generate a remaining floor area, and the remaining number of people is obtained according to the remaining floor area and the compartment volume parameter. The elevator passenger space detection system as described in claim 8, wherein the processor determines an important floor area of the second floor image, and generates an area weight corresponding to the important floor area, when in the important floor area , when the real-time floor features corresponding to each of the second floor blocks at the same position do not match the empty floor features corresponding to each of the second floor blocks, the processor combines the area weight with the The remaining floor area is combined to generate a remaining floor weighted area, and the remaining number of people is obtained according to the remaining floor weighted area and the compartment volume parameter, wherein the key floor area is at least one corner of the second floor image or the second floor image An area of a second door image in the second floor image that is close to the real-time image. The elevator passenger space detection system as described in claim 1, wherein, when a difference degree between the closed door empty car image and the real-time image reaches an abnormal threshold, the processor stops judging whether the elevator is in the half closed state or the closed state.

Images