CN109353907B - Safety prompting method and system for elevator operation - Google Patents

Safety prompting method and system for elevator operation Download PDF

Info

Publication number
CN109353907B
CN109353907B CN201710792185.8A CN201710792185A CN109353907B CN 109353907 B CN109353907 B CN 109353907B CN 201710792185 A CN201710792185 A CN 201710792185A CN 109353907 B CN109353907 B CN 109353907B
Authority
CN
China
Prior art keywords
area
image
elevator
depth
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710792185.8A
Other languages
Chinese (zh)
Other versions
CN109353907A (en
Inventor
陈刚
黄立明
仲兆峰
李斌
陈成
刘贤钊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi Building Technology Guangzhou Co Ltd
Original Assignee
Hitachi Building Technology Guangzhou Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Building Technology Guangzhou Co Ltd filed Critical Hitachi Building Technology Guangzhou Co Ltd
Priority to CN201710792185.8A priority Critical patent/CN109353907B/en
Publication of CN109353907A publication Critical patent/CN109353907A/en
Application granted granted Critical
Publication of CN109353907B publication Critical patent/CN109353907B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
    • B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
    • B66B5/0006—Monitoring devices or performance analysers
    • B66B5/0018—Devices monitoring the operating condition of the elevator system
    • B66B5/0031—Devices monitoring the operating condition of the elevator system for safety reasons
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
    • B66B3/00—Applications of devices for indicating or signalling operating conditions of elevators
    • B66B3/002—Indicators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B66—HOISTING; LIFTING; HAULING
    • B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
    • B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
    • B66B5/02—Applications of checking, fault-correcting, or safety devices in elevators responsive to abnormal operating conditions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Indicating And Signalling Devices For Elevators (AREA)
  • Elevator Door Apparatuses (AREA)

Abstract

The invention relates to a safety prompting method and a safety prompting system for elevator running, wherein a depth image of an elevator use area is obtained, when image information of an identification object exists in the depth image, whether the identification object is in a specified state is judged according to the image information, and when the identification object is not in the specified state and a car door of an elevator is in an action state or an opening state, safety prompting information representing the state of paying attention to the elevator running is sent. In the scheme, whether the identification object in the elevator use area is in the designated state or not can be known through the identification analysis processing of the depth image of the elevator use area, if the identification object is not in the designated state, the identification object can be prompted to pay attention to the elevator running state when a car door of the elevator is in an action or opening state, so that the attention of the identification object is focused on the elevator, and dangerous accidents such as the identification object is clamped or steps empty are avoided.

Description

Safety prompting method and system for elevator operation
Technical Field
The invention relates to the technical field of elevator detection, in particular to a safety prompting method and system for elevator operation.
Background
The normal operation of the elevator is ensured by a series of detection devices, and the failure of any one component can ensure that the elevator is not operated normally.
The passengers should pay attention to the actual operation condition of the elevator in the process of taking the elevator into and out of the car, and then enter or leave the car after confirming the safety (for example, the passengers should confirm that the elevator has reached the flat floor, the elevator is normally opened, and the like), so as to avoid the occurrence of dangerous accidents.
In the actual operation process of the elevator at present, many passengers look at mobile phones or other objects with low heads in the process of getting in and out of the elevator car, attention is not paid to the elevator, and dangerous accidents such as passengers being clamped and stepping empty can be caused if the elevator is abnormally operated.
Disclosure of Invention
Based on this, it is necessary to provide a safety prompting method and system for elevator operation, aiming at the problem that the conventional elevator operation may cause passengers to be clamped and step empty.
A safety prompting method for elevator operation comprises the following steps:
acquiring a depth image of an elevator use area, wherein the depth image is shot from the upper part of the elevator use area;
when image information of an identification object exists in the depth image, judging whether the identification object is in a specified state or not according to the image information;
and if the identified object is not in the designated state and the car door of the elevator is in the action state or in the opening state, sending safety prompt information representing the attention to the running state of the elevator.
A safety prompting system for elevator operation, comprising:
the system comprises an image acquisition unit, a storage unit and a control unit, wherein the image acquisition unit is used for acquiring a depth image of an elevator use area, and the depth image is shot from the upper part of the elevator use area;
an image recognition unit for judging whether the recognition object is in a designated state according to the image information when the image information of the recognition object exists in the depth image;
and the information prompting unit is used for sending safety prompting information representing the operation state of the elevator to be noticed when the identification object is not in the designated state and the car door of the elevator is in the action state or in the opening state.
According to the elevator operation safety prompting method and system, the depth image of the elevator use area is obtained, when the image information of the identification object exists in the depth image, whether the identification object is in the designated state or not is judged according to the image information, and when the identification object is not in the designated state and the car door of the elevator is in the action state or the opening state, the safety prompting information representing the state of paying attention to the elevator operation is sent. In the scheme, whether the identification object in the elevator use area is in the designated state or not can be known through the identification analysis processing of the depth image of the elevator use area, if the identification object is not in the designated state, the identification object can be prompted to pay attention to the elevator running state when a car door of the elevator is in an action or opening state, so that the attention of the identification object is focused on the elevator, and dangerous accidents such as the identification object is clamped or steps empty are avoided.
A readable storage medium, on which an executable program is stored, which program, when being executed by a processor, carries out the steps of the above-mentioned safety-alerting method of elevator operation.
A prompt device comprises a memory, a processor and an executable program which is stored on the memory and can be operated on the processor, and the steps of the safety prompt method for elevator operation are realized when the processor executes the program.
According to the safety prompting method for elevator operation, the invention also provides a readable storage medium and a prompting device, which are used for realizing the safety prompting method for elevator operation through a program. The readable storage medium and the prompting device can know whether the identification object in the elevator use area is in the designated state or not through the identification analysis processing of the depth image of the elevator use area, and if the identification object is not in the designated state, when a car door of the elevator is in an action or opening state, the identification object can be prompted to pay attention to the elevator running state, so that the attention of the identification object is focused on the elevator, and dangerous accidents such as clamping and empty stepping of the identification object are avoided.
Drawings
Fig. 1 is a schematic flow chart of a safety prompting method for elevator operation according to one embodiment;
fig. 2 is a schematic structural diagram of a safety prompt system for elevator operation according to one embodiment;
FIG. 3 is a schematic diagram of a segmentation of a real-time foreground image according to one embodiment;
FIG. 4 is a schematic diagram of a local maximum region of a live foreground image according to one embodiment;
FIG. 5 is a schematic of a height threshold in one embodiment;
fig. 6(a) to 6(e) are schematic diagrams of human body postures according to one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
Referring to fig. 1, a flow chart of a safety prompting method for elevator operation according to an embodiment of the present invention is shown. The safety prompting method for elevator operation in the embodiment comprises the following steps:
step S101: acquiring a depth image of an elevator use area, wherein the depth image is shot from the upper part of the elevator use area;
in the step, a depth image is shot from the upper part to the lower part of an elevator use area, and the depth image comprises information such as image depth values and the like;
step S102: when image information of an identification object exists in the depth image, judging whether the identification object is in a specified state or not according to the image information;
in this step, the recognition objects are various objects located in the elevator use area, and the designated state is a designated posture state of the recognition objects;
step S103: and if the identified object is not in the designated state and the car door of the elevator is in the action state or in the opening state, sending safety prompt information representing the attention to the running state of the elevator.
In this step, when the identification object is not in the designated state, if the elevator car is in the action state or in the open state, the identification object may be clamped or stepped to be empty, so that the safety prompt message can be sent when the identification object is not in the designated state and the car door of the elevator is in the action state, or when the identification object is not in the designated state and the car door of the elevator is in the open state;
in the embodiment, a depth image of an elevator use area is obtained, when image information of an identification object exists in the depth image, whether the identification object is in a specified state is judged according to the image information, and when the identification object is not in the specified state and a car door of an elevator is in an action state or an open state, safety prompt information representing the state of paying attention to the operation of the elevator is sent. In the scheme, whether the identification object in the elevator use area is in the designated state or not can be known through the identification analysis processing of the depth image of the elevator use area, if the identification object is not in the designated state, the identification object can be prompted to pay attention to the elevator running state when a car door of the elevator is in an action or opening state, so that the attention of the identification object is focused on the elevator, and dangerous accidents such as the identification object is clamped or steps empty are avoided.
Alternatively, the depth image may be acquired by a depth camera installed above the elevator use area, and a camera is included in the relevant part of the elevator at present, and the depth camera may be integrated on the camera.
In one embodiment, the safety prompting method for elevator operation further comprises the following steps:
and obtaining a foreground image according to the depth image and the background image of the elevator use area, carrying out area depth analysis on the foreground image, and judging that the image information of the identification object exists in the depth image if the foreground image has an area with the depth value within a preset identification object characteristic value range.
In this embodiment, a foreground image can be obtained by using a depth image and a background image, because the depth value of the recognition object is higher than that of the background image, the recognition object can be more prominently located in the foreground image, the image information of the recognition object in an elevator can be more directly obtained, the regional depth analysis is performed on the foreground image, the depth value information of the foreground image can be obtained, and the feature value of the recognition object can be embodied in the form of the depth value in the foreground image, so that whether the image information of the recognition object exists in the corresponding depth image can be determined according to the relationship between the depth value and the preset recognition object feature value range, and thus, the image information of the recognition object can be simply and quickly determined.
Optionally, the area of the depth value within the preset recognition object feature value range belongs to the image information of the recognition object, and various different image information algorithms are used to calculate the area of the depth value within the preset recognition object feature value range, so as to obtain the state information of the recognition object, thereby determining whether the recognition object is in the formulated state.
Optionally, when the elevator is in an idle and to-be-called state, acquiring a depth image of an elevator use area as a background image of the elevator use area; when the elevator is in an idle waiting state, the elevator use area does not have any identification object.
Optionally, the pixel data of the foreground image may be obtained by performing subtraction operation on the pixel data of the background image and the pixel data of the depth image.
In one embodiment, the step of performing the regional depth analysis on the foreground image comprises the steps of:
dividing the foreground image into a plurality of areas, and calculating the average depth value of each area;
determining a first area to which the average depth maximum value belongs according to the average depth value of each area, wherein the average value of the average depth values of other areas in a first preset length range around the first area is less than or equal to the average depth maximum value;
if an area with a depth value within a preset identification object characteristic value range exists in the foreground image, the step of judging that the image information of the identification object exists in the depth image comprises the following steps:
and if the foreground image has a first area with the maximum average depth value within the preset identification object characteristic value range, judging that the image information of the identification object exists in the depth image.
In this embodiment, since the image area occupied by the recognition object in the foreground image is limited, and in the scene of the elevator use area, the average depth value corresponding to the feature value of the recognition object is much larger than the average depth value of the image area within the first preset length range around the recognition object, that is, the average depth value corresponding to the feature value of the recognition object is the maximum value of the average depth value of the area where the recognition object is located, that is, the maximum value of the average depth, the area where the recognition object is located can be obtained by calculating the maximum value of the average depth of each area in the foreground image and matching the range of the feature value of the preset recognition object, so as to determine that the image information of the recognition object exists in the depth image.
Optionally, the foreground image may be divided into a plurality of rectangular regions, a window region formed by k × k (k is an odd number greater than 1) rectangular regions may be used as a region for calculating an average depth maximum value, all the rectangular regions of the foreground image are traversed, a window region in which an average depth value of a central rectangular region is greater than or equal to an average depth value of other (k × k-1) rectangular regions is searched, the central rectangular region is the first region, and the other (k × k-1) rectangular regions are other regions within a first preset length range around the first region. The first preset length can be adjusted according to actual needs.
In one embodiment, the safety prompting method for elevator operation further comprises the following steps:
selecting a second area in other areas within a first preset length range around the first area, wherein the difference value between the maximum average depth value and the depth value of each pixel point of the second area is less than or equal to a first depth threshold value of the designated part of the identification object;
calculating the area of the maximum value region by taking the whole of the first region and the second region as the maximum value region;
if a first area with the maximum average depth value within the preset identification object characteristic value range exists in the foreground image, the step of judging that the image information of the identification object exists in the depth image comprises the following steps:
and if the foreground image has the first region and the area of the maximum value region is within the preset area range, judging that the image information of the identification object exists in the depth image.
In this embodiment, the image area occupied by the recognition object in the foreground image is not only the first area, but also the second area, the difference between the average depth maximum and the depth value of each pixel point in the second area is less than or equal to the first depth threshold of the designated portion, the areas of the maximum areas formed by the first area and the second area of different recognition objects are different and are all within the preset area range, and the area of the maximum area formed by the first area and the second area is calculated and is matched with the preset area range, so that the area where the recognition object is located can be further accurately obtained, and the image information of the recognition object in the depth image is determined.
In one embodiment, the safety prompting method for elevator operation further comprises the following steps:
carrying out Hough circle detection on all the maximum value areas, and judging whether a target maximum value area meeting Hough circle detection conditions exists or not;
if the foreground image has the first area and the area of the maximum area is within the preset area range, the step of judging that the image information of the identification object exists in the depth image comprises the following steps:
and if the foreground image has the first region, the area of the maximum value region is within the preset area range, and the target maximum value region exists, judging that the image information of the identification object exists in the depth image.
In this embodiment, the first region and the second region occupied by the recognition object in the foreground image belong to a designated region of the recognition object, and the designated region may be a circle or an approximate circle, and hough circle detection is performed on all the maximum value regions, so that a target maximum value region with a circular or approximate circle shape can be obtained, and the situation that the non-circular maximum value region with an area within a preset area range is mistakenly regarded as belonging to the region where the recognition object is located in the foreground image can be avoided, and the region where the recognition object is located can be further accurately obtained, thereby determining that the image information of the recognition object exists in the depth image.
In one embodiment, the safety prompting method for elevator operation further comprises the following steps:
taking the whole of the target maximum value area and other areas within a second preset length range around the target maximum value area as an interested area, carrying out binarization processing on the interested area, and obtaining a third area in which the depth value of a pixel point in the interested area is greater than or equal to a binarization threshold value, wherein the binarization threshold value is the difference value between the average depth maximum value of the target maximum value area and a second depth threshold value;
calculating the ratio of the area of the target maximum value region to the area of the third region;
if the foreground image has the first area, the area of the maximum area is within the preset area range, and the target maximum area exists, the step of judging that the image information of the identification object exists in the depth image comprises the following steps:
and if the foreground image has the first region, the area of the maximum region is within a preset area range, the target maximum region exists, and the ratio is within a preset ratio range, judging that the image information of the identification object exists in the depth image.
In this embodiment, other regions around the target maximum value region and the target maximum value region form a region of interest (ROI), binarization processing is performed on the region of interest, a third region in which the depth value of a pixel point in the region of interest is greater than or equal to a binarization threshold is obtained, the third region can represent all regions occupied by the identification object in the foreground image, and since a ratio of an area of a designated region of the identification object to an area of all regions of the identification object is within a preset ratio range, by calculating a ratio of the area of the target maximum value region to the area of the third region and matching the preset ratio range, a region where the identification object is located can be further accurately obtained, so that image information of the identification object in the depth image is determined.
Optionally, the second depth threshold is a threshold of a difference between a lowest depth value and a highest depth value of the recognition object in the foreground image, and the second preset length may be adjusted according to actual needs.
In one embodiment, the step of determining whether the recognition object is in the designated state based on the image information includes the steps of:
acquiring a first central point of a target maximum value area and a second central point of a third area, and calculating a distance value between the first central point and the second central point;
if the distance value is smaller than or equal to the distance threshold value, judging that the identification object is in a specified state;
and if the distance value is larger than the distance threshold value, judging that the identification object is not in the specified state.
In this embodiment, the first central point of the target maximum value region and the second central point of the third region are obtained, and whether the identification object is in the designated state can be simply and quickly determined based on the distance value between the first central point and the second central point.
In one embodiment, if the distance value is greater than the distance threshold, the step of determining that the recognition object is not in the designated state includes the steps of:
and if the distance value is greater than the distance threshold value and the connecting line of the first central point and the second central point is perpendicular to the major axis of the third area, judging that the recognition object is in a downward overlooking state.
In this embodiment, when the recognition object is in the downward-looking state, the first central point of the target maximum value region is offset from the second central point of the third region, and the offset direction is perpendicular to the main long axis of the third region, and by comparing the distance value between the first central point and the second central point with the distance threshold and determining the relationship between the connection line between the first central point and the second central point and the main long axis of the third region, it can be determined whether the recognition object is in the downward-looking state, the process is simple, and the determination speed is high.
In one embodiment, the identification object comprises an elevator passenger, the designated state comprises a state in which the elevator passenger is looking forward, the identification object feature value comprises a passenger height value, the designated portion comprises a passenger head, and the second depth threshold is a preset depth value from the top of the passenger head to the shoulder.
In this embodiment, the identification object may be an elevator passenger, when the passenger is not in a front-view state, a safety prompt message representing that the passenger pays attention to the elevator operation state may be sent to remind the passenger of the elevator operation state, thereby avoiding a dangerous accident such as the passenger being caught or stepping on the elevator, whether the passenger presents a difference in the foreground image in front-view state is represented as a change in the passenger's head, the specified portion may be the passenger's head, since the depth image is captured from above the elevator use area, the image area of the passenger in the foreground image displays an area from the top of the passenger's head to the shoulder, the second depth threshold is an image area for obtaining the passenger, and may be a preset depth value from the top of the passenger's head to the shoulder.
In one embodiment, the elevator usage zone comprises an elevator car interior zone or a preset range zone in front of an elevator car door.
In this embodiment, the elevator use area may be an elevator car interior area or a preset range area in front of an elevator car door, and when a depth image of the elevator car interior area is obtained, safety prompt information representing that an elevator operation state is noticed can be sent to an identification object in the elevator car; when the depth image of the preset range area in front of the elevator car door is acquired, safety prompt information representing the attention to the running state of the elevator can be sent to the identification object in front of the elevator car door, so that the identification object using the elevator is comprehensively reminded.
In one embodiment, the step of obtaining a depth image of the elevator use area comprises the steps of:
obtaining a depth image inside an elevator car;
the safety prompting method for the operation of the elevator further comprises the following steps:
when image information of the identification object exists in the depth image, acquiring a target depth value of the identification object in the depth image;
and if the target depth value is smaller than the preset depth threshold value, sending out prompt information representing leaving the elevator.
In this embodiment, through the recognition analysis processing of the depth image inside the elevator car, the target depth value of the recognition object in the elevator use area can be obtained, and when the target depth value is smaller than the preset depth threshold value, it indicates that the recognition object is not suitable for taking the elevator, and prompt information representing leaving the elevator can be sent to prompt the recognition object to leave the elevator.
Alternatively, the target depth value may be an average depth maximum value of the first area, or may be an average depth value of the maximum area.
In one embodiment, the step of obtaining a target depth value of the identified object in the depth image comprises the steps of:
identifying all areas with average depth values within a preset identification object characteristic value range, and acquiring the depth values of the identified areas;
if the target depth value is smaller than the preset depth threshold value, the step of sending out prompt information representing leaving the elevator comprises the following steps:
and if the depth values of the identified areas are smaller than the preset depth threshold value, sending prompt information representing leaving the elevator.
In one embodiment, the step of issuing a prompt message representative of leaving the elevator comprises the steps of:
controlling the elevator to enter and keep the door opening state, and sending out voice prompt information representing the leaving of the elevator.
In this embodiment, when the voice prompt message is sent, the elevator can be controlled to enter and keep the door opening state, so that the identification object can leave the elevator conveniently.
In one embodiment, the step of issuing a prompt message representative of leaving the elevator comprises the steps of:
the elevator is driven to reach the preset floor, the elevator is controlled to enter and keep the door opening state, and voice prompt information representing the elevator leaving is sent.
In this embodiment, when sending voice prompt information, can drive the elevator and reach preset floor, control the elevator and get into and keep the state of opening the door, the discernment object of being convenient for leaves the elevator, guarantees the safety of discernment object.
In one embodiment, the step of obtaining a depth image of the elevator use area comprises the steps of:
obtaining a depth image inside an elevator car;
the safety prompting method for the operation of the elevator further comprises the following steps:
detecting whether the elevator is in a fault state or not when image information of the identification object exists in the depth image;
if so, sending prompt information representing that the elevator is abnormally closed.
In the embodiment, whether the identification object exists in the elevator car can be judged through the depth image in the elevator car, and when the identification object exists and the elevator is detected to be in a fault state, prompt information representing that the elevator is abnormally closed can be sent out; whether the identification object exists in the elevator is determined firstly, whether the elevator breaks down is judged, and the process can be carried out immediately before the elevator breaks down due to the fact that the consumed time is short, so that the adverse effect of the lighting fault in the elevator car on the elevator fault detection can be avoided, and the abnormal closing accident of the elevator can be accurately prompted.
In one of the embodiments, the step of detecting whether the elevator is in a fault state comprises the steps of:
and detecting whether the elevator has door opening failure or stays in a non-leveling zone.
In this embodiment, the elevator fault state includes an elevator door opening fault or an elevator staying in a non-leveling zone, and when the elevator door opening fault or the elevator staying in the non-leveling zone, the identification object in the elevator car cannot safely enter or exit the elevator, and an abnormal closing accident of the elevator needs to be prompted.
In one embodiment thereof, the step of detecting whether the elevator is malfunctioning comprises the steps of:
and detecting whether the time for which the elevator is continuously in the door closing state exceeds preset time.
In this embodiment, the time that the elevator is continuously in the door closing state exceeds the preset time, and the identification object in the elevator car can not normally enter or exit the elevator, and the elevator needs to be prompted to have an abnormal closing accident.
According to the safety prompting method for elevator operation, the invention also provides a safety prompting system for elevator operation, and the embodiment of the safety prompting system for elevator operation is explained in detail below.
Referring to fig. 2, a schematic diagram of a safety prompting system for elevator operation according to an embodiment of the present invention is shown. The safety prompt system for elevator operation in this embodiment includes:
an image obtaining unit 210 for obtaining a depth image of an elevator use area, wherein the depth image is obtained by shooting from above the elevator use area;
an image recognition unit 220 for judging whether the recognition object is in a designated state according to the image information when the image information of the recognition object exists in the depth image;
and an information prompt unit 230 for sending a safety prompt message representing the state of paying attention to the operation of the elevator when the identification object is not in the designated state and the car door of the elevator is in the motion state or in the open state.
In one embodiment, the image recognition unit 220 obtains a foreground image according to the depth image and a background image of the elevator use area, and performs area depth analysis on the foreground image; when an area with a depth value within a preset identification object characteristic value range exists in the foreground image, the image information of the identification object in the depth image is judged to exist.
In one embodiment, the image recognition unit 220 divides the foreground image into a plurality of regions, calculates an average depth value of each region; determining a first area to which the average depth maximum value belongs according to the average depth value of each area, wherein the average value of the average depth values of other areas in a first preset length range around the first area is less than or equal to the average depth maximum value; when a first region with the maximum average depth value within a preset identification object characteristic value range exists in the foreground image, the image information of the identification object exists in the depth image.
In one embodiment, the image recognition unit 220 selects a second area of other areas within a first preset length range around the first area, wherein the difference between the average depth maximum and the depth value of each pixel point in the second area is less than or equal to a first depth threshold of the designated portion of the recognition object; calculating the area of the maximum value region by taking the whole of the first region and the second region as the maximum value region; when the first region exists in the foreground image and the area of the maximum value region is within the preset area range, the image information of the identification object exists in the depth image.
In one embodiment, the image recognition unit 220 performs hough circle detection on all the maximum value regions, and determines whether a target maximum value region satisfying the hough circle detection condition exists; and when the first region exists in the foreground image, the area of the maximum value region is within a preset area range, and the target maximum value region exists, judging that the image information of the identification object exists in the depth image.
In one embodiment, the image recognition unit 220 uses the whole of the target maximum value region and other regions within a second preset length range around the target maximum value region as the region of interest, performs binarization processing on the region of interest, and obtains a third region in which the depth value of a pixel point in the region of interest is greater than or equal to a binarization threshold, where the binarization threshold is a difference between an average depth maximum value of the target maximum value region and a second depth threshold; calculating the ratio of the area of the target maximum value region to the area of the third region; and when the ratio is within a preset ratio range, judging that the image information of the identification object exists in the depth image.
In one embodiment, the image recognition unit 220 obtains a first center point of the target maximum region and a second center point of the third region, and calculates a distance value between the first center point and the second center point; when the distance value is smaller than or equal to the distance threshold value, judging that the identification object is in a specified state; when the distance value is greater than the distance threshold value, it is determined that the recognition object is not in the designated state.
In one embodiment, the image recognition unit 220 determines that the recognition object is in a downward-looking state when the distance value is greater than the distance threshold and the connection line between the first center point and the second center point is perpendicular to the major axis of the third area.
In one embodiment, the identification object comprises an elevator passenger, the designated state comprises a state in which the elevator passenger is looking forward, the identification object feature value comprises a passenger height value, the designated portion comprises a passenger head, and the second depth threshold is a preset depth value from the top of the passenger head to the shoulder.
In one embodiment, the elevator usage zone comprises an elevator car interior zone or a preset range zone in front of an elevator car door.
The terms "first," "second," and the like are used merely to distinguish one element from another, and do not limit the other elements.
The safety prompting system for elevator operation and the safety prompting method for elevator operation correspond to each other one by one, and the technical characteristics and the beneficial effects explained in the embodiment of the safety prompting method for elevator operation are all suitable for the embodiment of the safety prompting system for elevator operation.
According to the safety prompt method for the elevator operation, the embodiment of the invention also provides a readable storage medium and a prompt device.
The readable storage medium stores executable program, which realizes the procedure of the safety prompting method for elevator operation when the program is executed by the processor; the prompting device comprises a memory, a processor and an executable program which is stored on the memory and can be operated on the processor, and the steps of the safety prompting method for elevator operation are realized when the processor executes the program.
The readable storage medium and the prompting device can know whether the identification object in the elevator use area is in the designated state or not through the identification analysis processing of the depth image of the elevator use area, and if the identification object is not in the designated state, when a car door of the elevator is in an action or opening state, the identification object can be prompted to pay attention to the elevator running state, so that the attention of the identification object is focused on the elevator, and dangerous accidents such as clamping and empty stepping of the identification object are avoided.
In a particular embodiment, the solution of the invention can be applied in a scenario in which it is detected whether a passenger using an elevator is in a heads-down state.
The prompt method for the safe operation of the elevator utilizes the depth camera to shoot from top to bottom, detects the state of the recognition object in real time, and reminds the recognition object of the safety of the elevator in a voice mode if the recognition object is found to be low and the situation of the elevator is not noticed.
The specific process is as follows:
human body detection is performed by a depth camera, and after a passenger is confirmed, the behavior of the passenger is recognized.
(1) Human body detection
1. When no passenger or other articles exist in the current area, collecting a frame of depth image as a background image; the current zone can be an elevator car interior zone or a preset range zone in front of an elevator car door;
2. acquiring a depth image in real time;
3. extracting a foreground image: real-time foreground image-background image-real-time depth image
4. Setting the pixel width of a depth sensor of a depth camera as W, the pixel height as H, and the pixel of each region as A, dividing a real-time foreground image into
Figure BDA0001399502980000131
A region, as shown in fig. 3.
5. After segmenting the real-time foreground image, calculating an average depth value of each region
Figure BDA0001399502980000132
Figure BDA0001399502980000133
Using kA × kA (k is 3, 5, …, set according to the size of the segmented region) as a window, traversing the real-time foreground image, and finding a region corresponding to the average depth value of the central region that is the region having the maximum average depth value in the real-time foreground image, where the average depth value of the central region is greater than or equal to the average depth values of the other (kA × kA-1) regions.
6. The average depth maximum reflects the height of the human body, and only when the value is within a certain range, the current object is considered to be the human body.
Height threshold value of 1 ≤ avgGmHeight threshold value 2 (1)
7. And taking the area with the maximum average depth value as a central area of the kA & kA area, and comparing the depth value of each pixel point of all the areas in the kA & kA area with the average depth value of the central area:
everyGj≥(avgGm-height threshold h1)(0≤j＜kA*kA) (2)
Height threshold h1The difference between the depth value of the top of the human head and the depth value corresponding to the edge of the human head in the depth image may be found out, and the maximum area of the foreground image is found out according to the formula (2), and the result is shown in fig. 4.
8. Limiting the area S of the maximum value area in the previous step:
area threshold value of 1 is less than or equal to head area S is less than or equal to area threshold value of 2
9. And (3) the head of the human body is similar to a circle, Hough circle detection is carried out on the result screened in the previous step, and the area with the contour shape not meeting the requirement is removed.
10. Through the above steps, the human head can be successfully identified, but the detection is not enough, and the possibility of false detection exists for the ball with proper size and proper height. Therefore, the judgment and detection of the human body are combined on the basis of the steps.
Setting the head region and the nearby region judged by the steps as ROI regions, and carrying out binarization processing on the region aiming at each ROI region, wherein the binarization threshold value is as follows:
the binary threshold value is the depth value-height threshold value h corresponding to the head top2
Height threshold h2The difference between the depths of the human head and the human shoulder may be used, as shown in fig. 5, the binarization processing is performed to obtain the body area of the human body, and the body area S2 is extracted.
11. The size of the human head is generally proportional to the size of the human body:
ratio R-head area S/body ladder area S2
If the head-body ratio R is within a specific range, the human body is judged.
(2) Human body posture detection
When the human body stands upright, the central moment m1 of the human head contour in the real-time foreground image is basically coincident with the central moment m2 of the human body contour, as shown in fig. 6(a), when the human body tilts head, swings left head and swings right head, the relationship between the central moment m1 and the central moment m2 is respectively shown in fig. 6(b), (c), (d) and (e), wherein the arrow directions indicate the right front of the human body.
As can be seen from fig. 6, when the human body is not looking straight ahead, there will be a distance between the head central moment m1 and the body torso central moment m 2. Since it is difficult for the depth camera to distinguish between the front and rear of the human body, it is impossible to distinguish between the two cases of fig. 6(b) (c) and the two cases of fig. 6(d) (e).
In the two cases of fig. 6(d) and (e), it is considered that the passengers do the left-right head swinging motion and the stretching motion, and the attention is directly in front, and the two cases do not generally occur when the passengers get in and out of the car, so the two cases can be ignored.
In the two cases of fig. 6(b) and (c), it is considered that the walking operation is not performed when the human body is raised, and the backward walking operation is not performed when the human body is lowered, so that only the case where the human body is lowered to the front can be considered.
For fig. 6(b), when the deviation D between the human head contour center moment m1 and the human body contour center moment m2 in the y direction is detected to exceed a certain threshold, it is considered that the passenger looks at something else (e.g., looks at a mobile phone) at the head low, and does not pay attention to the running condition of the elevator. If the passenger is detected to enter/leave the elevator car at the moment, the voice reminds the passenger to pay attention to the running condition of the elevator, and safety accidents such as clamping, stepping on empty and the like are avoided.
(3) Child detection
When the object is a passenger, the height is used to judge whether the object is a child, and when the object is a passenger, the height is used to judge whether the object is a child
Average depth value of foreground image head region < height threshold 3
The passenger is judged to be a child.
Because children do not have safety consciousness, injury accidents easily occur in the process of taking the elevator. The children take the elevator and the escalator and are accompanied by parents or other guardians, and the parents or the other guardians are obligated to protect the personal safety of the children in the elevator taking process.
1) When detecting that only a single or a plurality of children enter the elevator car, the elevator does not close the door and does not walk, and sends out voice prompt to remind the children to leave the elevator as soon as possible.
2) When an adult and a child enter the car at the same time, the elevator can normally walk because whether the relationship between the child and the adult exists cannot be judged. When the elevator is on a certain floor, adults leave the elevator car, and only children are detected to be left in the elevator car, the elevator does not respond to internal and external calls, and directly returns to a relatively safe station floor (such as a base floor) with high passenger flow, and the door is kept open until all the children leave the elevator car. In the process, the background of the elevator control system informs the manager of the movement of the children and informs parents of avoiding missing.
(4) Trapped person detection
1) If the passenger exists in the elevator car according to the human body detection algorithm in the step (1), the elevator detects that the elevator breaks down, and if the door cannot be opened or the elevator stays in a non-flat zone, the elevator is judged to be trapped.
2) If the passenger exists in the elevator car and the elevator is not opened beyond a certain time threshold value according to the human body detection algorithm in the step (1), the elevator is judged to be trapped.
The depth camera can adopt TOF technology, can not receive the influence of the interior illumination environment of car when gathering the depth image in the car, even the car illumination breaks down, does not influence the acquirement of depth image under the condition of not lighting yet.
When the trapped person is in fault, the background of the ladder control system informs maintenance personnel to process the trapped person, and passengers are pacified by voice.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
Those skilled in the art will appreciate that all or part of the steps in the method for implementing the above embodiments may be implemented by a program instructing the relevant hardware. The program may be stored in a readable storage medium. Which when executed comprises the steps of the method described above. The storage medium includes: ROM/RAM, magnetic disk, optical disk, etc.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (11)

1. A safety prompting method for elevator operation is characterized by comprising the following steps:
acquiring a depth image of an elevator use area, wherein the depth image is shot from the upper part of the elevator use area;
when image information of an identification object exists in the depth image, judging whether the identification object is in a specified state according to the image information;
if the identification object is not in the designated state and the car door of the elevator is in an action state or an open state, sending safety prompt information representing the state of paying attention to the running of the elevator;
obtaining a foreground image according to the depth image and a background image of the elevator use area, performing area depth analysis on the foreground image, and judging that image information of an identification object exists in the depth image if an area with a depth value within a preset identification object characteristic value range exists in the foreground image;
the step of performing regional depth analysis on the foreground image comprises the following steps:
dividing the foreground image into a plurality of areas, and calculating the average depth value of each area;
determining a first area to which an average depth maximum value belongs according to the average depth value of each area, wherein the average value of the average depth values of other areas in a first preset length range around the first area is less than or equal to the average depth maximum value;
if an area with a depth value within a preset identification object characteristic value range exists in the foreground image, the step of judging that the image information of the identification object exists in the depth image comprises the following steps:
and if the foreground image has a first area with the maximum average depth value within the range of the preset characteristic value of the identification object, judging that the image information of the identification object exists in the depth image.
2. The safety prompting method for elevator operation according to claim 1, characterized by further comprising the steps of:
selecting a second area in other areas within a first preset length range around the first area, wherein the difference value between the maximum average depth value and the depth value of each pixel point in the second area is less than or equal to a first depth threshold value of the designated part of the identification object;
calculating an area of a maximum value region by using the entirety of the first region and the second region as the maximum value region;
if a first area with the maximum average depth value within a preset identification object characteristic value range exists in the foreground image, the step of judging that the image information of the identification object exists in the depth image comprises the following steps:
and if the foreground image has the first region and the area of the maximum value region is within a preset area range, judging that the depth image has image information of an identification object.
3. The safety prompting method for elevator operation according to claim 2, characterized by further comprising the steps of:
carrying out Hough circle detection on all the maximum value areas, and judging whether a target maximum value area meeting Hough circle detection conditions exists or not;
if the foreground image has the first region and the area of the maximum value region is within a preset area range, the step of judging that the image information of the identification object exists in the depth image comprises the following steps:
and if the foreground image has the first region, the area of the maximum value region is within a preset area range, and the target maximum value region exists, judging that the image information of the identification object exists in the depth image.
4. The safety prompting method for elevator operation according to claim 3, characterized by further comprising the steps of:
taking the whole of the target maximum value area and other areas within a second preset length range around the target maximum value area as an area of interest, performing binarization processing on the area of interest, and acquiring a third area in which the depth value of a pixel point in the area of interest is greater than or equal to a binarization threshold value, wherein the binarization threshold value is the difference value between the average depth maximum value of the target maximum value area and a second depth threshold value;
calculating a ratio of an area of the target maximum region to an area of the third region;
if the foreground image has the first region, the area of the maximum region is within a preset area range, and the target maximum region exists, the step of determining that the image information of the identification object exists in the depth image comprises the following steps:
and if the foreground image has the first region, the area of the maximum region is within a preset area range, the target maximum region exists, and the ratio is within a preset ratio range, judging that the image information of the identification object exists in the depth image.
5. The method for prompting safety of elevator operation according to claim 4, wherein the step of judging whether the recognition object is in a designated state according to the image information comprises the steps of:
acquiring a first central point of the target maximum value area and a second central point of the third area, and calculating a distance value between the first central point and the second central point;
if the distance value is smaller than or equal to a distance threshold value, judging that the identification object is in the specified state;
and if the distance value is larger than the distance threshold value, judging that the identification object is not in the specified state.
6. The method of claim 5, wherein the step of determining that the identified object is not in the designated state if the distance value is greater than the distance threshold comprises the steps of:
and if the distance value is larger than the distance threshold value and the connecting line of the first central point and the second central point is perpendicular to the major axis of the third area, judging that the identification object is in a downward overlooking state.
7. The safety prompting method for elevator operation according to any one of claims 4 to 6, wherein the identification object comprises an elevator passenger, the designated state comprises a state that the elevator passenger is looking ahead, the identification object characteristic value comprises a passenger height value, the designated part comprises a passenger head, and the second depth threshold is a preset depth value from the top of the passenger head to the shoulder.
8. The safety prompting method for elevator operation according to any one of claims 1 to 6, wherein the elevator use zone comprises an elevator car interior zone or a preset range zone in front of an elevator car door.
9. An elevator operation safety prompt system, comprising:
the system comprises an image acquisition unit, a display unit and a control unit, wherein the image acquisition unit is used for acquiring a depth image of an elevator use area, and the depth image is shot from the upper part of the elevator use area;
the image recognition unit is used for judging whether the recognition object is in a specified state or not according to the image information when the image information of the recognition object exists in the depth image;
the information prompting unit is used for sending safety prompting information representing the running state of the elevator to be noticed when the identification object is not in the designated state and the car door of the elevator is in an action state or an opening state;
the image recognition unit is also used for acquiring a foreground image according to the depth image and a background image of an elevator use area and carrying out area depth analysis on the foreground image; when an area with a depth value within a preset identification object characteristic value range exists in the foreground image, judging that image information of an identification object exists in the depth image;
the image identification unit is also used for dividing the foreground image into a plurality of areas and calculating the average depth value of each area; determining a first area to which the average depth maximum value belongs according to the average depth value of each area, wherein the average value of the average depth values of other areas in a first preset length range around the first area is less than or equal to the average depth maximum value; when a first region with the maximum average depth value within a preset identification object characteristic value range exists in the foreground image, the image information of the identification object exists in the depth image.
10. A readable storage medium, on which an executable program is stored, characterized in that the program, when being executed by a processor, carries out the steps of the safety warning method of elevator operation of any one of claims 1 to 8.
11. A prompting device comprising a memory, a processor and an executable program stored on the memory and executable on the processor, wherein the processor executes the program to implement the steps of the safety prompting method for elevator operation according to any one of claims 1 to 8.
CN201710792185.8A 2017-09-05 2017-09-05 Safety prompting method and system for elevator operation Active CN109353907B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710792185.8A CN109353907B (en) 2017-09-05 2017-09-05 Safety prompting method and system for elevator operation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710792185.8A CN109353907B (en) 2017-09-05 2017-09-05 Safety prompting method and system for elevator operation

Publications (2)

Publication Number Publication Date
CN109353907A CN109353907A (en) 2019-02-19
CN109353907B true CN109353907B (en) 2020-09-15

Family

ID=65349678

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710792185.8A Active CN109353907B (en) 2017-09-05 2017-09-05 Safety prompting method and system for elevator operation

Country Status (1)

Country Link
CN (1) CN109353907B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110127468B (en) * 2019-04-12 2023-02-07 深圳壹账通智能科技有限公司 Elevator control method, elevator control device, computer-readable storage medium and computer equipment
CN111986253B (en) * 2020-08-21 2023-09-15 日立楼宇技术（广州）有限公司 Method, device, equipment and storage medium for detecting elevator crowding degree
CN114283305A (en) * 2020-12-25 2022-04-05 上海商汤智能科技有限公司 Image detection method and device, electronic equipment and storage medium
CN112990038A (en) * 2021-03-24 2021-06-18 杭州海康威视数字技术股份有限公司 Escalator safety reminding method and device and computer storage medium
CN113955594B (en) * 2021-10-18 2024-02-27 日立楼宇技术（广州）有限公司 Elevator control method and device, computer equipment and storage medium
CN115028046A (en) * 2022-05-10 2022-09-09 菱王电梯有限公司 Elevator management system and method

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102556788A (en) * 2012-02-03 2012-07-11 上海新时达电气股份有限公司 Elevator forecast information display and elevator forecast information display method
CN102800126A (en) * 2012-07-04 2012-11-28 浙江大学 Method for recovering real-time three-dimensional body posture based on multimodal fusion
CN104899883A (en) * 2015-05-29 2015-09-09 北京航空航天大学 Indoor object cube detection method for depth image scene
CN106144816A (en) * 2015-04-03 2016-11-23 奥的斯电梯公司 Occupant detection based on depth transducer
CN106315324A (en) * 2015-06-16 2017-01-11 奥的斯电梯公司 Elevator system capable of monitoring child's use and control method for same
CN107416629A (en) * 2017-09-05 2017-12-01 广州日滨科技发展有限公司 The control method and system of elevator safety operation

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102556788A (en) * 2012-02-03 2012-07-11 上海新时达电气股份有限公司 Elevator forecast information display and elevator forecast information display method
CN102800126A (en) * 2012-07-04 2012-11-28 浙江大学 Method for recovering real-time three-dimensional body posture based on multimodal fusion
CN106144816A (en) * 2015-04-03 2016-11-23 奥的斯电梯公司 Occupant detection based on depth transducer
CN104899883A (en) * 2015-05-29 2015-09-09 北京航空航天大学 Indoor object cube detection method for depth image scene
CN106315324A (en) * 2015-06-16 2017-01-11 奥的斯电梯公司 Elevator system capable of monitoring child's use and control method for same
CN107416629A (en) * 2017-09-05 2017-12-01 广州日滨科技发展有限公司 The control method and system of elevator safety operation

Also Published As

Publication number Publication date
CN109353907A (en) 2019-02-19

Similar Documents

Publication Publication Date Title
CN109353907B (en) Safety prompting method and system for elevator operation
CN107416630B (en) Method and system for detecting abnormal closing of elevator
US10059569B2 (en) Monitoring system of a passenger conveyor, a passenger conveyor, and a monitoring method thereof
CN107416629B (en) The control method and system of elevator safety operation
KR102270677B1 (en) Apparatus for safety-driving of vehicle
CN110084184B (en) Safety belt unfastening detection system and method based on image processing technology
JP5473801B2 (en) Monitoring device
US10818172B2 (en) Method, device and system for processing startup of preceding vehicle
JP6786279B2 (en) Image processing device
US7620216B2 (en) Method of tracking a human eye in a video image
US20190185295A1 (en) User detection system and image processing device
JP5106356B2 (en) Image monitoring device
JP5809751B2 (en) Object recognition device
CN113076856B (en) Bus safety guarantee system based on face recognition
CN110619300A (en) Correction method for simultaneous recognition of multiple faces
US11161470B2 (en) Occupant observation device
JP6739459B2 (en) Person matching device
KR101667835B1 (en) Object localization using vertical symmetry
JP4992823B2 (en) Face detection apparatus and face detection method
JP4989249B2 (en) Eye detection device, dozing detection device, and method of eye detection device
CN112784669A (en) Method for object re-recognition
CN111383248B (en) Pedestrian red light running judging method and device and electronic equipment
CN106022458A (en) People fast counting method for school bus safety
CN209657454U (en) Coal-mine fire identifying system
JP2018142173A (en) Image monitoring device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant