CN110642113B

CN110642113B - Elevator door state detection method and device

Info

Publication number: CN110642113B
Application number: CN201911001460.5A
Authority: CN
Inventors: 刘永光; 孙虎昌
Original assignee: Beijing Orion Star Technology Co Ltd
Current assignee: Beijing Orion Star Technology Co Ltd
Priority date: 2019-10-21
Filing date: 2019-10-21
Publication date: 2021-04-16
Anticipated expiration: 2039-10-21
Also published as: CN110642113A

Abstract

The invention discloses a method and a device for detecting the state of an elevator door, which relate to the technical field of artificial intelligence and are used for solving the problem that the communication between a robot and an internet-of-things controller is needed when the state of the elevator door is detected at present, and the process is complicated, wherein the method comprises the following steps: the robot acquires an image of an area where an appointed elevator door is located through a camera, wherein at least one preset label is arranged on the elevator door; detecting a preset label in the image; the robot determines whether the elevator door is in an open state or not according to the number of the preset labels in the image, and the robot can judge the state of the elevator door according to the number of the preset labels in the acquired image by fully utilizing the installed hardware equipment camera on the robot, only needs to arrange the necessary preset labels on the elevator door, is simple to construct, does not need to communicate with the Internet-of-things controller, and is simpler and more convenient.

Description

Elevator door state detection method and device

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a method and a device for detecting the state of an elevator door.

Background

Service robots have entered the market and are finding application in more and more commercial scenarios, such as shopping malls, hotels, museums, etc. However, most service robots can only be limited to a certain floor, and if the service range is expanded to a plurality of floors, the problem that the robots independently enter/exit the elevator must be solved.

In the related art, whether an elevator is opened or closed is detected through an internet of things mode, a certain data exchange mode is established between an elevator controller and a robot, and the robot inquires the elevator controller or the elevator controller actively informs the robot of the opening/closing state of an elevator door. Because the mode that adopts the thing networking needs research and development dedicated thing networking controller to need carry out thing networking transformation to original elevator, it is great to fall to the ground the construction degree of difficulty.

To sum up, communication between the robot and the internet-of-things controller is needed when the state of the elevator door is detected at present, and the process is complicated.

Disclosure of Invention

The invention provides a method and a device for detecting the state of an elevator door, which are used for solving the problem that the communication between a robot and an internet-of-things controller is needed when the state of the elevator door is detected in the prior art, and the process is complicated.

In a first aspect, a method for detecting a state of an elevator door provided in an embodiment of the present invention includes:

the robot acquires an image of an area where an appointed elevator door is located through a camera, wherein at least one preset label is arranged on the elevator door;

detecting a preset label in the image;

and determining whether the elevator door is in an opening state or not according to the number of preset labels in the image.

Optionally, after the robot acquires the image of the area where the designated elevator door is located through the camera, before detecting the preset tag in the image, the method further includes:

and carrying out distortion correction and/or denoising treatment on the image.

Optionally, determining whether the elevator door is in an open state according to the number of preset tags in the image includes:

and if the number of the preset labels in the image is 0, determining that the elevator door is in an open state.

Optionally, the elevator door is provided with N preset tags, where N is a positive integer greater than 1;

determining whether the elevator door is in an open state according to the number of preset tags in the image, including:

if the number of the preset labels in the image is N, determining that the elevator door is in a closed state;

and if the number of the preset labels in the image is more than 0 and less than N, determining that the elevator door is in an opening and closing state.

Optionally, the method further includes:

and if the duration of the opening and closing state of the elevator door is determined to exceed the preset duration, determining that the elevator door is abnormal.

Optionally, the elevator door comprises at least one preset label arranged on the edges of the left side and the right side of each door; and/or

The elevator door comprises at least one preset label arranged on the edges of the upper side and the lower side of each door.

Optionally, before the robot acquires an image of an area where the designated elevator door is located through the camera, the method further includes:

the robot determines the distance between the robot and the elevator door to be a set acquisition distance, wherein the acquisition distance is determined according to the acquisition angle of the camera.

In a second aspect, an elevator door state detection apparatus provided in an embodiment of the present invention includes:

the system comprises an acquisition unit, a monitoring unit and a control unit, wherein the acquisition unit is used for acquiring an image of an area where an appointed elevator door is located through a camera, and the elevator door is provided with at least one preset label;

the detection unit is used for detecting a preset label in the image;

and the first determining unit is used for determining whether the elevator door is in an opening state or not according to the number of the preset labels in the image.

Optionally, the apparatus further comprises:

the preprocessing unit is used for carrying out distortion correction and/or denoising processing on the image after the acquisition unit acquires the image of the area where the appointed elevator door is located through the camera and before the detection unit detects the preset label in the image.

Optionally, the first determining unit is specifically configured to:

the first determining unit is specifically configured to:

Optionally, the apparatus further comprises:

and the second determining unit is used for determining that the elevator door is abnormal if the duration of the opening and closing state of the elevator door is determined to exceed the preset duration.

Optionally, the apparatus further comprises:

and the third determining unit is used for determining the distance between the acquisition unit and the elevator door to be a set acquisition distance before the acquisition unit acquires the image of the area where the appointed elevator door is located through the camera, wherein the acquisition distance is determined according to the acquisition angle of the camera.

In a third aspect, an embodiment of the present invention further provides a robot, including a processor and a memory, where the memory stores program codes, and when the program codes are executed by the processor, the steps of the method according to the first aspect are implemented.

In a fourth aspect, the present invention also provides a computer storage medium having a computer program stored thereon, which when executed by a processor, performs the steps of the method of the first aspect.

The invention has the following beneficial effects:

the embodiment of the invention fully utilizes the camera of the hardware equipment installed on the robot, directly collects images through the camera and judges the state of the elevator door by detecting the number of the preset labels in the images. Only need set up necessary label on the lift-cabin door can, the construction is simple, and the robot need not to communicate with thing networking controller, and is more simple and convenient.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic diagram of a method for detecting a state of an elevator door according to an embodiment of the present invention;

fig. 2A is a schematic diagram of a default label according to an embodiment of the present invention;

fig. 2B is a schematic diagram of another preset tag according to an embodiment of the present invention;

fig. 3 is a schematic diagram of a collecting distance according to an embodiment of the present invention;

FIG. 4A is a schematic view of a first midsplit double door in a closed state according to an embodiment of the invention;

fig. 4B is a schematic diagram illustrating a first midsplit double acting door according to an embodiment of the invention in an opening and closing state;

fig. 4C is a schematic diagram illustrating a second midsplit double acting door according to an embodiment of the invention in an opening and closing state;

fig. 4D is a schematic view illustrating a third split double door according to an embodiment of the present invention in an opening/closing state;

FIG. 4E is a schematic diagram illustrating an opened state of a first midsplit double-acting door according to an embodiment of the invention;

FIG. 5A is a schematic view of a second type of center-opening double-folding door according to an embodiment of the present invention in a closed state;

fig. 5B is a schematic view illustrating a fourth split double door according to an embodiment of the present invention in an opening/closing state;

fig. 5C is a schematic view illustrating a fifth kind of midsplit double acting door according to an embodiment of the present invention in an opening and closing state;

FIG. 5D is a diagram illustrating an opened state of a second midsplit double door according to an embodiment of the invention;

FIG. 6A is a schematic diagram illustrating a closed state of a center door according to an embodiment of the present invention;

fig. 6B is a schematic view illustrating a first center opening door according to an embodiment of the present invention in an opening/closing state;

fig. 6C is a schematic view illustrating a second type of middle opening door according to the embodiment of the present invention in an opening/closing state;

FIG. 6D is a schematic diagram illustrating an open state of the center opening door according to an embodiment of the present invention;

FIG. 7A is a schematic view of a side-opening bi-fold door in a closed position according to an embodiment of the present invention;

FIG. 7B is a schematic view of a first side-opening bi-fold door according to an embodiment of the present invention in an open state;

FIG. 7C is a schematic view of a second side-opening bi-fold door according to an embodiment of the present invention in an open state;

FIG. 7D is a diagram illustrating a third side-opening bi-fold door according to an embodiment of the present invention in an open state;

FIG. 7E is a schematic view of an opened state of a side-opening double-folding door according to an embodiment of the present invention;

FIG. 8A is a schematic view of a closed state of a side door according to an embodiment of the present invention;

fig. 8B is a schematic diagram illustrating the opening and closing state of the side door according to the embodiment of the present invention;

FIG. 8C is a diagram illustrating an open state of a side door according to an embodiment of the present invention;

FIG. 9A is a schematic view of a closed state of a dual-leaf vertical sliding door according to an embodiment of the present invention;

fig. 9B is a schematic view of the opening and closing state of the two-leaf vertical sliding door according to the embodiment of the present invention;

FIG. 9C is a schematic view of an opened state of the double-leaf vertical sliding door according to the embodiment of the present invention;

FIG. 10A is a schematic view of a closed state of a single vertical sliding door according to an embodiment of the present invention;

fig. 10B is a schematic view illustrating an opening/closing state of a single vertical sliding door according to an embodiment of the present invention;

FIG. 10C is a schematic view of an opened state of a single-leaf vertical sliding door according to an embodiment of the present invention;

fig. 11A is a schematic view of a plurality of preset labels of a first door according to an embodiment of the present invention;

fig. 11B is a schematic view of a plurality of preset labels of a second door according to an embodiment of the present invention;

fig. 12 is a schematic view of a complete flow of a method for detecting a status of an elevator door according to an embodiment of the present invention;

fig. 13 is a schematic view of an elevator door status detecting device according to an embodiment of the present invention;

FIG. 14 is a schematic view of a robot according to an embodiment of the present invention;

fig. 15 is a schematic diagram of a computing device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Some of the words that appear in the text are explained below:

1. the term "and/or" in the embodiments of the present invention describes an association relationship of associated objects, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

2. The term "aprilat" in the embodiment of the present invention is a visual reference library, and is widely used in the fields of AR (Augmented Reality), robot, and camera calibration. By means of specific markers (similar to two-dimensional codes, but with reduced complexity to meet real-time requirements), it is possible to detect the markers quickly and calculate the relative position.

3. In the embodiment of the present invention, the term "OpenCV (Open Source Computer Vision Library)" is a cross-platform Computer Vision Library issued based on BSD (Berkeley Software suite) license (Open Source), and can be run on Linux, Windows, Android, and Mac OS operating systems. The method is light in weight and high in efficiency, is composed of a series of C functions and a small number of C + + classes, provides interfaces of languages such as Python, Ruby, MATLAB (Matrix Laboratory) and the like, and realizes a plurality of general algorithms in the aspects of image processing and computer vision.

4. The term "ARToolKit" in the embodiment of the present invention is a library written in C/C + + language, by which we can easily write augmented reality applications. AR (Augmented Reality) is a technology for overlaying virtual images of a computer into real world pictures, and has great potential in both industrial and theoretical research aspects. The most difficult part to develop an AR program is to overlay the virtual image in real time into the user's viewport and align it precisely with the objects in the real world. The ARToolKit uses computer vision techniques to calculate the relative position between the camera and the marker card, thereby enabling programmers to overlay their virtual objects onto the marker card. The rapid and accurate marker tracking provided by ARToolKit enables developers to quickly develop many newer, more interesting AR programs. The ARToolKit contains the trace libraries and the complete source code of these libraries, and developers can adjust the interfaces according to different platforms, and can also use their own trace algorithms to replace them.

5. The term "radial distortion" in the present embodiment is the change dr of the vector end point along the length direction, i.e. the change of the sagittal diameter. The radial distortion of the image is the position deviation of image pixel points generated along the radial direction by taking a distortion center as a central point, so that the image formed in the image is deformed.

6. The term "tangential distortion" in the present embodiment is the change of vector end point along the tangential direction, i.e. the change dt of angle. Tangential distortion is caused by imperfections in the camera fabrication causing the lens itself to be non-parallel to the image plane.

The application scenario described in the embodiment of the present invention is for more clearly illustrating the technical solution of the embodiment of the present invention, and does not form a limitation on the technical solution provided in the embodiment of the present invention, and it can be known by a person skilled in the art that with the occurrence of a new application scenario, the technical solution provided in the embodiment of the present invention is also applicable to similar technical problems. In the description of the present invention, the term "plurality" means two or more unless otherwise specified.

In recent years, the robot industry relying on artificial intelligence has been greatly developed, and in some application occasions, the robot may be used in a cross-floor scene, such as cross-floor user guidance, article delivery and the like, and in the scene, the robot is often based on taking an elevator to realize cross-floor operation. However, for the problem that the robot independently enters/exits the elevator, the implementation process of the internet of things communication mode in the related art is complicated.

In view of this, an embodiment of the present invention provides a method and an apparatus for detecting an elevator door, in which at least one preset tag is preset on an elevator door, an image including the elevator door is detected by using a camera, and a state of the elevator door is determined according to the number of the preset tags in the image, so that a robot can independently enter or exit the elevator.

With respect to the above scenario, the following describes an embodiment of the present invention in further detail with reference to the drawings of the specification.

As shown in fig. 1, the method for detecting the state of an elevator door according to the embodiment of the present invention specifically includes the following steps:

step 100: the robot acquires an image of an area where an appointed elevator door is located through a camera, wherein at least one preset label is arranged on the elevator door;

step 101: detecting a preset label in the image;

step 102: and determining whether the elevator door is in an opening state or not according to the number of preset labels in the image.

According to the scheme, the camera of the hardware equipment installed on the robot is fully utilized, the camera is directly used for collecting images, and the state of the elevator door is judged by detecting the number of the preset labels in the images. Only need set up necessary preset label on the lift-cabin door can, the construction is simple, and the robot need not to communicate with thing networking controller, and is more simple and convenient.

In the embodiment of the invention, the state of the elevator door is one of an opening state, a closing state and an opening and closing state, wherein the opening and closing state is a transition state between opening and closing, which means that the elevator door is in the process of opening or closing.

According to the embodiment of the invention, the robot is provided with the camera, the resolution of the camera is not lower than 640 x 480 pixels, the transverse field angle is not less than 90 degrees, and a gray image can be captured.

Because the state of the elevator door is judged based on the detection of the preset tag, the preset tag needs to be arranged on the elevator door in advance, wherein the type of the preset tag can be AprilTag; in addition to aprilat, there are also some series of tag systems developed based on artoollit, such as ARTag.

The aprilat is a visual reference label system designed by Edwin Olson et al of michigan university, and is mainly used for assisting the robot in visual navigation. The label generation algorithm and the detection algorithm are open sources, and the method has good adaptability to illumination change, partial shielding, lens distortion, different visual angles and the like. If the open source is comprehensively considered, the system is free and robust. Fig. 2A is a schematic diagram of several conventional aprilats shown in the embodiment of the present invention, which are Tag36h11, Tag standard41h12, Tag standard52h13, Tag circuit 21h7, Tag circuit 49h12 and Tag custom48h 12.

The ARTag (AR tag) is a reference mark system, can be understood as a reference object of other objects, looks like a two-dimensional code, but has a coding system which is greatly different from the two-dimensional code, is mainly used in application occasions such as camera calibration, robot positioning, augmented reality and the like, and has the main function of reflecting the pose relation of a camera and a tag. And further, the reference relation between the object in the scene and the camera can be reflected.

A common ARTag pattern is shown in fig. 2B, where the pattern represents a number, and fig. 2B is the pattern resulting from the number 1 encoding. It is a black and white rasterized image, the smallest unit is a small square, the whole ARTag is a 10 × 10 grid, the edge is composed of black squares with 2 unit thickness, the center is an effective data area of 6 × 6, black is 0, white is 1, and this represents a binary number of 36 bits. Of these 36 bits, 10 bits represent valid data, and the other 26 bits are used for error-tolerant encoding, and after a series of encoding, the ARTag tag can represent 2002 patterns, and here artificially excludes several patterns that are not easily recognized.

Aprilat, ARTag and the like mentioned in the above embodiments can be used as preset tags in the embodiments of the present invention, and are provided on elevator doors.

The elevator door is at least provided with one preset tag, and when the number of the preset tags on the elevator door is larger, the accuracy of elevator door state detection is higher. In addition, the position, the number, the size and the like of the preset label also influence the judgment of the closed state and the opening and closing state, and the preset label on the elevator door adopts AprilTag with the size of A4 paper.

Wherein, can adopt modes such as spraying, seal, paste when setting up on the lift-cabin door and predetermineeing the label.

In the embodiment of the present invention, the robot and the elevator car can be divided into two cases, i.e., "in elevator" and "out of elevator", according to the position relationship between the robot and the elevator car, and the method described in the embodiment of the present invention is applicable to both cases, and the implementation process is not different, and the following description will be given mainly by taking the robot out of elevator as an example.

In an alternative embodiment, when the robot encounters a situation that the robot takes an elevator with a guest, the robot can determine that a living obstacle exists between the robot and the elevator door, for example, the guest living in a hotel, and then the guest can be prompted by voice to give way to the robot, so that the robot and the elevator door are not shielded too much.

Specifically, a reasonably designed navigation system should guide the robot to a reasonable range close to the elevator, and then acquire an image of the area where the designated elevator door is located through a camera.

Optionally, in order to ensure the accuracy of the robot in detecting the door state of the elevator, an acquisition distance may be preset, where the acquisition distance is determined according to an acquisition angle of the camera, for example, when the acquisition angle of the camera is 120 degrees, the set acquisition distance is L1; and when the collection angle of the camera is 170 degrees, the set collection distance is L2 and the like.

When the state of a certain appointed elevator door is detected, the robot can be guided to walk to a reasonable position corresponding to the elevator door through the navigation system, for example, the robot is guided to a position which is away from the elevator door by a set acquisition distance, so that when the state of the elevator door of a certain elevator is detected, the acquired image only contains the appointed elevator door, and the state of the elevator door can be detected more accurately.

As shown in fig. 3, for the schematic diagram of the collection distances of different elevators when the robot is outside the elevator provided by the embodiment of the present invention, since it can be known that every two of elevators 1 to 3 are adjacent, every two of elevators 4 to 6 are adjacent, and the elevators 4 to 6 are located opposite to the elevators 1 to 3, assuming that the set collection distance is 1 meter of the elevator right in front of the elevator, when the images of the areas where the elevator doors 1 to 6 are located are respectively collected, the robot can be respectively located at positions a to F for collection; the same applies when the robot is in an elevator, depending on the actual situation.

When the image of the area where the appointed elevator door is located is collected, when the robot is located at the position A, the robot only comprises the elevator door of the elevator No. 1 in the image collected by the elevator door of the elevator No. 1 in the direction towards the elevator No. 1 through the camera, the elevator doors of other elevators such as the elevator No. 2 and the elevator No. 3 cannot appear in the image, and therefore the accuracy of elevator door state detection can be improved.

Optionally, after the image including the elevator door is acquired, the image is preprocessed, specifically including distortion correction and/or denoising.

Wherein, the distortion correction is mainly divided into radial distortion and tangential distortion, and can be realized by adopting OpenCV;

denoising refers to removing noise points in an image through image filtering, and common image denoising methods include: median filtering, mean filtering, gaussian filtering, soft threshold method based on wavelet transform, hard threshold method, etc.

After the image is preprocessed, an AprilTag detection model is called to process the image, and the number of preset labels contained in the image is detected.

In the embodiment of the invention, if the number of the preset tags in the image is 0, the elevator door is determined to be in the open state.

Specific time division also needs to be determined according to the position, the number, the size and the like of the preset label aiming at the closed state and the opening and closing state.

An alternative embodiment is that the elevator door comprises at least one preset label on each of the left and right sides of each door.

Common elevators are split and side-by-side. Generally, the center opening door is in a split mode, door leaves are split in a centering mode, opening and closing efficiency is high, and the center opening door is commonly used for passenger elevators and passenger and goods elevators, as shown in fig. 4A, 5A and 6A; the side door is generally used in a side opening manner, in which two or more door leaves are stacked and stored in one direction, and is commonly used in a freight elevator and a hospital bed elevator, as shown in fig. 7A and 8A.

The side door and the middle door can be respectively folded or unfolded. For example, as shown in fig. 4A, fig. 5A and fig. 7A, wherein fig. 4A and fig. 5A are a center-opening double-folding door, which is composed of 4 door panels, two sides of which are opened and closed, and two door panels on the same side are folded by a door arm; fig. 7A shows a side-opening double-folding door, which is composed of two doors, one side of which is opened and closed.

It should be noted that the folding elevator door may be in a form of three folding or the like in addition to the double folding, and the present invention is mainly described in detail by taking the double folding as an example.

The change situation of the number of the preset labels of the elevator doors in the forms is described in detail below in the process of opening and closing the elevator:

the first form is a center-split double-folding door.

An optional preset tag setting mode is that 8 aprilats are respectively stuck on the inner side and the outer side of the elevator door and used for detecting the state of the elevator door when the robot is in the elevator or out of the elevator, as shown in fig. 4A, wherein L1 and R1 are respectively stuck on the left and right side edges of the door No. 1, L2 and R2 are respectively stuck on the left and right side edges of the door No. 2, L3 and R3 are respectively stuck on the left and right side edges of the door No. 3, and L4 and R4 are respectively stuck on the left and right side edges of the door No. 4. Wherein the size of the predetermined label is about a4 paper size. Fig. 4A to 4E show several possible situations where the number of preset tags in the image changes during the opening process of the elevator door.

It should be noted that, in the embodiment of the present invention, for the preset tag disposed at the edge of the elevator door, the distance between the preset tag and the edge of the elevator door can be ensured to be within a preset range, wherein a value of the preset range is related to a size of the preset tag, and generally, the smaller the preset tag is, the smaller the preset range is.

Taking fig. 4A as an example, after the robot is located at the acquisition distance corresponding to the elevator door shown in the figure, the camera acquires an image of the area where the elevator door is located, wherein the image should cover all aprilat when acquiring the image.

After the image is preprocessed, an aprilatag detection model is called to process the image, and eight tags stuck to the elevator door in fig. 4A are detected. Wherein the detector is designed to be open source.

As can be seen from fig. 4A, there are 8 preset tags in the image, where N is 8, and the elevator door is in a closed state.

As shown in fig. 4B to 4D, the elevator doors are all in the open/close state. Wherein the number of preset tags in fig. 4B is 6, 0<6< N; the number of preset tags in fig. 4C is 4, 0<4< N; the number of preset tags in fig. 4D is 2, 0<2< N.

As shown in fig. 4E, in which the number of the preset tags is 0, the elevator door is in an open state.

Alternatively, fig. 5A shows another preset label arrangement in the midsplit double door, where L1 is arranged at the left side edge of the elevator door No. 1, L2 is arranged at the right side edge of the elevator door No. 2, and R2 and R1 are also arranged at the left side edge of the elevator door No. 3 and the right side edge of the elevator door No. 4, respectively.

As can be seen from fig. 5A, there are 4 preset tags in the image, where N is 4, and the elevator door is in a closed state.

As shown in fig. 5B to 5C, the elevator doors are all in the open/close state. Wherein, the number of the preset tags in fig. 5B is 2, the number of the preset tags in fig. 5C is 2, and 0<2< N.

As shown in fig. 5D, in which the number of the preset tags is 0, the elevator door is in an open state.

And the second form is no folding center opening door.

When the elevator door is a folding-free center opening door, as shown in fig. 6A to 6D, there are several possible situations that the number of the preset tags in the image changes during the opening process of the elevator door, fig. 6A shows that 4 preset tags exist in the image when the elevator door is in a closed state, fig. 6B and 6C show that 2 preset tags exist in the image when the elevator door is in an open state, and fig. 6D shows that the number of the preset tags in the image is 0 when the elevator door is in an open state.

Taking the elevator door shown in fig. 5A as an example, considering that when the elevator operates normally, two elevator doors are both opened and closed simultaneously, and therefore, when the tags are set, the tags can also be set only on one elevator door, in this way, during detection, if 2 preset tags exist in the image, it is determined that the elevator doors are in the closed state, if 1 preset tag exists in the image, it is determined that the elevator doors are in the open and closed state, and if the number of the preset tags in the image is 0, it is determined that the elevator doors are in the open state.

It should be noted that, if "only one side of the door is provided with the preset tag", the detection of the state of the elevator door can be realized, but considering that the elevator fault causes the door to be jammed, the detection is more reliable, and the reliability of the detection result can be improved by the redundant design of more preset tags arranged on the elevator door.

In the present embodiment, the "label only on one side door" covers 90% of the scene, but considering that some elevators have only one side door, the customization process is performed separately.

Elevators with only one elevator door, such as freight elevators, can also be classified as being foldable and non-foldable.

And the third form is a side-opening double-folding door.

As shown in fig. 7A, in order to set up a preset tag, a preset tag is set up at each of the left and right side edges of the elevator door No. 1 and No. 2, which are L1 and R1; l2, R2, N ═ 4.

To the setting of the label of presetting of this mode, when judging the state of lift-cabin door:

if the image has four preset tags, determining that the elevator door is in a closed state, as shown in fig. 7A;

if the number of the preset tags in the image is 0, determining that the elevator door is in an open state, as shown in fig. 7E;

if there are 3 preset tags in the image: l1, R1, L2, 0<3< N, it can be determined that the elevator door is in an open/close state, as shown in fig. 7B;

if there are 2 preset tags in the image: l1, R1, 0<2< N, it can be determined that the elevator door is in an open/close state, as shown in fig. 7C;

if there are 1 preset tags L1 in the image, 0<1< N, it can be determined that the elevator door is in the open/close state, as shown in fig. 7D.

And the fourth mode is that the folding side door is not provided.

As shown in fig. 8A, a preset label is set on each of the left and right edges of the elevator door, where L and R are provided, and N is 2.

As can be seen from fig. 8A, there are 2 preset tags in the image, where N is 2, and the elevator door is in a closed state.

As shown in fig. 8B, where the number of the preset tags is 1, 0<1< N, the elevator door is in an opening and closing state. ,

as shown in fig. 8C, in which the number of the preset tags is 0, the elevator door is in an open state.

In addition, a vertical sliding door that is opened and closed in a vertical direction is generally used in commercial and industrial applications, as shown in fig. 9A and 10A. Wherein, fig. 9A shows a double-leaf vertical sliding door (split type), the split door leaf is opened in the middle, and respectively moves to the upper and lower of the door; fig. 10A shows a single vertically sliding door with the elevator door opening by moving up the doorway.

In the elevator door shown in fig. 9A, the upper and lower edges of the upper door and the lower door are respectively provided with a preset label, which is U1 and U2; d2 and D1. Wherein N is 4.

As can be seen from fig. 9A, there are 4 preset tags in the image, where N is 4, and the elevator door is in a closed state.

As shown in fig. 9B, where the number of the preset tags is 2, 0<2< N, the elevator door is in an opening and closing state. ,

as shown in fig. 9C, in which the number of the preset tags is 0, the elevator door is in an open state.

A predetermined label U, D is provided on each of the upper and lower edges of the elevator door shown in fig. 10A. Wherein N is 2.

As can be seen from fig. 10A, there are 2 preset tags in the image, where N is 2, and the elevator door is in a closed state.

As shown in fig. 10B, where the number of the preset tags is 1, 0<1< N, the elevator door is in an opening and closing state. ,

as shown in fig. 10C, in which the number of the preset tags is 0, the elevator door is in an open state.

That is, the elevator door is provided with N preset labels, wherein N is a positive integer greater than 1; if the number of the preset labels in the image is N, the robot determines that the elevator door is in a closed state; if the number of the preset labels in the image is larger than 0 and smaller than N, the robot determines that the elevator door is in an opening and closing state; if the number of the preset labels in the image is 0, the robot determines that the elevator door is in an open state.

It should be noted that, when the elevator door that slides perpendicularly also can be divided into folding and not have folding, the specific judgement mode is similar with the judgement mode of the elevator door of horizontal direction switch, and the difference is set to the label setting position only, and specific conditions are no longer repeated one by one.

Optionally, several preset labels can be additionally arranged on the elevator door. As shown in fig. 11A and 11B, this method can improve the accuracy of elevator door state detection.

It should be noted that the several cases listed in the embodiment of the present invention that the states of the elevator door are distinguished according to the number of the preset tags in the image are only examples, and the specific implementation still needs to be adjusted according to the size, the position, and the like of the preset tags, which are not listed.

The elevator door can be provided with the preset label through spraying, engraving, pasting and other modes, wherein the preset label can fall off in the pasting mode.

In the embodiment of the present invention, the robot can also detect whether the elevator door is abnormal, wherein there are many abnormal situations, two of which are listed below:

in the abnormal condition I, the duration of the elevator door in the opening and closing state exceeds the preset duration.

For example, the preset time is 5 minutes, and if the elevator door is in the open-close state for 10 minutes, that is, the elevator door is in the open-close state for a long time, it can be determined as abnormal.

And in the abnormal condition II, a certain preset label pasted on the elevator door falls off in the image.

As shown in fig. 11B, in which the tag 2 is detached, it can be detected according to the number and position of the tags in the image, for example, if the

tags

1 and 3 are detected, but the tag 2 is not detected, it can be considered that the tag 2 is detached, and in this case, it can be determined that the elevator door is abnormal.

In the embodiment of the invention, after the robot detects the abnormality of the elevator door, other elevators are searched for detection, and the detection can be prompted in an alarm mode.

It should be noted that, in the embodiment of the present invention, when the state of the elevator door is determined according to the number of the preset tags in the image, the number refers to the number of the detected complete preset tags.

As shown in fig. 12, a complete method for detecting an elevator door device according to an embodiment of the present invention includes:

step 1200, the robot judges whether the distance between the robot and the appointed elevator door is a set acquisition distance, if so, the step 1202 is executed, otherwise, the step 1201 is executed;

step 1201, the robot moves to a reasonable position according to the set acquisition distance and the current position;

step 1202, the robot collects an image of an area where the designated elevator door is located;

step 1203, performing distortion correction and denoising processing on the acquired image by the robot;

step 1204, detecting a preset label in the processed image by the robot;

step 1205, the robot judges whether the image has four preset labels of L1, L2, R1 and R2, if so, the step 1206 is executed, otherwise, the step 1207 is executed;

step 1206, the robot determines that the elevator door is in a door closing state;

step 1207, the robot judges whether any preset label does not exist in the image, if so, step 1208 is executed, otherwise, step 1209 is executed;

step 1208, the robot determines that the elevator door is in an open state;

step 1209, the robot determines that the elevator door is in an open/close state.

It should be noted that, the steps 1206 to 1209 are only one feasible execution sequence, and the robot may also determine whether there is no preset tag in the image, and further determine whether there are four preset tags, i.e., L1, L2, R1, and R2, if any.

Based on the same inventive concept, the embodiment of the invention also provides a device for detecting the state of the elevator door, and as the device is a device corresponding to the method in the embodiment of the invention and the principle of solving the problem of the device is similar to the method, the implementation of the device can refer to the implementation of the method, and repeated details are not repeated.

As shown in fig. 13, an elevator door status detecting device 1300 according to an embodiment of the present invention includes: the acquisition unit 1301, the detection unit 1302 and the first determination unit 1303:

the collecting unit 1301 is used for collecting an image of an area where an appointed elevator door is located through a camera, wherein at least one preset label is arranged on the elevator door;

a detecting unit 1302, configured to detect a preset tag in the image;

the first determining unit 1303 is configured to determine whether the elevator door is in an open state according to the number of preset tags in the image.

Optionally, the apparatus further comprises:

the preprocessing unit 1304 is configured to, after the acquisition unit 1301 acquires an image of an area where the specified elevator door is located through a camera, perform distortion correction and/or denoising on the image before the detection unit 1302 detects a preset tag in the image.

Optionally, the first determining unit 1303 is specifically configured to:

the first determining unit 1303 is specifically configured to:

Optionally, the apparatus further comprises:

the second determining unit 1305 is configured to determine that the elevator door is abnormal if the duration of the elevator door being in the open-close state exceeds a preset duration.

Optionally, the apparatus further comprises:

a third determining unit 1306, configured to determine, before the acquiring unit acquires the image of the area where the specified elevator door is located through the camera, that the distance between the elevator door and the acquiring unit is a set acquiring distance, where the acquiring distance is determined according to an acquiring angle of the camera.

Based on the same inventive concept, the embodiment of the present invention further provides a robot 1400, as shown in fig. 14, the robot includes: a processor 1401, a memory 1402, wherein said memory 1402 stores program code, which when executed by said processor 1401 causes the processor to perform the following:

acquiring an image of an area where an appointed elevator door is located through a camera, wherein at least one preset label is arranged on the elevator door;

detecting a preset label in the image;

Optionally, the processor 1401 is further configured to:

after the image of the area where the appointed elevator door is located is collected through the camera, distortion correction and/or denoising processing is carried out on the image before a preset label in the image is detected.

Optionally, the processor 1401 is specifically configured to:

the processor 1401 is further configured to:

Optionally, the processor 1401 is further configured to:

before the camera collects the image of the area where the appointed elevator door is located, the distance between the camera door and the elevator door is determined to be a set collection distance, wherein the collection distance is determined according to the collection angle of the camera.

An embodiment of the present invention further provides a computer-readable non-volatile storage medium, which includes program codes, and when the program codes are run on a processor, the steps of any one of the elevator door state detection methods in the embodiments of the present invention described above are implemented.

In another embodiment of the present invention, a computing device is also provided. The computing device 150 shown in fig. 15 is only an example and should not bring any limitations to the functionality or scope of use of embodiments of the present invention.

As in fig. 15, the computing device 150 is embodied in the form of a general purpose computing device. Components of computing device 150 may include, but are not limited to: at least one processing unit 151, at least one memory unit 152, and a bus 153 that couples various system components including the memory unit 152 and the processing unit 151.

The processing unit 151 may implement the functions of the processor 1401, and the storage unit 152 may implement the functions of the memory 1402, i.e. the robot 1400 may be implemented by using the specific structure of the computing device 150.

Bus 153 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

The storage unit 152 may include readable media in the form of volatile memory, such as Random Access Memory (RAM)1521 and/or cache memory unit 1522, and may further include Read Only Memory (ROM) 1523.

The storage unit 152 may also include a program/utility 1525 having a set (at least one) of program modules 1524, such program modules 1524 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

The computing device 150 may also communicate with one or more external devices 154 (e.g., keyboard, pointing device, etc.), may also communicate with one or more devices that enable a user to interact with the computing device 150, and/or may communicate with any devices (e.g., router, modem, etc.) that enable the computing device 150 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 155. Also, the computing device 150 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) through the network adapter 156. As shown, the network adapter 156 communicates with other modules for the computing device 150 over the bus 153. It should be understood that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the computing device 150, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

In some possible embodiments, the various aspects of the method for elevator door state detection provided by the present invention may also be implemented in the form of a program product comprising program code for causing a computer device to perform the steps in the method for elevator door state detection according to various exemplary embodiments of the present invention described above in this specification when the program product is run on a computer device, for example the computer device may perform the steps as shown in fig. 12.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The program product for elevator door state detection of embodiments of the present invention may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a computing device. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with a command execution system, apparatus, or device.

A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with a command execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user computing device, partly on the user equipment, as a stand-alone software package, partly on the user computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

The present invention is described above with reference to block diagrams and/or flowchart illustrations of methods, apparatus (systems) and/or computer program products according to embodiments of the invention. It will be understood that one block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, and/or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer and/or other programmable data processing apparatus, create means for implementing the functions/acts specified in the block diagrams and/or flowchart block or blocks.

Accordingly, the present invention may also be embodied in hardware and/or in software (including firmware, resident software, micro-code, etc.). Furthermore, the invention can take the form of a computer program product on a computer-usable or computer-readable storage medium having computer-usable or computer-readable program code embodied in the medium for use by or in connection with an instruction execution system. In the context of this document, a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims

1. A method of detecting a status of an elevator door, the method comprising:

the robot acquires an image of an area where an appointed elevator door is located through a camera, wherein N preset labels are arranged on the elevator door, and N is a positive integer greater than 1;

detecting a preset label in the image;

determining whether the elevator door is in an opening state or not according to the number of preset labels in the image;

wherein, according to the number of the preset labels in the image, determining whether the elevator door is in an open state comprises:

2. The method of claim 1, wherein after the robot captures an image of an area where the specified elevator door is located through a camera, and before detecting the preset tag in the image, the method further comprises:

and carrying out distortion correction and/or denoising treatment on the image.

3. The method of claim 1, wherein determining whether the elevator door is in an open state based on the number of preset tags in the image comprises:

4. The method of claim 1, wherein determining whether the elevator door is in an open state based on a number of preset tags in the image, further comprises:

and if the number of the preset labels in the image is N, determining that the elevator door is in a closed state.

5. The method of claim 1, wherein the method further comprises:

6. The method of claim 1, wherein the elevator door includes at least one preset tag provided on each of left and right side edges of each door; and/or

7. The method of any of claims 1 to 6, wherein before the robot captures an image of the area where the designated elevator door is located via a camera, the method further comprises:

8. An elevator door state detecting device, comprising:

the system comprises an acquisition unit, a monitoring unit and a control unit, wherein the acquisition unit is used for acquiring an image of an area where a designated elevator door is located through a camera, N preset tags are arranged on the elevator door, and N is a positive integer greater than 1;

the detection unit is used for detecting a preset label in the image;

the first determining unit is used for determining whether the elevator door is in an opening state or not according to the number of preset labels in the image;

wherein the first determining unit is specifically configured to:

9. A robot characterized by comprising a processor and a memory, wherein the memory stores program code which, when executed by the processor, carries out the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 7.