CN115215185B - Elevator door closing control method, system, device and medium based on machine vision - Google Patents
Elevator door closing control method, system, device and medium based on machine vision Download PDFInfo
- Publication number
- CN115215185B CN115215185B CN202210809280.5A CN202210809280A CN115215185B CN 115215185 B CN115215185 B CN 115215185B CN 202210809280 A CN202210809280 A CN 202210809280A CN 115215185 B CN115215185 B CN 115215185B
- Authority
- CN
- China
- Prior art keywords
- elevator
- image information
- peer
- door closing
- historical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000009471 action Effects 0.000 claims abstract description 44
- 238000012549 training Methods 0.000 claims description 73
- 230000006399 behavior Effects 0.000 claims description 49
- 238000013527 convolutional neural network Methods 0.000 claims description 23
- 230000003993 interaction Effects 0.000 claims description 10
- 238000012360 testing method Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims 2
- 230000006870 function Effects 0.000 description 28
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000006467 substitution reaction Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008093 supporting effect Effects 0.000 description 1
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B13/00—Doors, gates, or other apparatus controlling access to, or exit from, cages or lift well landings
- B66B13/02—Door or gate operation
- B66B13/14—Control systems or devices
- B66B13/143—Control systems or devices electrical
- B66B13/146—Control systems or devices electrical method or algorithm for controlling doors
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/0006—Monitoring devices or performance analysers
- B66B5/0012—Devices monitoring the users of the elevator system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B5/00—Applications of checking, fault-correcting, or safety devices in elevators
- B66B5/0006—Monitoring devices or performance analysers
- B66B5/0037—Performance analysers
Abstract
The invention discloses a door closing control method, a door closing control system, a door closing control device and a door closing control medium for an elevator based on machine vision, wherein the method comprises the following steps: acquiring first image information, wherein the first image information comprises at least one of real-time image information in an elevator car and real-time image information outside an elevator hall; inputting the first image information into a pre-trained peer person identification model to obtain a peer person identification result; and controlling the door closing action of the elevator according to the identification result of the staff. According to the elevator car door closing control method, the control of the door closing action is carried out by identifying the fellow staff, so that the fellow staff can leave or enter the elevator car together, on one hand, the fellow staff is prevented from independently taking an elevator, the safety of taking the elevator is improved, on the other hand, the door opening button is not required to be pressed continuously or repeatedly, the elevator taking time of passengers is saved, and the elevator taking efficiency and elevator taking experience of the passengers are improved. The invention can be widely applied to the technical field of elevator control.
Description
Technical Field
The invention relates to the technical field of elevator control, in particular to an elevator door closing control method, system, device and medium based on machine vision.
Background
In elevator systems, the door closing time of the elevator car is generally preset, that is, when no person continuously presses a call button, the elevator car automatically closes after waiting for the preset door closing time, and if many persons are waiting for the elevator at the same time, the phenomenon that passengers waiting for the elevator automatically close the door without entering the elevator may be caused. Particularly in the scene that parents take the elevator together with the child, the child generally enters the elevator first, and the parents immediately follow the elevator, if the parents do not enter the elevator in time and the child does not continuously press the door opening button in the elevator car, the elevator door can be automatically closed at the moment, so that the child takes the elevator alone, and a large potential safety hazard exists. In addition, the elevator can wait for the preset door closing time and then automatically close after arriving at the landing floor to open the door, if more passengers exist in the elevator car, the phenomenon that the passengers of the same person can not go out of the elevator and then close the door can possibly exist, and at the moment, the door opening button needs to be pressed down again to open the door again, so that the time of the passengers is wasted, and the elevator taking efficiency and the elevator taking experience of the passengers are influenced.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems existing in the prior art to a certain extent.
Therefore, an object of the embodiments of the present invention is to provide a door closing control method for an elevator based on machine vision, which improves the elevator taking efficiency and elevator taking experience of passengers by identifying control of door closing actions of peers.
It is another object of an embodiment of the present invention to provide a door closing control system for an elevator based on machine vision.
In order to achieve the technical purpose, the technical scheme adopted by the embodiment of the invention comprises the following steps:
in a first aspect, an embodiment of the present invention provides a door closing control method for an elevator based on machine vision, including the following steps:
acquiring first image information, wherein the first image information comprises at least one of real-time image information in an elevator car and real-time image information outside an elevator hall;
inputting the first image information into a pre-trained peer person identification model to obtain a peer person identification result;
and controlling the door closing action of the elevator according to the identification result of the staff.
Further, in one embodiment of the present invention, the door closing control method of the elevator further includes a step of training a peer identification model in advance, which specifically includes:
acquiring second image information, wherein the second image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
performing face recognition on the second image information to determine a plurality of second target persons, and acquiring historical elevator taking behavior information of the second target persons, wherein the historical elevator taking behavior information comprises a historical elevator taking section and historical elevator taking time;
determining the peer relationship information of the second target person according to the historical elevator taking behavior information, and marking the second image information according to the peer relationship information to obtain a peer person tag;
and determining a first training data set according to the second image information and the peer personnel labels, and inputting the first training data set into a first convolutional neural network constructed in advance for training to obtain a trained peer personnel identification model.
Further, in one embodiment of the present invention, the door closing control method of the elevator further includes a step of training a peer identification model in advance, which specifically includes:
acquiring second image information, wherein the second image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
performing face recognition on the second image information to determine a plurality of second target persons, and performing human behavior recognition on the second image information to determine interaction information of the second target persons;
determining the peer relationship information of the second target person according to the interaction information, and marking the second image information according to the peer relationship information to obtain a peer person tag;
and determining a first training data set according to the second image information and the peer personnel labels, and inputting the first training data set into a first convolutional neural network constructed in advance for training to obtain a trained peer personnel identification model.
Further, in an embodiment of the present invention, the step of inputting the first training data set into a first convolutional neural network constructed in advance to perform training, and obtaining a trained peer identification model specifically includes:
inputting the first training data set into a first convolutional neural network constructed in advance to obtain a peer prediction result;
determining a loss value of the first convolutional neural network according to the peer prediction result and the peer label;
updating parameters of the first convolutional neural network through a back propagation algorithm according to the loss value;
and stopping training when the loss value reaches a preset first threshold value or the iteration number reaches a preset second threshold value or the test precision reaches a preset third threshold value, and obtaining a trained peer identification model.
Further, in one embodiment of the present invention, the step of controlling the door closing action of the elevator according to the identification result of the staff specifically includes:
determining a plurality of peer groups according to the peer identification result, wherein the peer groups comprise at least two first target persons with peer relations;
when the first target person is detected to leave the elevator car, keeping the elevator open until all members of the corresponding peer person group leave the elevator car, and controlling the elevator to perform door closing action;
or alternatively, the first and second heat exchangers may be,
when the first target person is detected to enter the elevator car, keeping the elevator open until all members of the corresponding peer person group enter the elevator car, and controlling the elevator to perform door closing action.
Further, in one embodiment of the present invention, the step of controlling the door closing action of the elevator according to the identification result of the staff further includes the following steps:
inputting the first image information into a pre-trained riding behavior intention recognition model to obtain a behavior intention recognition result;
when the first intention of leaving the elevator car of the first target person is identified, keeping the elevator open until all members of the corresponding peer personnel group leave the elevator car, and controlling the elevator to perform door closing action;
or alternatively, the first and second heat exchangers may be,
and when the first target person is identified to have the second action intention of entering the elevator car, keeping the elevator open until all members of the corresponding peer person group enter the elevator car, and controlling the elevator to perform door closing action.
Further, in one embodiment of the present invention, the door closing control method of the elevator further includes a step of training an intention recognition model of riding behavior in advance, which specifically includes:
acquiring third image information, wherein the third image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
performing face recognition on the third image information to determine a plurality of third target persons, and performing human body posture recognition on the third image information to determine human body posture information of the third target persons;
determining the riding behavior intention of the third target person according to the human body posture information, and marking the third image information according to the riding behavior intention to obtain a behavior intention label;
and determining a second training data set according to the third image information and the behavior intention label, and inputting the second training data set into a second convolutional neural network constructed in advance for training to obtain a trained riding behavior intention recognition model.
In a second aspect, an embodiment of the present invention provides a door closing control system for an elevator based on machine vision, including:
the system comprises an image acquisition module, a display module and a display module, wherein the image acquisition module is used for acquiring first image information, and the first image information comprises at least one of real-time image information in an elevator car and real-time image information outside an elevator hall;
the peer person identification module is used for inputting the first image information into a pre-trained peer person identification model to obtain a peer person identification result;
and the door closing action control module is used for controlling the door closing action of the elevator according to the identification result of the staff.
In a third aspect, an embodiment of the present invention provides a door closing control device for an elevator based on machine vision, including:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement a machine vision based elevator door closing control method as described above.
In a fourth aspect, an embodiment of the present invention further provides a computer readable storage medium having stored therein a processor executable program for performing the above-described machine vision-based door closing control method for an elevator when executed by a processor.
The advantages and benefits of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
The embodiment of the invention acquires real-time image information in an elevator car or real-time image information outside an elevator hall, inputs the real-time image information into a pre-trained peer person identification model to obtain a peer person identification result, and then controls the door closing action of the elevator according to the peer person identification result. According to the embodiment of the invention, the control of the door closing action is performed by identifying the fellow staff, so that the fellow staff can leave or enter the elevator car together, on one hand, the fellow staff is prevented from taking the elevator independently, the safety of taking the elevator is improved, on the other hand, the door opening button is not required to be pressed continuously or repeatedly, the elevator taking time of passengers is saved, and the elevator taking efficiency and elevator taking experience of the passengers are improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following description will refer to the drawings that are needed in the embodiments of the present invention, and it should be understood that the drawings in the following description are only for convenience and clarity to describe some embodiments in the technical solutions of the present invention, and other drawings may be obtained according to these drawings without any inventive effort for those skilled in the art.
Fig. 1 is a flow chart of steps of a door closing control method for an elevator based on machine vision according to an embodiment of the present invention;
fig. 2 is a block diagram of an elevator door closing control system based on machine vision according to an embodiment of the present invention;
fig. 3 is a block diagram of a door closing control device for an elevator based on machine vision according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention. The step numbers in the following embodiments are set for convenience of illustration only, and the order between the steps is not limited in any way, and the execution order of the steps in the embodiments may be adaptively adjusted according to the understanding of those skilled in the art.
In the description of the present invention, the plurality means two or more, and if the description is made to the first and second for the purpose of distinguishing technical features, it should not be construed as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the precedence of the indicated technical features. Furthermore, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.
Referring to fig. 1, an embodiment of the invention provides a door closing control method for an elevator based on machine vision, which specifically includes the following steps:
s101, acquiring first image information, wherein the first image information comprises at least one of real-time image information in an elevator car and real-time image information outside an elevator hall.
Specifically, the first image information can be obtained by setting a camera in the elevator car and/or outside the elevator hall door to take a snapshot, and also can be obtained by capturing image frames in a monitoring video stream in the elevator car and/or outside the elevator hall door.
It can be understood that when the first image information is real-time image information in the elevator car, the following identification is performed on the staff in the elevator car; when the first image information is real-time image information outside the elevator hall, identifying the same person outside the elevator hall in the follow-up process; in some alternative embodiments, real-time image information within the elevator car and real-time image information outside the hoistway door may be acquired simultaneously to identify both the personnel in the elevator car and outside the hoistway door.
S102, inputting the first image information into a pre-trained peer person identification model to obtain a peer person identification result.
Specifically, the peer person recognition model can be obtained through training of a neural network model, and after the first image information is input into the peer person recognition model, peer person recognition results output by the model, namely a plurality of target persons with peer relationship in the first image information, can be obtained.
In the embodiment of the invention, training of the identification model of the staff in the same person has two implementation modes, and the following description is given one by one.
As an alternative embodiment, the door closing control method of the elevator further comprises a step of pre-training a peer identification model, which specifically comprises:
a1, acquiring second image information, wherein the second image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
a2, carrying out face recognition on the second image information to determine a plurality of second target personnel, and acquiring historical elevator taking behavior information of the second target personnel, wherein the historical elevator taking behavior information comprises a historical elevator taking section and historical elevator taking time;
a3, determining the peer relationship information of the second target personnel according to the historical elevator taking behavior information, and marking the second image information according to the peer relationship information to obtain peer personnel labels;
and A4, determining a first training data set according to the second image information and the peer personnel labels, and inputting the first training data set into a first convolutional neural network constructed in advance for training to obtain a trained peer personnel identification model.
Specifically, in the embodiment of the invention, the historical image information in the elevator car and/or the historical image information outside the elevator hall door are obtained, a plurality of second target persons are determined by face recognition, and for each second target person, the historical elevator taking behavior information of the second target person, such as a starting floor and a destination floor (i.e. a historical elevator taking section) and an elevator taking time period (i.e. a historical elevator taking time), can be determined according to the obtained historical image information; determining whether two second target persons in the same historical image information have a peer relationship according to the historical elevator riding behavior information, and marking the second image information according to the obtained peer relationship information to obtain peer person labels; and determining a first training data set for training the identification model of the same person according to the second image information and the label of the same person.
As another alternative embodiment, the door closing control method of the elevator further comprises a step of pre-training a recognition model of the staff, which specifically comprises:
b1, acquiring second image information, wherein the second image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
b2, carrying out face recognition on the second image information to determine a plurality of second target persons, and carrying out human behavior recognition on the second image information to determine interaction information of the second target persons;
b3, determining the peer relationship information of the second target person according to the interaction information, and marking the second image information according to the peer relationship information to obtain a peer person label;
and B4, determining a first training data set according to the second image information and the peer personnel labels, and inputting the first training data set into a first convolutional neural network constructed in advance for training to obtain a trained peer personnel identification model.
Specifically, in the embodiment of the invention, the historical image information in the elevator car and/or the historical image information outside the elevator hall door are acquired, the face recognition is firstly carried out on the historical image information to determine a plurality of second target persons, and for each second target person, the interaction action information of the second target person and other second target persons, such as talking action, hand pulling action and supporting action, is determined through the existing human body action recognition technology; determining whether two second target persons in the same historical image information have a peer relationship according to the interaction information, and marking the second image information according to the obtained peer relationship information to obtain peer person labels; and determining a first training data set for training the identification model of the same person according to the second image information and the label of the same person.
Further as an optional implementation manner, the step of inputting the first training data set into a first convolutional neural network constructed in advance to train and obtain a trained peer identification model specifically includes:
c1, inputting a first training data set into a first convolutional neural network constructed in advance to obtain a peer prediction result;
c2, determining a loss value of the first convolutional neural network according to the peer prediction result and the peer label;
c3, updating parameters of the first convolutional neural network through a back propagation algorithm according to the loss value;
and C4, stopping training when the loss value reaches a preset first threshold value or the iteration number reaches a preset second threshold value or the test precision reaches a preset third threshold value, and obtaining a trained peer personnel identification model.
Specifically, after the data in the first training data set is input into the initialized first convolutional neural network, a peer prediction result output by the model can be obtained, and the accuracy of the peer identification model can be evaluated according to the peer prediction result and the peer label, so that parameters of the model are updated. For the peer identification model, the accuracy of the model identification result can be measured by a Loss Function (Loss Function), wherein the Loss Function is defined on single training data and is used for measuring the prediction error of one training data, and particularly determining the Loss value of the training data through the label of the single training data and the prediction result of the model on the training data. In actual training, one training data set has a lot of training data, so that a Cost Function (Cost Function) is generally adopted to measure the overall error of the training data set, and the Cost Function is defined on the whole training data set and is used for calculating the average value of the prediction errors of all the training data, so that the prediction effect of the model can be better measured. For a general machine learning model, based on the cost function, a regular term for measuring the complexity of the model can be used as a training objective function, and based on the objective function, the loss value of the whole training data set can be obtained. There are many kinds of common loss functions, such as 0-1 loss function, square loss function, absolute loss function, logarithmic loss function, cross entropy loss function, etc., which can be used as the loss function of the machine learning model, and will not be described in detail herein. In the embodiment of the application, one loss function can be selected to determine the loss value of training. Based on the trained loss value, updating the parameters of the model by adopting a back propagation algorithm, and iterating for several rounds to obtain the trained peer identification model. Specifically, the number of iteration rounds may be preset, or training may be considered complete when the test set meets the accuracy requirements.
And S103, controlling the door closing action of the elevator according to the identification result of the staff.
Specifically, after the identification model of the same person determines the same person in the elevator car and/or outside the elevator hall door, the elevator can be controlled to perform door closing action after the same person is determined to leave and/or enter the elevator car together.
Further as an optional implementation manner, the step of controlling the door closing action of the elevator according to the identification result of the staff specifically comprises the following steps:
s1031, determining a plurality of peer groups according to peer identification results, wherein each peer group comprises at least two first target persons with peer relations;
s1032, when the first target person is detected to leave the elevator car, keeping the elevator open until all members of the corresponding peer group leave the elevator car, and controlling the elevator to perform door closing action;
or alternatively, the first and second heat exchangers may be,
s1033, when the first target person is detected to enter the elevator car, keeping the elevator open until all members of the corresponding peer person group enter the elevator car, and controlling the elevator to perform door closing action.
Specifically, the first target person can be continuously monitored through the camera, and when the first target person leaves/enters the elevator car, the situation that the same person leaves/enters the elevator car and then the door is closed is ensured.
Further as an alternative embodiment, the step of controlling the door closing action of the elevator according to the identification result of the staff, further comprises the following steps:
s1034, inputting the first image information into a pre-trained riding behavior intention recognition model to obtain a behavior intention recognition result;
s1035, when the first intention of the first target person to leave the elevator car is identified, keeping the elevator open until all members of the corresponding peer person group leave the elevator car, and controlling the elevator to perform door closing action;
or alternatively, the first and second heat exchangers may be,
and S1036, when the first target person is identified to have the second action intention of entering the elevator car, keeping the elevator open until all members of the corresponding peer person group enter the elevator car, and controlling the elevator to perform door closing action.
Specifically, when more passengers in the elevator car or elevator hall door outside elevator waiting personnel are more, there may be a situation that the passengers do not leave/enter the elevator car in time, in order to avoid the situation, the behavior intention recognition can be performed on the target personnel, namely, the first image information is input into a pre-trained elevator taking behavior intention recognition model to judge whether the target personnel have the behavior intention of leaving/entering the elevator car, and when the behavior intention of leaving/entering the elevator car exists, the elevator is controlled to leave/enter the elevator car in the whole of the passengers, and then the door closing action is performed.
Further as an optional embodiment, the elevator door closing control method further includes a step of pre-training an intention recognition model of riding behavior, which specifically includes:
d1, acquiring third image information, wherein the third image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
d2, carrying out face recognition on the third image information to determine a plurality of third target persons, and carrying out human body posture recognition on the third image information to determine the human body posture information of the third target persons;
d3, determining the riding behavior intention of the third target person according to the human body posture information, and marking the third image information according to the riding behavior intention to obtain a behavior intention label;
and D4, determining a second training data set according to the third image information and the behavior intention label, and inputting the second training data set into a second convolutional neural network constructed in advance for training to obtain a trained riding behavior intention recognition model.
Specifically, in the embodiment of the invention, the historical image information in the elevator car and/or the historical image information outside the elevator hall are acquired, the face recognition is firstly carried out on the historical image information to determine a plurality of third target persons, and for each third target person, the human body posture information of the third target person is determined through the existing human body posture recognition technology; according to the human body posture information, whether the third target person has the elevator taking action intention of leaving/entering the elevator car or not can be determined, and whether the third target person leaves/enters the elevator car or not can be verified according to the third image information of the next time period so as to correct the determined elevator taking action intention; marking the third image information according to the obtained elevator taking action intention to obtain an action intention label; and determining a second training data set for training the riding behavior intention recognition model according to the third image information and the behavior intention label.
It can be understood that the training process of the riding behavior intention recognition model is similar to the training process of the peer recognition model, and is also obtained through iterative learning training of the convolutional neural network, and details are not repeated here.
The method flow of the embodiment of the invention is described above. It can be understood that the embodiment of the invention enables the fellow staff to leave or enter the elevator car together by identifying the control of the door closing action of the fellow staff, thereby avoiding the fellow staff to take the elevator independently, improving the safety of taking the elevator, avoiding continuously or repeatedly pressing the door opening key, saving the elevator taking time of passengers, and improving the elevator taking efficiency and elevator taking experience of the passengers.
Referring to fig. 2, an embodiment of the present invention provides a door closing control system for an elevator based on machine vision, including:
the system comprises an image acquisition module, a display module and a display module, wherein the image acquisition module is used for acquiring first image information, and the first image information comprises at least one of real-time image information in an elevator car and real-time image information outside an elevator hall;
the peer person identification module is used for inputting the first image information into a pre-trained peer person identification model to obtain a peer person identification result;
and the door closing action control module is used for controlling the door closing action of the elevator according to the identification result of the staff.
The content in the method embodiment is applicable to the system embodiment, the functions specifically realized by the system embodiment are the same as those of the method embodiment, and the achieved beneficial effects are the same as those of the method embodiment.
Referring to fig. 3, an embodiment of the present invention provides an elevator door closing control device based on machine vision, including:
at least one processor;
at least one memory for storing at least one program;
the at least one program, when executed by the at least one processor, causes the at least one processor to implement a machine vision based elevator door closing control method as described above.
The content in the method embodiment is applicable to the embodiment of the device, and the functions specifically realized by the embodiment of the device are the same as those of the method embodiment, and the obtained beneficial effects are the same as those of the method embodiment.
The embodiment of the invention also provides a computer readable storage medium, in which a program executable by a processor is stored, which when being executed by the processor is used for executing the elevator door closing control method based on machine vision.
The computer readable storage medium of the embodiment of the invention can execute the elevator door closing control method based on machine vision, can execute any combination implementation steps of the embodiment of the method, and has the corresponding functions and beneficial effects of the method.
Embodiments of the present invention also disclose a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions may be read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, to cause the computer device to perform the method shown in fig. 1.
In some alternative embodiments, the functions/acts noted in the block diagrams may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed, and in which sub-operations described as part of a larger operation are performed independently.
Furthermore, while the present invention has been described in the context of functional modules, it should be appreciated that, unless otherwise indicated, one or more of the functions and/or features described above may be integrated in a single physical device and/or software module or one or more of the functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to an understanding of the present invention. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be apparent to those skilled in the art from consideration of their attributes, functions and internal relationships. Accordingly, one of ordinary skill in the art can implement the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative and are not intended to be limiting upon the scope of the invention, which is to be defined in the appended claims and their full scope of equivalents.
The above functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied in essence or a part contributing to the prior art or a part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above-described method of the various embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Logic and/or steps represented in the flowcharts or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium upon which the program described above is printed, as the program described above may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the foregoing description of the present specification, reference has been made to the terms "one embodiment/example", "another embodiment/example", "certain embodiments/examples", and the like, means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
While the preferred embodiment of the present invention has been described in detail, the present invention is not limited to the above embodiments, and various equivalent modifications and substitutions can be made by those skilled in the art without departing from the spirit of the present invention, and these equivalent modifications and substitutions are intended to be included in the scope of the present invention as defined in the appended claims.
Claims (8)
1. The elevator door closing control method based on machine vision is characterized by comprising the following steps of:
acquiring first image information, wherein the first image information comprises at least one of real-time image information in an elevator car and real-time image information outside an elevator hall;
inputting the first image information into a pre-trained peer person identification model to obtain a peer person identification result;
controlling the door closing action of the elevator according to the identification result of the staff;
the elevator door closing control method further comprises the step of pre-training a peer identification model, and specifically comprises the following steps:
acquiring second image information, wherein the second image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
performing face recognition on the second image information to determine a plurality of second target persons;
acquiring historical elevator taking behavior information of the second target person, wherein the historical elevator taking behavior information comprises a historical elevator taking section and a historical elevator taking time, determining peer relationship information of the second target person according to the historical elevator taking behavior information, or performing human body behavior recognition on the second image information to determine interaction information of the second target person, and determining peer relationship information of the second target person according to the interaction information;
labeling the second image information according to the peer relationship information to obtain a peer label;
and determining a first training data set according to the second image information and the peer personnel labels, and inputting the first training data set into a first convolutional neural network constructed in advance for training to obtain a trained peer personnel identification model.
2. The machine vision-based elevator door closing control method according to claim 1, wherein the step of inputting the first training data set into a first convolutional neural network constructed in advance to train and obtain a trained peer identification model specifically comprises the following steps:
inputting the first training data set into a first convolutional neural network constructed in advance to obtain a peer prediction result;
determining a loss value of the first convolutional neural network according to the peer prediction result and the peer label;
updating parameters of the first convolutional neural network through a back propagation algorithm according to the loss value;
and stopping training when the loss value reaches a preset first threshold value or the iteration number reaches a preset second threshold value or the test precision reaches a preset third threshold value, and obtaining a trained peer identification model.
3. The machine vision-based door closing control method for an elevator according to claim 1, wherein the step of controlling the door closing operation of the elevator according to the recognition result of the staff specifically comprises:
determining a plurality of peer groups according to the peer identification result, wherein the peer groups comprise at least two first target persons with peer relations;
when the first target person is detected to leave the elevator car, keeping the elevator open until all members of the corresponding peer person group leave the elevator car, and controlling the elevator to perform door closing action;
or alternatively, the first and second heat exchangers may be,
when the first target person is detected to enter the elevator car, keeping the elevator open until all members of the corresponding peer person group enter the elevator car, and controlling the elevator to perform door closing action.
4. The machine vision-based door closing control method of an elevator according to claim 3, wherein the step of controlling the door closing operation of the elevator according to the recognition result of the staff further comprises the steps of:
inputting the first image information into a pre-trained riding behavior intention recognition model to obtain a behavior intention recognition result;
when the first intention of leaving the elevator car of the first target person is identified, keeping the elevator open until all members of the corresponding peer personnel group leave the elevator car, and controlling the elevator to perform door closing action;
or alternatively, the first and second heat exchangers may be,
and when the first target person is identified to have the second action intention of entering the elevator car, keeping the elevator open until all members of the corresponding peer person group enter the elevator car, and controlling the elevator to perform door closing action.
5. The machine vision-based elevator door-closing control method of claim 4, further comprising the step of pre-training an elevator riding behavior intention recognition model, comprising:
acquiring third image information, wherein the third image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
performing face recognition on the third image information to determine a plurality of third target persons, and performing human body posture recognition on the third image information to determine human body posture information of the third target persons;
determining the riding behavior intention of the third target person according to the human body posture information, and marking the third image information according to the riding behavior intention to obtain a behavior intention label;
and determining a second training data set according to the third image information and the behavior intention label, and inputting the second training data set into a second convolutional neural network constructed in advance for training to obtain a trained riding behavior intention recognition model.
6. An elevator door closing control system based on machine vision, comprising:
the system comprises an image acquisition module, a display module and a display module, wherein the image acquisition module is used for acquiring first image information, and the first image information comprises at least one of real-time image information in an elevator car and real-time image information outside an elevator hall;
the peer person identification module is used for inputting the first image information into a pre-trained peer person identification model to obtain a peer person identification result;
the door closing action control module is used for controlling the door closing action of the elevator according to the identification result of the staff;
the peer identification model is obtained through training the following steps:
acquiring second image information, wherein the second image information comprises at least one of historical image information in an elevator car and historical image information outside an elevator hall;
performing face recognition on the second image information to determine a plurality of second target persons;
acquiring historical elevator taking behavior information of the second target person, wherein the historical elevator taking behavior information comprises a historical elevator taking section and a historical elevator taking time, determining peer relationship information of the second target person according to the historical elevator taking behavior information, or performing human body behavior recognition on the second image information to determine interaction information of the second target person, and determining peer relationship information of the second target person according to the interaction information;
labeling the second image information according to the peer relationship information to obtain a peer label;
and determining a first training data set according to the second image information and the peer personnel labels, and inputting the first training data set into a first convolutional neural network constructed in advance for training to obtain a trained peer personnel identification model.
7. An elevator door closing control device based on machine vision, comprising:
at least one processor;
at least one memory for storing at least one program;
when the at least one program is executed by the at least one processor, the at least one processor is caused to implement a machine vision based elevator door closing control method as claimed in any one of claims 1 to 5.
8. A computer readable storage medium, in which a processor executable program is stored, characterized in that the processor executable program, when being executed by a processor, is for performing a machine vision based elevator door closing control method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210809280.5A CN115215185B (en) | 2022-07-11 | 2022-07-11 | Elevator door closing control method, system, device and medium based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210809280.5A CN115215185B (en) | 2022-07-11 | 2022-07-11 | Elevator door closing control method, system, device and medium based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115215185A CN115215185A (en) | 2022-10-21 |
| CN115215185B true CN115215185B (en) | 2023-12-19 |
Family
ID=83610684
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210809280.5A Active CN115215185B (en) | 2022-07-11 | 2022-07-11 | Elevator door closing control method, system, device and medium based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115215185B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117246867A (en) * | 2023-09-22 | 2023-12-19 | 深圳市瀚强科技股份有限公司 | Elevator control method and related device |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004250150A (en) * | 2003-02-19 | 2004-09-09 | Toshiba Elevator Co Ltd | Operation management system for elevator |
| RU2378178C1 (en) * | 2006-01-12 | 2010-01-10 | Отис Элевейтэ Кампэни | Control system of elevators and method of control automation for elevators |
| CN102040145A (en) * | 2009-10-12 | 2011-05-04 | 西安金和光学科技有限公司 | Method for intelligently shortening closing time of elevator |
| JP2013071784A (en) * | 2011-09-26 | 2013-04-22 | Mitsubishi Electric Corp | Elevator control device |
| JP2018127349A (en) * | 2017-02-10 | 2018-08-16 | フジテック株式会社 | Door control system for elevator |
| JP6558762B1 (en) * | 2018-07-31 | 2019-08-14 | 東芝エレベータ株式会社 | Elevator and elevator control method |
| CN110304506A (en) * | 2019-07-09 | 2019-10-08 | 日立楼宇技术（广州）有限公司 | A kind of elevator control method, device, elevator device and storage medium |
| CN111325292A (en) * | 2020-03-11 | 2020-06-23 | 中国电子工程设计院有限公司 | Object behavior identification method and device |
| CN111573459A (en) * | 2020-05-26 | 2020-08-25 | 安徽德特信息技术有限公司 | Elevator intelligent management system based on Internet of things |
| CN112182365A (en) * | 2020-09-11 | 2021-01-05 | 北京三快在线科技有限公司 | Recommendation information generation method and device, electronic equipment and storage medium |
| KR20220003374A (en) * | 2020-07-01 | 2022-01-10 | 한국미쓰비시엘리베이터 주식회사 | Pet recognition opening time extension system and method thereof |
| CN113955594A (en) * | 2021-10-18 | 2022-01-21 | 日立楼宇技术（广州）有限公司 | Elevator control method and device, computer equipment and storage medium |
| CN114524339A (en) * | 2022-01-06 | 2022-05-24 | 广东博智林机器人有限公司 | Elevator car safe operation detection method, device, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016135114A1 (en) * | 2015-02-23 | 2016-09-01 | Inventio Ag | Elevator system with adaptive door control |
| CN110510486B (en) * | 2018-05-21 | 2023-03-14 | 奥的斯电梯公司 | Elevator door control system, elevator system and elevator door control method |
-
2022
- 2022-07-11 CN CN202210809280.5A patent/CN115215185B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004250150A (en) * | 2003-02-19 | 2004-09-09 | Toshiba Elevator Co Ltd | Operation management system for elevator |
| RU2378178C1 (en) * | 2006-01-12 | 2010-01-10 | Отис Элевейтэ Кампэни | Control system of elevators and method of control automation for elevators |
| CN102040145A (en) * | 2009-10-12 | 2011-05-04 | 西安金和光学科技有限公司 | Method for intelligently shortening closing time of elevator |
| JP2013071784A (en) * | 2011-09-26 | 2013-04-22 | Mitsubishi Electric Corp | Elevator control device |
| JP2018127349A (en) * | 2017-02-10 | 2018-08-16 | フジテック株式会社 | Door control system for elevator |
| JP6558762B1 (en) * | 2018-07-31 | 2019-08-14 | 東芝エレベータ株式会社 | Elevator and elevator control method |
| CN110304506A (en) * | 2019-07-09 | 2019-10-08 | 日立楼宇技术（广州）有限公司 | A kind of elevator control method, device, elevator device and storage medium |
| CN111325292A (en) * | 2020-03-11 | 2020-06-23 | 中国电子工程设计院有限公司 | Object behavior identification method and device |
| CN111573459A (en) * | 2020-05-26 | 2020-08-25 | 安徽德特信息技术有限公司 | Elevator intelligent management system based on Internet of things |
| KR20220003374A (en) * | 2020-07-01 | 2022-01-10 | 한국미쓰비시엘리베이터 주식회사 | Pet recognition opening time extension system and method thereof |
| CN112182365A (en) * | 2020-09-11 | 2021-01-05 | 北京三快在线科技有限公司 | Recommendation information generation method and device, electronic equipment and storage medium |
| CN113955594A (en) * | 2021-10-18 | 2022-01-21 | 日立楼宇技术（广州）有限公司 | Elevator control method and device, computer equipment and storage medium |
| CN114524339A (en) * | 2022-01-06 | 2022-05-24 | 广东博智林机器人有限公司 | Elevator car safe operation detection method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115215185A (en) | 2022-10-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107463888B (en) | Face emotion analysis method and system based on multi-task learning and deep learning | |
| CN109204007B (en) | Unmanned suspension type magnetic suspension train and control method thereof | |
| CN107679471B (en) | Indoor personnel air post detection method based on video monitoring platform | |
| Liu et al. | Learning proactive behavior for interactive social robots | |
| CN110874716A (en) | Interview evaluation method and device, electronic equipment and storage medium | |
| CN109431507A (en) | Cough disease identification method and device based on deep learning | |
| CN115215185B (en) | Elevator door closing control method, system, device and medium based on machine vision | |
| Wöllmer et al. | Analyzing the memory of BLSTM neural networks for enhanced emotion classification in dyadic spoken interactions | |
| WO2020056995A1 (en) | Method and device for determining speech fluency degree, computer apparatus, and readable storage medium | |
| Zhang et al. | Pattern recognition of momentary mental workload based on multi-channel electrophysiological data and ensemble convolutional neural networks | |
| CN112464760A (en) | Training method and device for target recognition model | |
| CN112712068B (en) | Key point detection method and device, electronic equipment and storage medium | |
| CN114400004A (en) | On-site service monitoring method based on intelligent voice and video behavior recognition technology | |
| CN111291672B (en) | Combined image text recognition and fuzzy judgment method, device and storage medium | |
| CN113255362A (en) | Method and device for filtering and identifying human voice, electronic device and storage medium | |
| Yu et al. | Speaking style based apparent personality recognition | |
| CN115871679A (en) | Driver fatigue detection method, driver fatigue detection device, electronic device, and medium | |
| CN110334244B (en) | Data processing method and device and electronic equipment | |
| WO2022252089A1 (en) | Training method for object detection model, and object detection method and device | |
| CN115196455A (en) | Elevator control method, elevator and computer readable storage medium | |
| CN114360533A (en) | Interaction method and system based on machine learning, elevator equipment and medium | |
| CN116363542A (en) | Off-duty event detection method, apparatus, device and computer readable storage medium | |
| CN110647622A (en) | Interactive data validity identification method and device | |
| Ziaratnia et al. | End-to-end deep learning for pNN50 estimation using a spatiotemporal representation | |
| CN110181503A (en) | Method for detecting abnormality, device, smart machine and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |