CN112261380B

CN112261380B - Elevator control method and device

Info

Publication number: CN112261380B
Application number: CN202011154786.4A
Authority: CN
Inventors: 任亦立
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2020-10-26
Filing date: 2020-10-26
Publication date: 2023-04-07
Anticipated expiration: 2040-10-26
Also published as: CN112261380A

Abstract

The embodiment of the application provides a control method and a control device of an elevator, wherein the method comprises the following steps: when the elevator is in a door opening state, acquiring a current monitoring video shot by a camera in the elevator in real time; detecting the current monitoring video based on the integral image of the preset article to obtain a first integral detection result of whether the preset article exists in the current monitoring video; if the first overall detection result is that the preset article exists in the current monitoring video, when the first preset time length is reached, a plurality of local images based on the preset article are obtained, the current monitoring video is detected, and whether a first local detection result of the preset article exists in the current monitoring video is obtained; if the first local detection result is that the preset article exists in the current monitoring video, a first door opening control signal is sent to the elevator, so that the elevator keeps a door opening state, and further, the preset article can be effectively prevented from taking the elevator.

Description

Elevator control method and device

Technical Field

The application relates to the technical field of video monitoring, in particular to a control method and a control device for an elevator.

Background

In order to facilitate the traveling of users, the building is generally provided with an elevator. The user can reach higher floor through the elevator fast, also can transport article to higher floor through the elevator.

However, in some scenarios, some items may be restricted from riding in the elevator. For example, a building in an office area may prohibit pets from taking an elevator, and thus, prevent pets from entering the office area. In addition, the building of residential area can forbid the storage battery car to take the elevator, and then, prevents that the resident from stopping the storage battery car at the fire control passageway of higher floor, perhaps, charges the storage battery car in the indoor of higher floor to avoid the potential safety hazard.

How to control the elevator to prevent the restricted goods from taking the elevator is a problem to be solved.

Disclosure of Invention

An object of the embodiments of the present application is to provide a method and apparatus for controlling an elevator, so as to effectively prevent a preset item from taking the elevator. The specific technical scheme is as follows:

in order to achieve the above object, embodiments of the present application disclose a control method for an elevator, the method including:

when the elevator is in a door opening state, acquiring a current monitoring video shot by a camera in the elevator in real time;

detecting the current monitoring video based on the integral image of the preset article to obtain a first integral detection result of whether the preset article exists in the current monitoring video;

if the first overall detection result is that the preset article exists in the current monitoring video, when the first preset time length is reached, a plurality of local images based on the preset article are obtained, the current monitoring video is detected, and the first local detection result of whether the preset article exists in the current monitoring video is obtained;

and if the first local detection result is that preset articles exist in the current monitoring video, sending a first door opening control signal to the elevator so as to enable the elevator to keep a door opening state.

Optionally, the method further includes:

if the first overall detection result is that a preset article exists in the current monitoring video, sending a second door opening control signal to the elevator to enable the elevator to keep a door opening state;

and if the first local detection result indicates that no preset article exists in the current monitoring video, stopping sending the second door opening control signal to the elevator.

Optionally, the detecting the current surveillance video based on the overall image of the preset article to obtain a first overall detection result of whether the preset article exists in the current surveillance video includes:

carrying out overall object detection on each first video frame within a second preset time before the current time based on an overall image of a preset object to obtain the probability that the whole preset object exists in each first video frame;

determining video frames with the probability that the whole preset article exists in each first video frame and the probability is larger than the first probability to serve as second video frames;

if the ratio of the number of the second video frames to the number of each first video frame is larger than a first threshold value, determining that a first overall detection result is that a preset article exists in the current monitoring video;

and if the ratio of the number of the second video frames to the number of the first video frames is not larger than a first threshold value, determining that the first overall detection result is that no preset article exists in the current monitoring video.

Optionally, the performing, based on the overall image of the preset article, the overall article detection on each first video frame within a second preset duration before the current time to obtain the probability that the whole preset article exists in each first video frame includes:

inputting each first video frame within a second preset time before the current moment into a first neural network model trained in advance respectively to obtain the probability that the whole preset article exists in each first video frame;

the first neural network model is obtained by training based on a plurality of first positive sample video frames and a plurality of first negative sample video frames; the first positive sample video frame is a video frame containing an overall image of a preset article, and the first negative sample video frame is a video frame not containing the overall image of the preset article.

Optionally, if the first overall detection result is that a preset article exists in the current surveillance video, when a first preset duration is reached, obtaining a plurality of local images based on the preset article, detecting the current surveillance video, and obtaining whether a first local detection result of the preset article exists in the current surveillance video, including:

if the first overall detection result indicates that a preset article exists in the current monitoring video, when the first preset time length is reached, a plurality of local images based on the preset article are obtained, article local detection is carried out on each third video frame within a third preset time length before the current moment, and the probability that the local part of the preset article exists in each third video frame is obtained;

for each part of a preset article, determining a fourth video frame with the probability that the part exists in each third video frame being greater than a second probability;

if a part with the ratio of the number of the corresponding fourth video frames to the number of the third video frames larger than a second threshold exists, determining that a first local detection result is that a preset article exists in the current monitoring video;

and if no part with the ratio of the number of the corresponding fourth video frames to the number of the third video frames larger than the second threshold exists, determining that the first local detection result is that no preset article exists in the current monitoring video.

Optionally, if the first overall detection result indicates that a preset article exists in the current surveillance video, when the first preset duration is reached, multiple local images based on the preset article are obtained, and article local detection is performed on each third video frame within a third preset duration before the current time, so as to obtain a probability that a local part of the preset article exists in each third video frame, where the probability includes:

if the first overall detection result indicates that a preset article exists in the current monitoring video, when the first preset time length is reached, obtaining that each third video frame within a third preset time length before the current time is respectively input into a pre-trained second neural network model, and obtaining the local probability of the preset article existing in each third video frame;

the second neural network model is obtained by training based on a plurality of second positive sample video frames and a plurality of second negative sample video frames; the second positive sample video frame is a video frame containing a local image of a preset article, and the second negative sample video frame is a video frame not containing a local image of a preset article.

Optionally, after sending the first door opening control signal to the elevator, the method further comprises:

and when a second overall detection result and a second local detection result obtained by detecting the current monitoring video do not have the preset article in the current monitoring video, stopping sending the first door opening control signal to the elevator.

Optionally, the method further includes:

and if the first local detection result is that the preset article exists in the current monitoring video, playing a reminding voice for forbidding the preset article to enter the elevator.

Optionally, the method further includes:

if the first local detection result is that a preset article exists in the current monitoring video, sending a first reminding message to a preset terminal to remind a user of entering the elevator of the preset article;

and when a control releasing message sent by the preset terminal is received, stopping sending the first door opening control signal to the elevator.

Optionally, before detecting the current surveillance video based on the overall image of the preset article to obtain a first overall detection result of whether the preset article exists in the current surveillance video, the method further includes:

determining whether the camera is shielded or not based on pixel values of pixel points of the current monitoring video;

if the door is determined to be blocked, sending a third door opening control signal to the elevator so as to enable the elevator to keep a door opening state;

when the fourth preset time length is reached, stopping sending the third door opening control signal to the elevator, and sending a second reminding message to a preset terminal to remind a user that the camera is shielded under the condition that the camera is detected to be shielded;

and if the whole image based on the preset article is determined not to be blocked, executing the step of detecting the current monitoring video to obtain a first whole detection result of whether the preset article exists in the current monitoring video.

In a second aspect, in order to achieve the above object, an embodiment of the present application discloses a control device for an elevator, the device including:

the monitoring video acquisition module is used for acquiring the current monitoring video shot by a camera in the elevator in real time when the elevator is in a door opening state;

the first overall detection result acquisition module is used for detecting the current monitoring video based on the overall image of the preset article to obtain a first overall detection result of whether the preset article exists in the current monitoring video;

a first local detection result obtaining module, configured to, if the first overall detection result is that a preset article exists in the current surveillance video, obtain, when a first preset duration is reached, a plurality of local images based on the preset article, detect the current surveillance video, and obtain a first local detection result of whether the preset article exists in the current surveillance video;

and the first door opening control signal sending module is used for sending a first door opening control signal to the elevator if the first local detection result indicates that a preset article exists in the current monitoring video, so that the elevator keeps a door opening state.

Optionally, the apparatus further comprises:

the second door opening control signal sending module is used for sending a second door opening control signal to the elevator to enable the elevator to keep a door opening state if the first overall detection result indicates that a preset article exists in the current monitoring video;

and the first processing module is used for stopping sending the second door opening control signal to the elevator if the first local detection result indicates that no preset article exists in the current monitoring video.

Optionally, the first overall detection result obtaining module includes:

the first probability obtaining submodule is used for carrying out overall object detection on each first video frame within a second preset duration before the current moment based on an overall image of a preset object to obtain the probability that the whole preset object exists in each first video frame;

the second video frame determining sub-module is used for determining a video frame, as a second video frame, in each first video frame, the probability of the whole preset article being greater than the first probability;

the first processing sub-module is used for determining that a first overall detection result is that a preset article exists in the current monitoring video if the ratio of the number of the second video frames to the number of each first video frame is larger than a first threshold;

and the second processing sub-module is used for determining that the first overall detection result is that no preset article exists in the current monitoring video if the ratio of the number of the second video frames to the number of each first video frame is not greater than a first threshold value.

Optionally, the first probability obtaining sub-module is specifically configured to input each first video frame within a second preset duration before the current time into a pre-trained first neural network model, so as to obtain a probability that the whole preset article exists in each first video frame;

Optionally, the first local detection result obtaining module includes:

a second probability obtaining sub-module, configured to, if the first overall detection result is that a preset article exists in the current surveillance video, obtain, when a first preset duration is reached, a plurality of local images based on the preset article, perform article local detection on each third video frame within a third preset duration before the current time, and obtain a probability that a local part of the preset article exists in each third video frame;

the fourth video frame determining submodule is used for determining a fourth video frame, of which the probability of the local part is greater than the second probability, in each third video frame aiming at each local part of the preset article;

the third processing submodule is used for determining that the first local detection result is that a preset article exists in the current monitoring video if a local part exists, wherein the ratio of the number of the corresponding fourth video frames to the number of each third video frame is larger than a second threshold;

and the fourth processing submodule is used for determining that the first local detection result is that no preset article exists in the current monitoring video if no part exists, wherein the ratio of the number of the corresponding fourth video frames to the number of the third video frames is greater than a second threshold value.

Optionally, the second probability obtaining sub-module is specifically configured to, if the first overall detection result is that a preset article exists in the current surveillance video, obtain, when the first preset duration is reached, that each third video frame within a third preset duration before the current time is respectively input into a pre-trained second neural network model, and obtain a probability that a local part of the preset article exists in each third video frame;

Optionally, the apparatus further comprises:

and the second processing module is used for stopping sending the first door opening control signal to the elevator when a second overall detection result and a second local detection result obtained by detecting the current monitoring video do not have preset articles in the current monitoring video after sending the first door opening control signal to the elevator.

Optionally, the apparatus further comprises:

and the playing module is used for playing a reminding voice for forbidding the preset article to enter the elevator if the first local detection result indicates that the preset article exists in the current monitoring video.

Optionally, the apparatus further comprises:

the first reminding message sending module is used for sending a first reminding message to a preset terminal to remind a user of entering a preset article into an elevator if the first local detection result is that the preset article exists in the current monitoring video;

and the third processing module is used for stopping sending the first door opening control signal to the elevator when receiving a control releasing message sent by the preset terminal.

Optionally, the apparatus further comprises:

the shielding detection module is used for determining whether the camera is shielded or not based on pixel values of pixel points of the current monitoring video before detecting the current monitoring video based on the integral image of the preset article and obtaining whether a first integral detection result of the preset article exists in the current monitoring video or not; if yes, triggering a third door opening control signal sending module, and if not, triggering the first overall detection result acquisition module;

the third door opening control signal sending module is used for sending a third door opening control signal to the elevator so as to enable the elevator to keep a door opening state;

and the fourth processing module is used for stopping sending the third door opening control signal to the elevator when a fourth preset time length is reached, and sending a second reminding message to a preset terminal to remind a user that the camera is shielded under the condition that the camera is detected to be shielded.

In another aspect of this application, in order to achieve the above object, an embodiment of this application further discloses an electronic device, where the electronic device includes a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;

the memory is used for storing a computer program;

the processor is configured to implement the elevator control method according to the first aspect described above when executing the program stored in the memory.

In yet another aspect of the present application, there is also provided a computer-readable storage medium having instructions stored therein, which when run on a computer, implement the control method of an elevator as described in the above first aspect.

In yet another aspect of this application, this application embodiment also provides a computer program product containing instructions, which when run on a computer, causes the computer to execute the control method of the elevator described in the above first aspect.

The embodiment of the application provides a control method of an elevator, which comprises the steps of acquiring a current monitoring video shot by a camera in the elevator in real time when the elevator is in a door opening state; detecting the current monitoring video based on the integral image of the preset article to obtain a first integral detection result of whether the preset article exists in the current monitoring video; if the first overall detection result is that the preset article exists in the current monitoring video, when the first preset time length is reached, a plurality of local images based on the preset article are obtained, the current monitoring video is detected, and whether a first local detection result of the preset article exists in the current monitoring video is obtained; and if the first local detection result is that the preset article exists in the current monitoring video, sending a first door opening control signal to the elevator so as to enable the elevator to keep the door opening state.

Based on the above processing, the preset articles can be detected to exist in the current monitoring video, namely, when the preset articles are detected to enter the elevator, the elevator is controlled to be in the door opening state, namely, the elevator does not run, and the preset articles can be effectively prevented from taking the elevator.

Of course, not all advantages described above need to be achieved at the same time in the practice of any one product or method of the present application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a flowchart of a control method of an elevator according to an embodiment of the present application;

fig. 2 is another flowchart of a control method of an elevator provided in an embodiment of the present application;

fig. 3 is another flowchart of a control method of an elevator provided in an embodiment of the present application;

fig. 4 is another flowchart of a control method of an elevator provided in an embodiment of the present application;

fig. 5 is another flowchart of a control method of an elevator according to an embodiment of the present application;

fig. 6 is another flowchart of a control method of an elevator according to an embodiment of the present application;

fig. 7 is another flowchart of a control method of an elevator according to an embodiment of the present application;

fig. 8 is a block diagram of a control system of an elevator according to an embodiment of the present application;

fig. 9 is a configuration diagram of a control device of an elevator according to an embodiment of the present application;

fig. 10 is a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

In order to effectively prevent preset goods from taking the elevator, the embodiment of the present application provides a control method of the elevator, and referring to fig. 1, fig. 1 is a flowchart of the control method of the elevator provided by the embodiment of the present application, and the method may include the following steps:

s101: when the elevator is in a door opening state, acquiring a current monitoring video shot by a camera in the elevator in real time;

s102: and detecting the current monitoring video based on the integral image of the preset article to obtain a first integral detection result of whether the preset article exists in the current monitoring video.

S103: if the first overall detection result is that the preset article exists in the current monitoring video, when the first preset time length is reached, a plurality of local images based on the preset article are obtained, the current monitoring video is detected, and whether the first local detection result of the preset article exists in the obtained current monitoring video or not is judged.

S104: and if the first local detection result is that the preset article exists in the current monitoring video, sending a first door opening control signal to the elevator so as to enable the elevator to keep the door opening state.

According to the control method of the elevator, the preset article can be detected to exist in the current monitoring video, namely, when the preset article is detected to enter the elevator, the elevator is controlled to be in the door opening state, namely, the elevator does not operate, and the preset article can be effectively prevented from taking the elevator.

In one implementation, the method of the embodiment of the present application may be applied to a camera installed in an elevator, or may also be applied to an electronic device (e.g., a server) capable of performing data communication with the camera in the elevator, and the electronic device may acquire a monitoring video from the camera and perform processing based on the method of the embodiment of the present application.

The preset articles can be determined according to the scene requirements of the elevator, for example, the preset articles can be pets for the elevator of a building in an office area; aiming at elevators of buildings in residential areas, the preset articles can be battery cars; for a dedicated freight elevator, the preset items may be people.

In an embodiment, each video frame collected by the camera may be detected, or the video frames collected by the camera may also be sampled, and the detection is performed based on the sampled video frames, that is, the obtained monitoring video may include the sampled video frames.

The first preset time period may be set by a user as needed, for example, the first preset time period may be 2 seconds, or may also be 4 seconds, but is not limited thereto.

In one embodiment, referring to fig. 2, the method may further comprise the steps of:

s105: and if the first overall detection result is that the preset article exists in the current monitoring video, sending a second door opening control signal to the elevator so as to enable the elevator to keep the door opening state.

S106: and if the first local detection result indicates that no preset article exists in the current monitoring video, stopping sending a second door opening control signal to the elevator.

In the embodiment of the application, in order to prevent the preset article from taking the elevator in time, if the first overall detection result is that the preset article exists in the current monitoring video, the second door opening control signal can be directly sent to the elevator to control the elevator to be in a door opening state.

Subsequently, if the first local detection result indicates that no preset article exists in the current monitoring video, the first overall detection result indicates false alarm, and therefore, the second door opening control signal can be stopped from being sent to the elevator, and the elevator can run normally.

In one embodiment, referring to fig. 3, step S102 may include the steps of:

s1021: and carrying out overall article detection on each first video frame within a second preset time before the current time based on the overall image of the preset article to obtain the probability that the whole preset article exists in each first video frame.

S1022: and determining the video frames with the probability that the whole preset article exists in each first video frame and the probability is greater than the first probability as second video frames.

S1023: and if the ratio of the number of the second video frames to the number of each first video frame is greater than a first threshold value, determining that the first overall detection result is that a preset article exists in the current monitoring video.

S1024: and if the ratio of the number of the second video frames to the number of each first video frame is not greater than a first threshold value, determining that the first overall detection result is that no preset article exists in the current monitoring video.

The second preset time period, the first probability and the first threshold may be set by a technician according to experience, for example, the second preset time period may be 2 seconds, or may also be 3 seconds, but is not limited thereto. The first threshold may be 0.5 or 0.6, but is not limited thereto. The first probability may be 50% or 60%, but is not limited thereto.

In this embodiment of the application, it may be determined that each detected video frame (i.e., a first video frame) is detected when the entire article detection is performed within a second preset duration before the current time, and in addition, it may be determined that a video frame (i.e., a second video frame) in which the probability that the entire preset article exists is greater than the first probability exists in each detected first video frame.

Correspondingly, if the ratio of the number of the second video frames to the number of the first video frames is larger than a first threshold value, it is indicated that the preset article exists in the current monitoring video based on the article overall detection, otherwise, it is determined that the preset article does not exist in the current monitoring video.

In one embodiment, for step S1021, a similarity between the first video frame and a video frame containing an overall image of a preset article may be calculated, and if the similarity is greater than a first similarity threshold, it may be determined that the entire preset article exists in the first video frame, otherwise, it is determined that the entire preset article does not exist in the first video frame.

In one embodiment, S1021 may comprise the steps of: and respectively inputting each first video frame within a second preset time length before the current moment into a first neural network model trained in advance to obtain the probability that the whole preset article exists in each first video frame.

The first neural network model is obtained by training based on a plurality of first positive sample video frames and a plurality of first negative sample video frames; the first positive sample video frame is a video frame containing an entire image of the preset article, and the first negative sample video frame is a video frame not containing the entire image of the preset article.

In an embodiment of the present application, the first neural network model may be a convolutional neural network model. In particular, the first neural network model may include a convolutional layer and a classification layer. For example, a first label of a first positive sample video frame may be set to 1, representing a whole image containing a preset item; the first label of the first negative example video frame is set to 1, which indicates that the whole image does not contain the preset article.

When the first neural network model is trained, the first positive sample video frame and the first negative sample video frame can be used as input parameters, the corresponding first label can be used as an output parameter, further, the actual output of the first neural network model can be obtained, a loss function between the actual output and the first label is calculated, and the model parameter of the first neural network model is adjusted according to the loss function until the first neural network model reaches a convergence condition.

Furthermore, the acquired first video frame can be input into the first neural network model, and the probability that the whole preset article exists in the first video frame can be obtained. If the probability is greater than the first probability, it may be determined that the entire preset item is present in the first video frame, otherwise, it is determined that the entire preset item is not present in the first video frame.

In one embodiment, referring to fig. 3, step S103 may include the steps of:

s1031: if the first overall detection result indicates that the preset article exists in the current monitoring video, when the first preset time length is reached, a plurality of local images based on the preset article are obtained, article local detection is carried out on each third video frame within a third preset time length before the current moment, and the probability that the local part of the preset article exists in each third video frame is obtained.

S1032: and for each part of the preset article, determining a fourth video frame with the probability that the part exists in each third video frame, wherein the probability is greater than the second probability.

S1033: and if a part with the ratio of the number of the corresponding fourth video frames to the number of each third video frame larger than a second threshold exists, determining that the first local detection result is that the preset article exists in the current monitoring video.

S1034: and if the part with the ratio of the number of the corresponding fourth video frames to the number of the third video frames larger than the second threshold does not exist, determining that the first local detection result is that the preset article does not exist in the current monitoring video.

The third preset time period, the second probability and the second threshold may be set by a technician according to experience, for example, the third preset time period may be 2 seconds, or may also be 3 seconds, but is not limited thereto. The second threshold may be 0.3 or 0.4, but is not limited thereto. The second probability may be 30% or, alternatively, may be 40%, but is not limited thereto.

For example, if the preset object is an electric vehicle, the part of the preset object may include a vehicle head, a vehicle tail, wheels, vehicle lights, and the like. The preset article is a pet dog, and parts of the preset article can comprise the head of the dog, the tail of the dog, limbs of the dog and the like. The preset article is a human, and the part of the preset article may include the head of the human, the limbs of the human, and the like.

In an embodiment, the third preset time period may be a sum of the first preset time period and the second preset time period, or, if the first preset time period is not 0, the third preset time period may be the same as the first preset time period.

In this embodiment of the application, when the first preset duration is reached, it may be determined that each detected video frame (i.e., the third video frame) is detected when the local detection of the article is performed within a third preset duration before the current time.

In addition, for each part, it may also be determined that, in each third video frame detected within a third preset time period, a video frame of the part exists (that is, a video frame with a probability that the part exists is greater than the second probability, that is, a fourth video frame in this application).

Correspondingly, if a part exists, and the ratio of the number of the fourth video frames corresponding to the part to the number of the third video frames is greater than the second threshold, it may be determined that the first local detection result is that a preset article exists in the current surveillance video, otherwise, it is determined that the first local detection result is that the preset article does not exist in the current surveillance video.

That is to say, as long as it is determined based on the article local detection that the proportion of any one local video frame included in the video frames detected within the third preset time period is greater than the second threshold, it may be determined that the preset article exists in the current surveillance video.

In one embodiment, steps S1021-S1024 described above may be employed for step S102, while steps S1031-S1034 may be employed for step S103. Alternatively, the steps S1021 to S1024 described above may be adopted for step S102, and step S103 is not limited. Alternatively, the above steps S1031 to S1034 may be adopted for step S103, but step S102 is not limited thereto.

In one embodiment, the overall article detection and the local article detection may be performed simultaneously based on a video frame acquired in real time to obtain a probability that a whole preset article exists in the video frame and a probability that a local preset article exists in the video frame, and then, based on the obtained probabilities, a first overall detection result corresponding to a first video frame within a second preset duration before the current time may be determined, and after the first preset duration passes, a first local detection result corresponding to a third video frame within a third preset duration before the current time may be obtained.

Or, the article overall detection may be performed based on a video frame acquired in real time to obtain a probability that the whole preset article exists in the video frame, and then, a first overall detection result corresponding to a first video frame within a second preset time period before the current time may be determined based on the obtained probability, and after the first preset time period passes, the article local detection may be performed based on a third video frame within a third preset time period before the current time to obtain a first local detection result.

In one embodiment, for step S1031, a similarity between the third video frame and the video frame containing the local image of the preset item may be calculated, and if the similarity is greater than a second similarity threshold, it may be determined that a local portion of the preset item exists in the third video frame, otherwise, it is determined that a local portion of the preset item does not exist in the third video frame.

In one embodiment, step S1031 may include the steps of: if the first overall detection result indicates that the preset article exists in the current monitoring video, when the first preset time length is reached, the method obtains the probability that each third video frame within a third preset time length before the current moment is respectively input into a pre-trained second neural network model, and the local part of the preset article exists in each third video frame.

The second neural network model is obtained by training based on a plurality of second positive sample video frames and a plurality of second negative sample video frames; the second positive sample video frame is a video frame containing a local image of the preset article, and the second negative sample video frame is a video frame not containing a local image of the preset article.

In an embodiment of the present application, the second neural network model may be a convolutional neural network model. In particular, the second neural network model may include a convolutional layer and a classification layer.

For example, for a part of a preset article, if a second positive sample video frame includes the part, a second label of the second positive sample video frame corresponding to the part may be 1; if the second negative-swatch video frame does not contain the part, then the second label of the second negative-swatch video frame corresponding to the part may be 0.

When the second neural network model is trained, the second positive sample video frame and the second negative sample video frame can be used as input parameters, the corresponding second label can be used as output parameters, further, the actual output of the second neural network model can be obtained, a loss function between the actual output and the second label is calculated, and the model parameters of the second neural network model are adjusted according to the loss function until the second neural network model reaches a convergence condition.

Furthermore, the obtained third video frame can be input into the second neural network model, and the probability that the part of the preset article exists in the third video frame can be obtained. For a part, if the probability that the part corresponds to is greater than the second probability, it may be determined that the part exists in the third video frame, otherwise, it is determined that the part does not exist in the third video frame.

In one embodiment, the object overall detection and the object local detection can be performed based on the first neural network model and the second neural network model respectively based on the video frames acquired in real time, that is, the two neural network models are used for performing the detection respectively.

In one embodiment, the same neural network model (which may be referred to as a third neural network model) may also be utilized to perform the overall detection of the article and the local detection of the article at the same time, i.e., the training sample of the third neural network model may have the first label and the second label at the same time.

And then, inputting the acquired video frame into a trained third neural network model, wherein the third neural network model can output the probability that the whole preset article exists in the video frame and the probability that the part of the preset article exists in the video frame.

In one embodiment, referring to fig. 4, on the basis of fig. 1, after step S104, the method may further include the steps of:

s107: and when a second overall detection result and a second local detection result obtained by detecting the current monitoring video do not have the preset article in the current monitoring video, stopping sending the first door opening control signal to the elevator.

In the embodiment of the application, after the first door opening control signal is sent to the elevator, the preset article may exit the elevator. In order to ensure the normal operation of the elevator, after the first door opening control signal is sent to the elevator, in the process of detecting the video frames acquired in real time subsequently, if the result obtained by carrying out the overall detection of the article (namely, the second overall detection result) and the result obtained by carrying out the local detection of the article (namely, the second local detection result) are the current monitoring video without the preset article, the preset article is indicated to exit from the elevator, and at the moment, the first door opening control signal can be stopped being sent to the elevator, so that the elevator is in a normal operation state.

In the embodiment of the present application, the method for obtaining the second overall detection result may refer to the method for obtaining the first overall detection result in the above embodiment; the method of obtaining the second local detection result may refer to the method of obtaining the first local detection result in the above embodiment.

In one embodiment, referring to fig. 5, on the basis of fig. 1, the method may further include the steps of:

s108: and if the first local detection result indicates that the preset article exists in the current monitoring video, playing a reminding voice for forbidding the preset article to enter the elevator.

In the embodiment of the application, if the first overall detection result and the first local detection result are both preset articles in the current monitoring video, a reminding voice for prohibiting the preset articles from entering the elevator can be played.

In one embodiment, referring to fig. 6, on the basis of fig. 1, the method may further include the steps of:

s109: and if the first local detection result is that the preset article exists in the current monitoring video, sending a first reminding message to a preset terminal to remind a user of entering the elevator of the preset article.

S1010: and when a control releasing message sent by a preset terminal is received, stopping sending the first door opening control signal to the elevator.

In this embodiment of the application, if the first overall detection result and the first local detection result are both preset items in the current monitoring video, the first warning message may be sent to a preset terminal, and the preset terminal may be a terminal for logging in by a person responsible for managing an elevator, for example, a terminal for logging in by a property person of a building.

In addition, if the elevator is entered by a storage battery car such as a special storage battery car for disabled people and allowed to enter the elevator, or the preset articles entering the elevator are temporarily allowed to enter, after the preset terminal receives the first reminding message, the person in charge of managing the elevator can send a releasing control message through the preset terminal, correspondingly, when the releasing control message is received, the person can stop sending the first door opening control signal to the elevator, so that the elevator is in a normal running state, and the normal use of a user is ensured.

In one embodiment, referring to fig. 7, before step S102, the method may further include the steps of:

s1011: and determining whether the camera is blocked or not based on the pixel values of the pixel points of the current monitoring video, if so, executing S1012, and if not, executing S102.

S1012: and sending a third door opening control signal to the elevator to enable the elevator to keep the door opening state.

S1013: and when the fourth preset time length is reached, stopping sending a third door opening control signal to the elevator, and sending a second reminding message to the preset terminal to remind the user that the camera is shielded under the condition that the camera is detected to be shielded.

The fourth preset time period may be set by a technician according to experience, for example, the fourth preset time period may be 4 seconds, or may also be 5 seconds, but is not limited thereto.

In the embodiment of the application, whether the user blocks the camera or not can be determined based on the pixel value of the pixel point of the current monitoring video. For example, when it is detected that the difference between the pixel values of the pixel points of the two adjacent video frames is greater than the preset difference, and/or when it is detected that the overall pixel value of the pixel point of the current video frame is smaller than the third threshold, it may be determined that the camera is blocked, at this time, a third opening control signal for controlling the opening of the elevator may be sent to the elevator, and the duration of the third opening control signal is a fourth preset duration.

Correspondingly, if the fourth preset time duration is reached, the camera is still detected to be shielded, and then a second reminding message can be sent to the preset terminal to remind the user that the camera is shielded.

In addition, a reminding voice for forbidding to shield the camera can be played.

In addition, if it is detected that the camera is not blocked, the current monitoring video can be detected based on the whole image of the preset article, a first whole detection result of whether the preset article exists in the current monitoring video is obtained, and subsequent processing is performed.

In one embodiment, the method provided by the embodiment of the present application may be executed by an electronic device, referring to fig. 8, fig. 8 is a structural diagram of a control system of an elevator provided by the embodiment of the present application, the system may include the electronic device and an elevator control panel 806, and the electronic device may include a video acquisition module 801, an alarm output unit 802, a voice playing module 803, a manual control module 804, and an occlusion detection module 805.

The alarm output unit 802 may include an overall detection module 8021, an alarm signal pre-triggering module 8022, a local detection module 8023, an alarm signal secondary confirmation module 8024, and an alarm signal cancellation module 8025.

Specifically, the video acquisition module 801 is configured to acquire a current monitoring video in the elevator in real time.

The whole detection module 8021 can detect the current monitoring video and determine the probability that the whole preset article exists in the video frame.

When the alarm signal pre-triggering module 8022 determines that the overall detection result is that the preset article exists in the current monitoring video based on the probability that the whole preset article exists in the video frame within 2 seconds between the current moments, the alarm signal pre-triggering module 8024 may send a second door opening control signal to the elevator control panel 806, where the second door opening control signal may last for 5 seconds, and send a pre-alarm signal to the alarm signal secondary confirmation module 8024.

The local detection module 8023 can detect the current surveillance video and determine the probability that the local part of the preset article exists in the video frame.

When the alarm signal secondary confirmation module 8024 receives the pre-alarm signal, timing may be started, and when the timing duration reaches 2 seconds, the probability that the whole preset article exists in the video frame within 2 seconds before the current time may be obtained, and it is determined whether the local detection result is that the preset article exists in the current monitoring video. If the door opening control signal is received, a first door opening control signal can be sent to the elevator control panel 806, a playing signal can be sent to the voice playing module 803, and a first reminding message can also be sent to a preset terminal.

When receiving the playing signal, the voice playing module 803 may play a reminding voice for prohibiting the preset article from entering the elevator.

Subsequently, if the overall detection result and the local detection result are both that no preset article exists in the current monitoring video, the alarm signal removing module 8025 may remove the first door opening control signal.

The user may send a release control message to the alarm signal release module 8025 through the manual control module 804, and accordingly, the alarm signal release module 8025 may also release the first door opening control signal.

The occlusion detection module 805 may determine whether the camera is occluded based on a pixel value of a pixel point of the current monitoring video, and if so, may send a third opening control signal to the elevator control panel 806, and when a fourth preset duration is reached, stop sending the third opening control signal, and if it is detected that the camera is still occluded, send a second reminding message to the preset terminal to remind the user that the camera is occluded. If not, the integral detection module 8021 may be triggered.

Based on the same inventive concept, the embodiment of the present application further provides a control device of an elevator, referring to fig. 9, where fig. 9 is a structural diagram of the control device of the elevator provided in the embodiment of the present application, and the device includes:

the monitoring video acquiring module 901 is used for acquiring the current monitoring video shot by a camera in the elevator in real time when the elevator is in a door opening state;

a first overall detection result obtaining module 902, configured to detect the current surveillance video based on an overall image of a preset article, to obtain a first overall detection result of whether the preset article exists in the current surveillance video;

a first local detection result obtaining module 903, configured to, if the first overall detection result is that a preset article exists in the current surveillance video, obtain a plurality of local images based on the preset article when a first preset duration is reached, detect the current surveillance video, and obtain a first local detection result whether the preset article exists in the current surveillance video;

a first door opening control signal sending module 904, configured to send a first door opening control signal to the elevator if the first local detection result is that a preset article exists in the current monitoring video, so that the elevator keeps a door opening state.

Optionally, the apparatus further comprises:

the second door opening control signal sending module is used for sending a second door opening control signal to the elevator if the first overall detection result indicates that a preset article exists in the current monitoring video, so that the elevator keeps a door opening state;

Optionally, the first overall detection result obtaining module 902 includes:

the first probability obtaining submodule is used for carrying out overall object detection on each first video frame within a second preset time before the current time based on an overall image of a preset object to obtain the probability that the whole preset object exists in each first video frame;

the second video frame determining submodule is used for determining a video frame, with the probability that the whole preset article exists in each first video frame and being greater than the first probability, as a second video frame;

the first processing submodule is used for determining that a first overall detection result is that a preset article exists in the current monitoring video if the ratio of the number of the second video frames to the number of the first video frames is greater than a first threshold;

and the second processing submodule is used for determining that the first overall detection result is that no preset article exists in the current monitoring video if the ratio of the number of the second video frames to the number of the first video frames is not greater than a first threshold value.

Optionally, the first probability obtaining sub-module is specifically configured to input each first video frame within a second preset duration before the current time into a first neural network model trained in advance, respectively, to obtain a probability that the whole preset article exists in each first video frame;

Optionally, the first local detection result obtaining module 903 includes:

the third processing sub-module is used for determining that the first local detection result is that a preset article exists in the current surveillance video if a local part exists, wherein the ratio of the number of the corresponding fourth video frames to the number of each third video frame is larger than a second threshold value;

Optionally, the apparatus further comprises:

the shielding detection module is used for determining whether the camera is shielded or not based on the pixel values of the pixel points of the current monitoring video before the current monitoring video is detected based on the whole image of the preset article and the first whole detection result of the preset article exists in the current monitoring video; if yes, triggering a third door opening control signal sending module, and if not, triggering the first overall detection result obtaining module 902;

The embodiment of the present application further provides an electronic device, as shown in fig. 10, including a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, where the processor 1001, the communication interface 1002, and the memory 1003 complete mutual communication through the communication bus 1004,

a memory 1003 for storing a computer program;

the processor 1001 is configured to implement the following steps when executing the program stored in the memory 1003:

if the first overall detection result is that the preset article exists in the current monitoring video, when a first preset time length is reached, a plurality of local images based on the preset article are obtained, the current monitoring video is detected, and whether a first local detection result of the preset article exists in the current monitoring video is obtained;

The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this is not intended to represent only one bus or type of bus.

The communication interface is used for communication between the electronic equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.

The electronic equipment that this application embodiment provided can have preset article in detecting current surveillance video, promptly, when detecting preset article and get into the elevator, the control elevator is in the state of opening the door, also does not operate promptly, just also can prevent effectively that preset article from taking the elevator.

The embodiment of the application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions which when run on a computer, make the computer execute the control method of the elevator provided by the embodiment of the application.

Specifically, the elevator control method includes:

It should be noted that other implementation manners of the elevator control method are the same as those of the foregoing method embodiment, and are not described again here.

By operating the instructions stored in the computer-readable storage medium provided by the embodiment of the application, when the preset article is detected to exist in the current monitoring video, namely, when the preset article is detected to enter the elevator, the elevator is controlled to be in a door-opening state, namely, the elevator does not operate, and the preset article can be effectively prevented from taking the elevator.

The embodiment of the application also provides another computer program product containing instructions, which when run on a computer causes the computer to execute the control method of the elevator provided by the embodiment of the application.

Specifically, the elevator control method includes:

By operating the computer program product provided by the embodiment of the application, the preset article can be detected to exist in the current monitoring video, namely, when the preset article is detected to enter the elevator, the elevator is controlled to be in the door opening state, namely, the elevator does not operate, and the preset article can be effectively prevented from taking the elevator.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.

It should be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on differences from other embodiments. In particular, for the apparatus, the electronic device, the computer-readable storage medium, and the computer program product embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiments.

The above description is only for the preferred embodiment of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the scope of protection of the present application.

Claims

1. A method for controlling an elevator, characterized in that the method comprises:

detecting the current monitoring video based on the integral image of the preset article to obtain a first integral detection result of whether the preset article exists in the current monitoring video; the first overall detection result is obtained by performing article overall detection on each first video frame within a second preset time before the current moment;

if the first local detection result is that a preset article exists in the current monitoring video, sending a first door opening control signal to the elevator to enable the elevator to keep a door opening state;

if the first overall detection result is that a preset article exists in the current surveillance video, when a first preset time length is reached, a plurality of local images based on the preset article are obtained, the current surveillance video is detected, and whether a first local detection result of the preset article exists in the obtained current surveillance video or not includes:

if the first overall detection result indicates that a preset article exists in the current monitoring video, when the first preset time length is reached, a plurality of local images based on the preset article are obtained, article local detection is carried out on each third video frame within a third preset time length before the current moment, and the probability that the local part of the preset article exists in each third video frame is obtained; the third preset time length is the sum of the first preset time length and the second preset time length;

if no part with the ratio of the number of the corresponding fourth video frames to the number of the third video frames larger than a second threshold exists, determining that the first local detection result is that no preset article exists in the current monitoring video;

the method further comprises the following steps:

if the first overall detection result indicates that a preset article exists in the current monitoring video, a second door opening control signal is sent to the elevator, so that the elevator keeps a door opening state;

2. The method according to claim 1, wherein the detecting the current surveillance video based on the overall image of the preset article to obtain a first overall detection result of whether the preset article exists in the current surveillance video comprises:

determining video frames with the probability that the whole preset article exists in each first video frame and the probability is larger than a first probability to serve as second video frames;

3. The method according to claim 2, wherein the performing the overall article detection on each first video frame within a second preset duration before the current time based on the overall image of the preset article to obtain the probability that the whole preset article exists in each first video frame comprises:

4. The method according to claim 1, wherein if the first overall detection result indicates that a preset article exists in the current surveillance video, when the first preset duration is reached, obtaining a plurality of local images based on the preset article, and performing article local detection on each third video frame within a third preset duration before the current time, so as to obtain a local probability that the preset article exists in each third video frame, the method comprises:

5. The method of claim 1, wherein after sending a first door open control signal to the elevator, the method further comprises:

6. The method of claim 1, further comprising:

7. The method of claim 1, further comprising:

if the first local detection result indicates that a preset article exists in the current monitoring video, sending a first reminding message to a preset terminal to remind a user of the preset article entering the elevator;

and when receiving a control releasing message sent by the preset terminal, stopping sending the first door opening control signal to the elevator.

8. The method according to claim 1, wherein before detecting the current surveillance video based on the overall image of the preset article to obtain a first overall detection result of whether the preset article exists in the current surveillance video, the method further comprises:

if the elevator is determined to be shielded, sending a third door opening control signal to the elevator so as to enable the elevator to keep a door opening state;

and if the whole image based on the preset article is determined not to be blocked, detecting the current monitoring video to obtain a step of judging whether a first whole detection result of the preset article exists in the current monitoring video.

9. A control device of an elevator, characterized in that the device comprises:

the first overall detection result acquisition module is used for detecting the current monitoring video based on the overall image of the preset article to obtain a first overall detection result of whether the preset article exists in the current monitoring video; the first overall detection result is obtained by performing overall object detection on each first video frame within a second preset time before the current moment;

a first local detection result obtaining module, configured to, if the first overall detection result is that a preset article exists in the current surveillance video, obtain a plurality of local images based on the preset article when a first preset duration is reached, detect the current surveillance video, and obtain a first local detection result whether the preset article exists in the current surveillance video;

the first door opening control signal sending module is used for sending a first door opening control signal to the elevator to enable the elevator to keep a door opening state if the first local detection result indicates that a preset article exists in the current monitoring video;

the first local detection result obtaining module includes:

a second probability obtaining sub-module, configured to, if the first overall detection result is that a preset article exists in the current surveillance video, obtain, when a first preset duration is reached, a plurality of local images based on the preset article, perform article local detection on each third video frame within a third preset duration before the current time, and obtain a probability that a local part of the preset article exists in each third video frame; the third preset time length is the sum of the first preset time length and the second preset time length;

the fourth video frame determining sub-module is used for determining a fourth video frame, of which the probability of the part is larger than the second probability, in each third video frame aiming at each part of the preset article;

the fourth processing sub-module is used for determining that the first local detection result is that no preset article exists in the current surveillance video if no part exists, wherein the ratio of the number of the corresponding fourth video frames to the number of the third video frames is larger than a second threshold;

the device further comprises:

10. The apparatus of claim 9, wherein the first overall detection result obtaining module comprises:

11. The apparatus according to claim 10, wherein the first probability obtaining sub-module is specifically configured to input each first video frame within a second preset duration before the current time into a first pre-trained neural network model, so as to obtain a probability that the whole preset item exists in each first video frame;

12. The apparatus according to claim 9, wherein the second probability obtaining sub-module is specifically configured to, if the first overall detection result indicates that a preset article exists in the current surveillance video, obtain, when a first preset duration is reached, a probability that a local part of the preset article exists in each third video frame obtained by inputting each third video frame within a third preset duration before the current time into a second pre-trained neural network model;

13. The apparatus of claim 9, further comprising:

14. The apparatus of claim 9, further comprising:

15. The apparatus of claim 9, further comprising:

16. The apparatus of claim 9, further comprising:

the shielding detection module is used for determining whether the camera is shielded or not based on the pixel values of the pixel points of the current monitoring video before the current monitoring video is detected based on the whole image of the preset article and the first whole detection result of the preset article exists in the current monitoring video; if yes, triggering a third door opening control signal sending module, and if not, triggering the first overall detection result acquisition module;

and the fourth processing module is used for stopping sending the third door opening control signal to the elevator when a fourth preset time length is reached, and sending a second reminding message to a preset terminal under the condition that the camera is detected to be blocked so as to remind a user that the camera is blocked.

17. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

the memory is used for storing a computer program;

the processor, when executing the program stored in the memory, implementing the method steps of any of claims 1-8.

18. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of the claims 1-8.