JP2023510477A - Image detection method and device, electronic device, and storage medium - Google Patents

Abstract

TECHNICAL FIELD The present application relates to an image detection method and apparatus, an electronic device, and a storage medium. The method includes obtaining a first image of an escalator, performing region detection on the first image to determine a first region of the escalator in the first image, and corresponding to the first region. and determining at least one state identification result of the escalator, wherein the state identification result indicates that the escalator is in an empty state or a non-empty state. and determining a state of the escalator based on the at least one state identification result.

Description

(Cross reference to related applications)
This application claims priority from a Chinese patent application with application number 202011559951.4 filed on Dec. 25, 2020, the entire content of which is incorporated herein by reference.

TECHNICAL FIELD The present application relates to the technical field of image detection and computer vision, and more particularly to an image detection method and apparatus, an electronic device, and a storage medium.

With the continuous improvement of economic growth and infrastructure development, escalators are also becoming more and more widely applied in shopping malls, office buildings, public transportation and other occasions. During the daily operation of the escalator, people need to strictly manage the area where the escalator is located in order to ensure the operation of the escalator and avoid accidents. In the related art, the automatic detection method for the escalator area generally uses infrared sensing or gravity sensing to realize the start and stop of the escalator.

Embodiments of the present application provide a technical solution for image detection.

According to one aspect of embodiments of the present application, an image detection method is provided. The method includes obtaining a first image of an escalator, performing region detection on the first image to determine a first region of the escalator in the first image, and corresponding to the first region. and determining at least one state identification result of the escalator, wherein the state identification result indicates that the escalator is in an empty state or a non-empty state. and determining a state of the escalator based on the at least one state identification result.

In a possible implementation, determining the state of the escalator based on the at least one state identification result comprises, if the state identification result is a plurality, a plurality of state identification results and a combination of the plurality of state identification results. determining a state discrimination value of the escalator based on the weight; and determining that the escalator is in an empty state when the state discrimination value is greater than or equal to a first threshold.

In a possible implementation, the state identification results comprise a first state identification result, performing state identification of an escalator on a first region image corresponding to the first region, and at least one state identification result of the escalator. includes performing a classification process on the first area image to obtain a first state identification result of the escalator.

In a possible implementation, the state identification results comprise a second state identification result, performing state identification of an escalator on a first region image corresponding to the first region, and at least one state identification result of the escalator. dividing the first area image into a background area and a foreground area where the escalator is located; and adjusting pixel values of the background area. to obtain an adjusted second area image; and performing a classification process on the second area image to obtain a second state identification result of the escalator.

In a possible implementation, the state identification results comprise a third state identification result, performing state identification of an escalator on a first region image corresponding to the first region, and at least one state identification result of the escalator. Determining is performing pixel matching on the first area image and a predetermined reference image, and determining a ratio of matching areas between the first area image and the reference image, wherein the reference image includes an area image corresponding to an escalator in an empty state; and if the proportion of the matching area is greater than or equal to a second threshold, the third state identification result is that the escalator is in an empty state. determining. In a possible implementation, the state identification results comprise a fourth state identification result, performing state identification of an escalator on a first region image corresponding to the first region, and at least one state identification result of the escalator. includes performing first target detection on the first area image to determine whether a first target is present in the first area image; is not present, determining that the fourth state identification result is that the escalator is in an empty state.

In a possible implementation, the method is to send an escalator stop signal when the escalator is in an empty state, the escalator stop signal for instructing the escalator to stop running. It further includes that there is.

In a possible implementation, the method is to send an escalator start signal when the escalator is in a non-empty state and the escalator has already stopped running, wherein the escalator start signal is for instructing the escalator to do so.

In a possible implementation, the method comprises performing a second target detection on the first image to determine a third region of the second target in the first image; Determining a detection result of the second target based on the positional relationship with the area, wherein the detection result indicates that the second target is located on the escalator or not located on the escalator. further comprising comprising; In a possible implementation mode, determining the detection result of the second target based on the positional relationship between the first area and the third area includes: 3 threshold or more, the detection result is determining that the second target is located on the escalator, and the fourth area is between the first area and the third area. including common areas.

In a possible implementation, the second target comprises prohibited items on an escalator, and the method further comprises sending warning information if the second target is located on the escalator. According to one aspect of an embodiment of the present application, an image detection apparatus is provided. The apparatus comprises an image acquisition module configured to acquire a first image of an escalator, and performing region detection on the first image to determine a first region of the escalator in the first image. and a state identification module configured to perform state identification of an escalator on a first region image corresponding to said first region to determine at least one state identification result of said escalator. a state identification module, wherein the state identification results include that the escalator is in an empty state or a non-idle state; and determining a state of the escalator based on the at least one state identification result. a state determination module configured to:

According to one aspect of an embodiment of the present application, an electronic device is provided. The electronic device comprises a processor and a memory configured to store instructions executable by the processor, the processor calling the instructions stored in the memory to perform the method. Configured.

According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided. The computer readable storage medium stores computer program instructions which, when executed by a processor, implement the above method.

According to one aspect of embodiments of the present application, a computer program product is provided. The computer program includes computer readable code, and when the computer readable code is executed in the device, the processor in the electronic device performs the method.

According to embodiments of the present application, detecting a location area of an escalator in an image, identifying at least one status identification result based on the area image of the escalator, and determining an empty status of the escalator based on the at least one status identification result. Or the non-empty status can be determined, the accuracy of locating the area of the escalator can be improved, and the accuracy rate of identifying the running status of the escalator can be improved.

It is to be understood that the general descriptions above and the detailed descriptions that follow are exemplary and explanatory only and are not restrictive. Other features and aspects of the present application will become apparent with reference to the following detailed description of exemplary embodiments based on the drawings.

4 shows a flow chart of an image detection method according to an embodiment of the present application; Fig. 2 shows a schematic diagram of region detection of an image detection method according to an embodiment of the present application; Fig. 2 shows a schematic diagram of region detection of an image detection method according to an embodiment of the present application; 1 shows a schematic diagram of a first region according to an embodiment of the present application; FIG. Fig. 2 shows a schematic diagram of a second target according to an embodiment of the present application; Fig. 2 shows a schematic diagram of a second target according to an embodiment of the present application; Fig. 3 shows a schematic diagram of a processing process of an image detection method according to an embodiment of the present application; 1 shows a block diagram of an image sensing device according to an embodiment of the present application; FIG. 1 shows a block diagram of an electronic device according to an embodiment of the present application; FIG. 1 shows a block diagram of an electronic device according to an embodiment of the present application; FIG.

The drawings attached hereto are incorporated into the specification and constitute a part of the specification, and the drawings show the embodiments compatible with the present application, and together with the description, interpret the technical solution of the present application. used for

Various exemplary embodiments, features and aspects of embodiments of the present application are described in detail below with reference to the drawings. The same reference numerals in the drawings indicate elements having the same or similar functions. The drawings, which illustrate various aspects of the embodiments, are not necessarily drawn to scale unless specifically stated otherwise.

As used herein, the term "exemplary" means "serving as an example, example, or for the purpose of explanation." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

As used herein, the term “and/or” is used to describe a related relationship between related objects, and indicates that there are three types of relationships. For example, A and/or B represents three cases: only A is present, A and B are present at the same time, and only B is present. Also, as used herein, the term "at least one" represents any one of the plurality or any combination of at least two of the plurality. For example, including at least one of A, B, and C means including any one or more elements selected from the set consisting of A, B, and C.

It is noted that many specific details are set forth in the specific embodiments below in order to better explain the present application. It should be understood by those skilled in the art that the present disclosure may be similarly practiced regardless of these specific details. In order to keep the subject matter of the present invention clear, in some instances methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail.

In recent years, with the trend of artificial intelligence based on deep learning, industry + artificial intelligence (AI) has become one of the goals of industrial development in the new era. New areas such as smart factories, smart stores, and smart agriculture are constantly emerging. Among them, the industrial application of using AI for crime prevention is called smart crime prevention, and such applications are being actively carried out. In the field of public safety, escalators are common alternatives to walking in modern life, and are public places that need to be governed by strict regulations. The purpose of the smart escalator is to improve the work efficiency of the management department and reinforce the monitoring and management of the daily operation of the escalator. In the related art, generally only partial detection functions for escalators can be realized, and there is no complete smart escalator solution.

Image detection methods according to embodiments of the present application may be used in scenes such as sales floors, office buildings, and public transportation. Based on the method of deep learning, the method processes and analyzes the image or video stream of the location area of the escalator in the scene to locate the escalator area, determine the escalator's vacant escalator and non-vacant state, And it can realize functions such as detection and identification of key targets (such as strollers, wheelchairs, luggage cases and other objects), thereby building a complete smart escalator solution, improving the detection effect and improving the escalator operating costs and reduce the risk of safety incidents.

FIG. 1 shows a flowchart of an image detection method according to an embodiment of the present application. As shown in FIG. 1, the image detection method includes: a.

In step S11, a first image of the escalator is acquired.

In step S12, region detection is performed on the first image to determine a first region of the escalator in the first image.

In step S13, state identification of the escalator is performed on the first region image corresponding to the first region, and at least one state identification result of the escalator is determined, and the state identification result indicates that the escalator is in an empty state. including being present or being in a non-idle state.

In step S14, a state of the escalator is determined based on the at least one state identification result.

In a possible implementation, the image detection method may be performed by an electronic device such as a terminal device or a server. The terminal equipment may be User Equipment (UE), a mobile equipment, a user terminal, a terminal, a handheld equipment, a computing equipment, an in-vehicle equipment, a wearable equipment, etc., and the method is stored in a memory by a processor. may be implemented by invoking computer readable instructions. Alternatively, the method may be performed by a server.

For example, at least one image acquisition device may be provided at the location of the escalator to be detected, for example, at least one camera facing the escalator, thereby collecting a video stream of the escalator, escalator, escalator in the video stream screen Detection is performed on targets (eg, passers-by, objects held by passers-by, etc.). Embodiments of the present application do not limit the mounting position of the image acquisition device, the acquisition method of the video stream, or the area corresponding to the video stream.

In a possible implementation, step S11 may obtain a first image of the escalator. Each image frame of the video stream may be the first image, the video stream may be sampled at regular time intervals, the sampled image frame may be the first image, and the key frame in the video stream may be the first image. can be obtained as Embodiments of the present application do not limit the acquisition method of the first image.

In a possible implementation, step S12 may perform region detection on the first image by a trained region detection network to determine the first region of the escalator in the first image. The region detection network may be, for example, a convolutional neural network, and the embodiments herein do not limit either the network structure or the training scheme of the region detection network.

2a and 2b show schematic diagrams of region detection of an image detection method according to an embodiment of the present application. FIG. 2a shows the first image of the escalator and FIG. 2b shows the region detection result in the first image.

In possible implementations, the region detection network may use a target detection model or a region segmentation model. When using the target detection model, the first detected area may be a rectangular detection window of the escalator location area, eg detection window 21 in FIG. 2b. When using a segmentation model, the first region detected may be the irregular region where the escalator is located, eg region 22 in FIG. 2b. Embodiments of the present application do not limit the network model used for the area detection network.

When processing using the target detection model, the full escalator area is included in the detection frame, and the background area is reduced to the maximum, thereby reducing the effect of background noise on subsequent processing processes. As shown in FIG. 2b, when the direction of the escalator deviates to the left, the lower left corner of the dark-colored escalator area is used as the reference for the lower left corner of the detection frame 21, and the upper right corner of the dark-colored escalator area is used as the reference for detection. The upper right vertex coordinates of the frame 21 are set. When processing using the region segmentation model, the dark escalator region in the detection frame may be obtained by segmentation, and a polygon may be drawn with the escalator handrail as the boundary, such as region 22 in FIG. 2b. as shown in

In a possible implementation, after obtaining the first area, in step S13, from the first image, cut out a first area image corresponding to said first area, and identify the state of the escalator based on the first area image. i.e., identify whether the escalator is empty or non-empty. When there are no pedestrians and/or articles on the escalator, it is in an empty state, and when there are pedestrians and/or articles on the escalator, it is in a non-empty state.

In a possible implementation, each of the at least one identification scheme can perform identification on the first area image to obtain a corresponding state identification result. The identification method includes, for example, performing classification directly on the first area image, adjusting the pixel values of the background area in the first area image, and performing classification on the adjusted image, and performing classification on the adjusted image. It may include at least one of comparing the one-region image to a reference image of an empty escalator and detecting whether there is a target, such as a passerby, on the escalator. Embodiments of the present application do not limit any of the identification schemes used.

In a possible implementation, in step S14, the state of the escalator may be determined based on the at least one state identification result. For example, based on the value and weight of each state identification result with reference to a voting mechanism, determine the state of the escalator collectively. When the overall weight of the state identification result being empty is higher, it is determined that the escalator is in an empty state, and conversely, it is determined that the escalator is in a non-empty state.

In a possible implementation, if the escalator is empty, an escalator stop signal may be sent, instructing the escalator to stop running. If the escalator is in a non-empty state and has already stopped running, an escalator activation signal may be sent to instruct the escalator to start running. Embodiments of the present application do not limit the operations performed in different escalator states.

According to the image detection method of the embodiment of the present application, the location area of the escalator in the image is detected, at least one state identification result is identified based on the area image of the escalator, and based on the at least one state identification result, The empty or non-empty status of the escalator can be determined, the accuracy of locating the area of the escalator can be improved, and the accuracy rate of identifying the running status of the escalator can be improved.

The image detection method according to the embodiments of the present application will be described in detail below.

As mentioned above, the camera may collect a video stream of the area where the escalator is located and transmit the collected video stream to an electronic device such as a local front server or a cloud server. The electronic device can perform decoding on the video stream to obtain a decoded video stream.

In step S11, a first image of the escalator may be obtained. The first image may be an image frame of a decoded video stream. In step S12, region detection is performed on the first image by a trained region detection network to determine a detection frame of a first region of the escalator in the first image, for example, the location region of the escalator.

In step S13, a first area image corresponding to the first area is cut out from the first image, each of the first area images is identified by at least one identification method, and at least one state identification result is obtained. You may get

FIG. 3 shows a schematic diagram of a first region according to an embodiment of the present application. As shown in FIG. 3, the first image includes first regions 31 and 32 of two escalators. The processing efficiency can be improved by cutting out the first area images corresponding to the first areas 31 and 32 and performing identification on the two first area images.

In a possible implementation, the state identification result includes a first state identification result. Step S13 is
The method may include performing a classification process on the first area image to obtain a first state identification result of the escalator.

That is, the classification may be performed directly on the first region by the trained classification network (herein referred to as the first classification network). The first classification network may be, for example, a convolutional neural network including a convolutional layer, a fully connected layer, an activation layer, etc., and the embodiments of the present application both the network structure and the training scheme of the first classification network Not limited.

In a possible implementation, a first area image may be input to a first classification network and output a first state identification result, indicating whether the escalator is in an empty or non-empty state, e.g. , outputs 1 if the escalator is in an empty state, and outputs a 0 if it is in a non-empty state. In this manner, the status of the escalator can be easily and effectively identified.

In a possible implementation, the state identification result includes a second state identification result. Step S13 is
dividing the first area image into a background area and a foreground area where the escalator is located;
adjusting the pixel values of the background area to obtain an adjusted second area image;
performing a classification process on the second area image to obtain a second state identification result of the escalator.

For example, if the first region is a rectangular detection frame, the trained segmentation network performs segmentation on the first region image corresponding to the first region, and divides the first region image into the background region and the location of the escalator. It can be divided into foreground regions. The partitioned network may be, for example, a convolutional neural network including convolutional layers, fully connected layers, activation layers, etc., and the embodiments of the present application do not limit either the network structure or the training scheme of the classification network.

In a possible implementation, if the first area is an irregular area in which an escalator is located, the area image corresponding to the circumscribed rectangular frame of the first area may be the first area image. By directly setting the first area as the foreground area and setting the area other than the first area in the first area image as the background area, the division of the first area image can be realized.

In a possible implementation, an adjustment may be made to the pixel values of the pixels in the background region, for example, all pixel values in the background region may be adjusted to zero (black) to obtain an adjusted second region image. . The pixel values of the background area may be adjusted to other values. Embodiments of the present application do not limit this.

In a possible implementation, classification may be performed on the second area image by a trained classification network (herein referred to as the second classification network). The second classification network may be, for example, a convolutional neural network comprising a convolutional layer, a fully connected layer, an activation layer, etc., wherein the second classification network and the first classification network have the same network structure. but the network parameters may be different. Embodiments of the present application do not limit either the network structure or the training scheme of the second classification network.

In a possible implementation, the second area image may be input to a second classification network and output a second state identification result indicating whether the escalator is in an empty or non-empty state, e.g. , outputs 1 if the escalator is in an empty state, and outputs a 0 if it is in a non-empty state. In this way, by combining image division and performing classification processing on the second area image after pixel adjustment, the accuracy rate of escalator state identification can be improved.

In a possible implementation, the state identification result includes a third state identification result. Step S13 is
performing pixel matching on the first area image and a predetermined reference image to determine a proportion of a matching area between the first area image and the reference image, wherein the reference image is in an empty state; including an area image corresponding to the escalator;
determining that the third state identification result is that the escalator is in an empty state if the percentage of matching areas is greater than or equal to a second threshold.

For example, if the escalator is in an empty state, single or multiple area images corresponding to the escalator may be acquired. For example, an image of an escalator is collected by a camera, area detection is performed on the image, and an area image is obtained.

In a possible implementation, if a single regional image is obtained, that regional image may be used as the reference image. If multiple area images are acquired, the multiple area images may be fused to obtain a reference image (empty escalator template). Long-term image acquisition, eg, 1-2 days, obtains labels for all pixels in the escalator region using a segmentation model, and obtains an empty escalator template using a Gaussian mixture model. Embodiments of the present application do not limit the method of generating the reference image.

In a possible implementation, the reference images may be stored in a database. When identifying the state of the escalator, pixel matching is performed on the first region image and the reference image, the number of matched pixels is determined, and the first A percentage of matching areas between the area image and the reference image may be determined.

In a possible implementation, it may be determined that the third state identification result is that the escalator is in an empty state if the percentage of matching areas is greater than or equal to a second threshold. Conversely, if the percentage of matching areas is less than the second threshold, it may be determined that the third status identification result is that the escalator is in a non-empty status. A person skilled in the art may set the second threshold according to the actual situation, for example, set it to 0.8. Embodiments of the present application do not limit the actual value of the second threshold.

The method of pixel matching can improve the efficiency of escalator status identification.

In a possible implementation, the state identification result includes a fourth state identification result. Step S13 is
performing a first target detection on the first area image to determine if a first target is present in the first area image;
and determining that the fourth state identification result is that the escalator is in an empty state if a first target is not present in the first area image.

For example, a trained target detection network (which may be referred to as a first detection network) performs first target detection on a first area image and determines whether the first target is present in the first area image. may decide. The first target may include, for example, passers-by, items, etc., and embodiments herein are not so limited.

In possible implementations, the first detection network may be, for example, a convolutional neural network, and the embodiments herein do not limit either the network structure or the training scheme of the first detection network.

In a possible implementation, if there is a first target in the first area image, it may be determined that the fourth state identification result is that the escalator is in a non-empty state. Conversely, if there is no first target in the first area image, it may be determined that the fourth state identification result is that the escalator is in an empty state. By setting the first target and performing target detection on the first area image, the versatility of the escalator state identification method can be improved.

After determining each state identification result, in step S14, the state of the escalator may be determined based on each state identification result. Step S14 determines a state discrimination value of the escalator based on a plurality of state identification results and weights of the plurality of state identification results when there are a plurality of state identification results;
determining that the escalator is in an empty state when the state discrimination value is greater than or equal to a first threshold.

For example, if there is one state identification result, the state of the escalator may be determined directly based on the state identification result. If there are a plurality of state identification results, the state of the escalator may be comprehensively determined based on the plurality of state identification results.

In a possible implementation, the weight of each state identification result is preset, the weight of the state identification result with a high accuracy rate is set high, the state identification result with a low accuracy rate is set low, and the weight of each state identification result is set to The sum is 1.

In a possible implementation, the state discrimination value of the escalator may be determined based on a plurality of state identification results and weights of the plurality of state identification results with reference to a voting mechanism. The state identification results may include at least two of said first, second, third and fourth state identification results.

For example, if the state identification results include the first state identification result a ₁ , the second state identification result a ₂ , the third state identification result a ₃ and the fourth state identification result a ₄ , the first, second, and third and the weights of the fourth _{state identification} results are _{w 1 , w 2 ,} w ₃ and w _{4 respectively .} be. where w ₁ +w ₂ +w ₃ +w ₄ =1 and a ₁ , a ₂ , a ₃ , a ₄ output 1 if the escalator is vacant and 0 if it is non-empty. to output

In a possible implementation, if the state discrimination value is greater than or equal to a predetermined first threshold, the escalator is considered to be in an empty state; conversely, if the state discrimination value is less than a predetermined first threshold, the escalator is considered non-empty. can be considered to be in a state As shown in FIG. 3, the escalator in area 31 is in an empty state and the escalator in area 32 is in a non-empty state.

A person skilled in the art may set the first threshold according to the actual situation, for example, set it to 0.5. Embodiments of the present application do not limit the value of the first threshold.

With the above-described method of judging by voting mechanism, the running status of the escalator can be accurately identified, and the error rate of judging the escalator status can be significantly reduced.

In a possible implementation, an image detection method according to embodiments of the present application comprises:
The method may further include transmitting an escalator stop signal when the escalator is in an empty state, the escalator stop signal being for instructing the escalator to stop running.

That is, if it is determined in step S14 that the escalator is in an empty state, an escalator stop signal may be generated and transmitted to instruct the escalator to stop operating. For example, sending an escalator stop signal to an escalator control device. The escalator control device thereby controls the escalator to stop running. Additionally, an escalator stop signal may be sent to the operator. The operator thereby controls the escalator to stop running. Embodiments of the present application do not limit the type and transmission scheme of the escalator stop signal.

Such a scheme allows the escalator to be shut down when it is idle, thereby reducing the operating cost of the escalator.

In a possible implementation, an image detection method according to embodiments of the present application comprises:
transmitting an escalator start signal when the escalator is in a non-empty state and the escalator has already stopped running, wherein the escalator start signal is for instructing the escalator to run may further include:

That is, if it is determined in step S14 that the escalator is in a non-empty state and that the escalator has already stopped operating, that is, if it is determined that there is a person riding the escalator during the stop period, an escalator start signal is generated and transmitted. , may instruct the escalator to drive. Similar to the above scheme, an escalator activation signal may be sent to the escalator control device. The escalator control device thereby controls the escalator to run. Additionally, an escalator activation signal may be sent to the operator. The operator thereby controls the escalator as it drives. Embodiments of the present application do not limit the type and transmission scheme of the escalator activation signal.

In this manner, when there are people on the escalator and the operation of the escalator is stopped, the escalator can be used normally by starting the escalator.

In a possible implementation, an image detection method according to embodiments of the present application comprises:
performing second target detection on the first image to determine a third region of the second target in the first image;
determining a detection result of the second target based on the positional relationship between the first area and the third area, wherein the detection result is that the second target is positioned on the escalator; or comprising not being located on the escalator.

For example, in step S11, after obtaining a first image of an escalator, a second target detection is further performed on the first image by a trained target detection network (which may be referred to as a second detection network). , may determine whether a second target is present in the first image. The secondary targets may include prohibited items on escalators, such as strollers, wheelchairs, large luggage cases, etc., and the embodiments herein do not limit the type of secondary targets.

In possible implementations, the second detection network may be, for example, a convolutional neural network, and the embodiments herein do not limit either the network structure or the training scheme of the second detection network.

In a possible implementation, if a second target is present in the first image, the second detection network can determine the region of the second target in the first image (which may be referred to as the third region). . Figures 4a and 4b show schematic diagrams of a second target according to embodiments of the present application. The second target in FIG. 4 a is the stroller 41 and the second target in FIG. 4 b is the luggage case 42 . As shown in Figures 4a and 4b, a detection window or third region in which the second target is located can be determined.

In a possible implementation, based on the first area of the escalator in the first image, the positional relationship between the first area and the third area may determine the detection result of the second target. That is, based on the positional relationship, it is determined whether the second target is positioned on the escalator.

In a possible implementation, the step of determining the detection result of the second target based on the positional relationship between the first area and the third area comprises:
determining that the detection result is that the second target is positioned on the escalator if an area ratio between the fourth area and the third area is greater than or equal to a third threshold; The region may include a common region of the first region and the third region.

In other words, the determination may be made based on the concept of finding the intersection-over-union (IOU). If the fourth area includes the common area of the first area and the third area, the area ratio of the fourth area and the third area, that is, the area of the common area of the second target location area and the escalator area and the area of the area where the second target is located may be calculated.

In a possible implementation, if the area ratio is greater than or equal to a predetermined third threshold, it may be determined that the detection result is that the second target is located on the escalator. Conversely, if the area ratio is less than a predetermined third threshold, it may be determined that the detection result is that the second target is not located on the escalator. A person skilled in the art may set the third threshold according to the actual situation, for example, set it to 0.6. Embodiments of the present application do not limit the value of the third threshold.

With such a method, it is possible to determine the location area of the second target.

In a possible implementation, the second detection network may perform the second target detection directly on the first area image of the escalator area to determine if the second target is present in the first area image. good. If the second target is present in the first area image, it may be determined that the second target is located on the escalator. Embodiments of the present application do not limit the processing schemes used.

In a possible implementation, the second target includes prohibited items on an escalator, and the image detection method according to the embodiments of the present application sends warning information if the second target is located on the escalator. may further include

In other words, for prohibited items on the escalator, if it is determined in the above step that the second target is located on the escalator, it will send warning information, remind the escalator, or directly stop the escalator. may be controlled.

For example, alert information may be sent to escalator monitoring equipment and/or workers in the monitoring room. The operator thereby controls the escalator to stop running and/or goes to the escalator for treatment. Alert information may be sent to the escalator control device. The escalator control device thereby controls the escalator to stop running. Embodiments of the present application do not limit the type and transmission method of alert information.

In a possible implementation, an alert is provided in response to the first appearance of a second target on the escalator, and subsequent successive appearances of the second target are repeatedly alerted at predetermined time intervals. , it is possible to avoid sending the warning information too frequently.

In this manner, a timely warning can be given when prohibited items enter the escalator, thereby reducing the risk of safety accidents.

FIG. 5 shows a schematic diagram of a processing process of an image detection method according to an embodiment of the present application. As shown in FIG. 5, in the processing process, in the image input step 51, the image of the escalator is input, and the image is subjected to the escalator positioning step 52 and the key target detection step 53, respectively, to improve the processing efficiency of the event. . An escalator location step 52 determines an escalator region 54 in the image. In addition, the escalator determination step 55 determines that the escalator is vacant/not vacant 57, and performs corresponding processing.

In the example, the key target detection step 53 determines a key target location 56 in the image, where the key target is a prohibited item (unauthorized item) on the escalator. Based on escalator area 54 and key target location 56, determine whether the key target is located on the escalator. If the key target is located on an escalator, a fraudulent item alert 58 is provided. In this way, the entire processing process of the image detection method according to the embodiments of the present application can be realized.

In the escalator positioning step 52, input the first image of the escalator as shown in FIG. 2a, i.e. the elevator, build a target detection or segmentation model based on the deep learning algorithm, and detect the escalator detection frame 21 as shown in FIG. 2b. or the segmentation result of the region 22 shown in FIG. 2b.

The target detection method should ensure that the detection window includes the escalator area and reduces the background area as much as possible to prevent background noise from interfering with subsequent escalator operation. Therefore, the bounding frame of the escalator in FIG. 2b is defined as detection frame 21 . At the same time, pay attention to where the escalator is heading. In FIG. 2b it is offset to the left and is referenced to the lower left corner of the escalator area (the lower left coordinate of the bounding box is determined by this) and the right boundary is referenced to the upper right corner of the escalator area. Using the segmentation model to build a more accurate escalator area positioning module, the escalator area is defined as the escalator area within the detection frame 21, and a polygon is drawn with the escalator handrail as the boundary, for example, Fig. 2b as shown in region 22 in .

In the vacant escalator determination step 55, based on deep learning, build a multi-parallel escalator state classification model, receive the processing result (detection result as an example) of the escalator positioning module, and use a plurality of technical solutions to Simultaneous data processing and finally fusion of the results to determine the empty status of the escalator, ie, an empty escalator or a non-empty escalator. As shown in FIG. 3, the left escalator 31 is in an empty state and the right escalator 32 is in a non-empty state. The process may be implemented in several ways as follows.

In method 1, the detection frame area is extracted, and the escalator status is output by the classification network model.

In method 2, the detection frame area is extracted, the pixel value of the background area within the detection frame is set to 0 (black) while referring to the divided areas, and the state of the escalator is output by the classification network model.

In method 3, according to the template matching method, the input and the empty escalator template in the database are matched, and the ratio of the matching area is output, and the matching ratio is T (corresponding to the second threshold, T≧0. 8), determine that the input escalator is an empty escalator, otherwise, determine that it is a non-empty escalator. After 1-2 days of long-term data acquisition, a segmentation model is used to obtain the labels of all pixels in all escalator regions, and finally, a Gaussian mixture model is used to generate the empty escalator template (corresponding to the reference image). to do). The percentage of matching area is given by the ratio of the same number of pixels/total number of pixels.

In method 4, the escalator area is taken as input based on the pedestrian and key target detection model. If there is no target of interest within the escalator area, the output is an empty escalator, otherwise a non-empty escalator.

The final result is determined according to a voting mechanism. The weights for schemes 1 through 4 are w ₁ , w ₂ , w ₃ and w ₄ respectively, where w ₁ +w ₂ +w ₃ +w ₄ =1. With the above voting mechanism, the error rate of escalator type determination can be greatly reduced. When the idle state gets a higher weight, it triggers an informational notification mechanism.

The key target location determination step S56 supports detection of key targets such as strollers, wheelchairs, luggage cases, etc., and supports warning for multiple events. The output results of the model are the detection target (corresponding to the second target) of the stroller 41 shown in FIG. 4a and the detection target (corresponding to the second target) of the luggage case 42 shown in FIG. 4b. Finally, a warning is provided by combining the processing results of the escalator positioning step 52 and determining whether the detected target is in the escalator area. The calculation method regarding whether the detection target exists in the escalator area is: overlap ratio=common area of detection target and escalator area/area of detection target. If the percentage of overlap is greater than T (0<T<1, typically between 0.5 and 1), it triggers a warning mechanism. Target tracking provides for alerting only when a single target appears, and repeatedly alerting subsequent consecutive appearances of the same target at specified time intervals.

The image detection method according to the embodiments of the present application builds the entrance of the smart escalator solution based on the escalator area positioning, and in the escalator area, using multiple parallel empty escalator determination methods and result voting mechanism, Judge whether an escalator is an empty escalator, improve the accuracy of escalator prediction for empty escalators, and guide the management department to reduce the escalator's running speed and even stop running by warning the escalator, thereby reducing energy consumption. while ensuring the safety and convenience of going out, based on the target detection algorithm, locate the key target in the escalator area, warn against unauthorized objects, and reinforce the monitoring and management of the administrative department. can do.

The image detection method according to the embodiments of the present application can locate the escalator area based on the method of deep learning target detection/dividing model, and can support multiple escalator areas to be located at the same time by multi-target detection. . And the two area definition methods proposed in the method can ensure the complete coverage of the escalator area and reduce the interference caused by background noise. In related art smart escalator solutions, there is no solution for escalator positioning.

The image detection method according to the embodiments of the present application is based on multiple parallel vacancy determination methods, combined with a result voting mechanism, can determine whether the escalator is vacant or not, and greatly reduces the error rate of the output result. It can also reduce and support parallel classification for multiple regions. The related art solutions generally use a single method to determine the escalator empty status, and the accuracy rate is low.

An image detection method according to embodiments of the present application detects specific fraudulent items and identifies all key targets within the field of view by training a deep neural network. In the related art, to solve the problem that illegal items such as strollers, wheelchairs, large luggage cases, etc. can not be detected, and based on the escalator area and the key target area, use a specific IOU calculation method to detect illegal items on the escalator Improve the accuracy rate of judgment by judging whether or not you are riding.

An image detection method according to embodiments of the present application provides a smart escalator system based on automatic positioning and key target detection. It can form a complete and stable set of smart escalator solutions and can be applied to all current public escalator scenes.

The method is applied to a smart camera to perform escalator area positioning and empty escalator discrimination for the escalator scene, reduce energy consumption when the escalator is in an empty state, and the escalator is closed before the escalator stops running. It can ensure that it is in a free state and improve the work efficiency of the management department. For the escalator scene, carry out escalator area positioning and key target detection, warn against illegal items on the escalator, and strengthen the monitoring and management of the management department.

The embodiments of the above methods referred to in the present application can be combined with each other to form combined embodiments without departing from the principle and logic, and due to the limited number of pages, they will not be described one by one in the present application. should be understood. It should be understood by those skilled in the art that in the above methods of specific embodiments, the specific execution order of each step depends on its function and possible underlying logic.

The embodiments of the present application further provide an image detection device, an electronic device, a computer-readable storage medium, and a program. Any of the above may be used to implement any one image detection method according to the embodiments of the present application, and the related technical solutions and descriptions refer to the description of the method and are omitted here.

FIG. 6 shows a block diagram of an image sensing device according to an embodiment of the present application. As shown in FIG. 6, the device comprises:
an image acquisition module 61 configured to acquire a first image of the escalator;
a region detection module 62 configured to perform region detection on the first image to determine a first region of the escalator in the first image;
a state identification module 63 configured to perform state identification of an escalator on a first region image corresponding to the first region and determine at least one state identification result of the escalator, wherein the state identification result includes that the escalator is in an empty or non-empty state; a status identification module 63;
a status determination module 64 configured to determine a status of the escalator based on the at least one status identification result.

In a possible implementation, the state determination module is configured to determine a state discrimination value of the escalator based on a plurality of state identification results and a weight of the plurality of state identification results when there are a plurality of the state identification results. and a state determination sub-module configured to determine that the escalator is in an empty state when the state discriminant value is greater than or equal to a first threshold.

In a possible implementation, the state identification result comprises a first state identification result, and the state identification module performs a classification operation on the first area image to obtain a first state identification result of the escalator. a first result determination sub-module configured as:

In a possible implementation, the state identification result comprises a second state identification result, and the state identification module performs a segmentation on the first area image, dividing the first area image into a background area and an area of the escalator. a segmentation sub-module configured to segment into a foreground region located therein; and a pixel adjustment sub-module configured to adjust pixel values of the background region to obtain an adjusted second region image; a second result determination sub-module configured to perform a classification process on the second area image to obtain a second state identification result of the escalator.

In a possible implementation, the state identification result comprises a third state identification result, and the state identification module performs pixel matching on the first area image and a predetermined reference image to obtain the first area image and a pixel matching sub-module configured to determine a proportion of a matching area with the reference image, wherein the reference image includes an area image corresponding to an escalator in an empty state; a third result determination sub-module configured to determine that the third state identification result is that the escalator is in an empty state if the percentage of matching areas is greater than or equal to a second threshold.

In a possible implementation, the state identification result comprises a fourth state identification result, the state identification module performing first target detection on the first area image, and detecting a first target on the first area image. and if the first target is not present in the first area image, the fourth state identification result is that the escalator is in an empty state. a fourth outcome determination sub-module configured to determine.

In a possible implementation, the device is a stop signal transmission module configured to transmit an escalator stop signal when the escalator is in an empty state, wherein the escalator stop signal causes the escalator to stop running. It further comprises a stop signal transmission module for instructing the escalator.

In a possible implementation, the device is an activation signal transmission module configured to transmit an escalator activation signal when the escalator is in a non-empty state and the escalator has already stopped running, The escalator activation signal further comprises an activation signal transmission module for instructing the escalator to run.

In a possible implementation, the apparatus is a target detection module configured to perform a second target detection on the first image to determine a third region of the second target in the first image; A detection result determination module configured to determine a detection result of the second target based on the positional relationship between the first area and the third area, wherein the detection result determines whether the second target is the a detection result determination module including being located on an escalator or not located on said escalator.

In a possible implementation, the detection result determination module determines that the detection result indicates that the second target is located on the escalator when the area ratio between the fourth area and the third area is greater than or equal to a third threshold. A determining sub-module configured to determine that the fourth area comprises an intersection area of the first area and the third area.

In a possible implementation, said second target comprises prohibited items on an escalator, and said device is configured to send warning information when said second target is located on said escalator. It further comprises a transmission module.

In some embodiments, the functions and modules in the apparatus provided in the examples of the present application are used to perform the methods described in the above method examples, and specific implementations are described in the above method examples. See description.

Embodiments of the present application further provide a computer-readable storage medium. The computer readable storage medium stores computer program instructions which, when executed by a processor, implement the above method. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

Embodiments of the present application further provide an electronic device. The electronic device comprises a processor and a memory configured to store instructions executable by the processor, wherein the processor invokes the instructions stored in the memory to perform the method. Configured.

Embodiments of the present application further provide a computer program product. The computer program product comprises computer readable code, and when the computer readable code is executed in a device, a processor in the device executes instructions for implementing the image detection method according to any one of the embodiments above.

Embodiments of the present application further provide another computer program product. The computer program product is configured to store computer readable instructions which, when executed, cause a computer to perform the operations of the image detection method according to any one of the embodiments above.

An electronic device may be provided as a terminal, server, or other form of device.

FIG. 7 shows a block diagram of an electronic device 800 according to an embodiment of the present application. For example, electronic device 800 may be a terminal such as a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical equipment, fitness equipment, personal digital assistant, and the like.

Referring to FIG. 7, the electronic device 800 includes a processing unit 802, a memory 804, a power supply unit 806, a multimedia unit 808, an audio unit 810, an Input/Output (I/O) interface 812, a sensor unit 814 and One or more of the communication units 816 may be provided.

Processing unit 802 generally controls the overall operation of electronic device 800 . For example, it controls operations related to display, phone calls, data communication, camera operation and recording operation. Processing unit 802 may include one or more processors 820 for executing instructions. All or part of the steps of the above method are thereby performed. Note that the processing unit 802 may comprise one or more modules for interaction with other units. For example, processing unit 802 may include a multimedia module to facilitate interaction between multimedia unit 808 and processing unit 802 .

Memory 804 is configured to support operations in electronic device 800 by storing various data. Examples of such data include instructions for any application or method operable on electronic device 800, contact data, phonebook data, messages, images, videos, and the like. Memory 804 may be implemented by any type of volatile or non-volatile storage, or a combination thereof. For example, static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), electrically erasable programmable read-only memory (EPROM) Programmable Read-Only Memory), Programmable Read-Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power supply unit 806 provides power to the various units of electronic device 800 . Power supply unit 806 may comprise a power management system, one or more power supplies, and other units related to power generation, management and distribution for electronic device 800 .

A multimedia unit 808 includes a screen for providing an output interface between the electronic device 800 and a user. In some embodiments, the screen includes a Liquid Crystal Display (LCD) and a Touch Panel (TP). When the screen includes a touch panel, it is implemented as a touch panel and receives input signals from the user. A touch panel comprises one or more touch sensors that sense touches, slides and gestures on the panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, multimedia unit 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operating mode such as a shooting mode or a video mode, the front camera and/or the rear camera can receive multimedia data from the outside. Each front and rear camera may have a fixed optical lens system or focus and optical zoom capabilities.

Audio unit 810 is configured to output/input audio signals. For example, the audio unit 810 includes a microphone (MIC). When the electronic device 800 is in operating modes such as call mode, recording mode and voice recognition mode, the microphone is configured to receive audio signals from the outside. The received audio signals can be further stored in memory 804 or transmitted via communication unit 816 . In some embodiments, audio unit 810 further comprises a speaker configured to output audio signals.

I/O interface 812 provides an interface between processing unit 802 and peripheral interface modules. The peripheral interface modules may be keyboards, click wheels, buttons, and the like. These buttons include, but are not limited to, home button, volume button, start button and lock button.

Sensor unit 814 comprises one or more sensors and is configured to perform various condition assessments for electronic device 800 . For example, the sensor unit 814 can detect the on/off state of the electronic device 800 and the relative positioning of the units. For example, the unit is the display and keypad of electronic device 800 . The sensor unit 814 detects changes in the position of the electronic device 800 or one unit in the electronic device 800, whether there is contact between the user and the electronic device 800, the orientation or acceleration/deceleration of the electronic device 800, and changes in the temperature of the electronic device 800. You can also Sensor unit 814 may comprise a proximity sensor and is configured to detect the presence of surrounding objects in the absence of any physical contact. The sensor unit 814 may comprise an optical sensor, such as a Complementary Metal-Oxide-Semiconductor (CMOS) or Charge Coupled Device (CCD) image sensor, and may be applied to imaging. configured to In some embodiments, the sensor unit 814 may comprise an acceleration sensor, gyro sensor, magnetic sensor, pressure sensor or temperature sensor.

Communication unit 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. The electronic device 800 complies with a communication standard such as a wireless network (WiFi Wireless Fidelity), a second generation mobile communication technology (2G: The 2nd Generation), a third generation mobile communication technology (3G: The 3rd Generation), or a combination thereof. based wireless network. In one exemplary embodiment, communication unit 816 receives broadcast signals or broadcast-related information from an external broadcast channel management system via a broadcast channel. In one exemplary embodiment, the communication unit 816 further comprises a Near Field Communication (NFC) module to facilitate near field communication. For example, the NFC module uses Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) Tooth) technology and other technologies.

In an exemplary embodiment, electronic device 800 includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processors (DSPDs). Signal Processing Device), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor or other electronic device to perform the above method. It may be configured as

Exemplary embodiments further provide a non-transitory computer-readable storage medium, such as memory 804, having computer program instructions stored thereon. The computer program instructions are executed by processor 820 of electronic device 800 to perform the method described above.

FIG. 8 is a block diagram illustrating another electronic device 1900 according to an embodiment of the application. For example, electronic device 1900 may be provided as a server. Referring to FIG. 8, electronic device 1900 comprises a processing unit 1922 . It further comprises one or more processors and memory resources represented by memory 1932 . The memory lease is for storing instructions to be executed by processing unit 1922, such as an application program. An application program stored in memory 1932 may include one or more modules each corresponding to a set of instructions. It should be noted that the processing unit 1922 is configured to execute instructions to perform the methods described above.

The electronic device 1900 includes a power supply unit 1926 configured to perform power management of the electronic device 1900; a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network; O) An interface 1958 may also be provided. The electronic device 1900 may run a Microsoft server operating system (Windows Server™), a graphics-based user interface operating system (Mac OS X™) provided by Apple Inc., a multi-user, multi-process computer operating system (Unix™). )), a free open source code-type Unix-like operating system (Uinux), an open source code-type Unix-like operating system (FreeBSD™), or the like, stored in memory 1932 can run any operating system.

Exemplary embodiments further provide a non-volatile computer-readable storage medium, such as memory 1932, which contains computer program instructions. The computer program instructions are executed by processing unit 1922 of electronic device 1900 to perform the method described above.

The present application may be a system, method and/or computer program product. The computer program product may comprise a computer readable storage medium having computer readable program instructions stored thereon for causing a processor to implement aspects of the present application.

A computer-readable storage medium may be a tangible device capable of holding or storing instructions for use in an instruction-executing device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination of the above. Computer-readable storage media include portable computer disks, hard disks, random access memory (RAM), read-only memory, erasable programmable read-only memory (EPROM or flash), static random access memory (SRAM), portable compact Disc read-only memory (CD-ROM: Compact Disc Read-Only Memory), digital multi-purpose disc (DVD: Digital Video Disc), memory stick, flexible disc, punch card in which instructions are stored, or protrusion structure in recessed groove and any suitable combination of the above. Computer-readable storage media, as used herein, include radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or through electrical wires. It should not be construed as being a transitory signal per se, such as a transmitted electrical signal.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing/processing device or network such as the Internet, local area networks, wide area networks and/or wireless networks. can be downloaded to an external computer or external storage device via A network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. A network interface card or network interface at each computing/processing device receives computer-readable program instructions from the network, transfers the computer-readable program instructions for storage on a computer-readable storage medium at each computing/processing device.

Computer readable program instructions for performing the operations herein may be assembler instructions, Industry Standard Architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or one or more It may be source code or target code written in multiple programming languages. The programming languages include object-oriented programming languages such as Smalltalk, C++, etc., and traditional procedural programming languages such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user computer, partially executed on the user computer, executed as a separate software package, or partially executed on the user computer. It may be executed locally and partially executed on a remote computer, or completely executed on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected externally. computer (eg, through the Internet using an Internet service provider). In some embodiments, state information in computer readable program instructions is used to customize electronic circuits such as programmable logic circuits, field programmable gate arrays or programmable logic arrays. The electronic circuitry may implement aspects of the present application by executing computer readable program instructions.

Aspects of the present application are now described with reference to flowchart illustrations and/or block diagrams of example methods, apparatus (systems) and computer program products of the present application. Each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing device, thereby producing a device which, when executed by the processor of the computer or other programmable data processing device, can be used as a flowchart. and/or produce an apparatus that performs the functions/operations specified in one or more of the blocks in the block diagrams. These computer readable program instructions may be stored on a computer readable storage medium. These instructions cause computers, programmable data processing devices, and/or other devices to operate in specific manners. Accordingly, a computer-readable storage medium having instructions stored thereon comprises an article of manufacture containing instructions for each aspect of implementing the functions/operations specified in one or more blocks in the flowcharts and/or block diagrams.

The computer readable program instructions may be loaded into a computer, other programmable data processing device, or other device. It causes a computer, other programmable data processing device, or other device to perform a series of operational steps to produce a computer-implemented process. Accordingly, instructions executed by a computer, other programmable data processing device, or other device, implement the functions/operations specified in one or more blocks in the flowchart illustrations and/or block diagrams.

The flowcharts and workbook diagrams in the drawings illustrate possible architectures, functionality, and operation of systems, methods and computer program products according to embodiments of the present application. In this regard, each block in a flowchart or block diagram can represent part of a module, program segment or instruction. Some of the modules, program segments or instructions contain executable instructions for implementing one or more predetermined logical functions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed essentially in parallel, or possibly in the opposite order, as determined from the functionality involved. Each block in the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, may be implemented by means of dedicated hardware-based systems, or dedicated hardware and computer instructions, to perform the specified functions or operations. It can be realized by a combination of

The computer program product may be implemented in hardware, software or a combination thereof. In one alternative embodiment, said computer program product may be embodied as a computer storage medium, and in another alternative embodiment, the computer program product is, for example, a Software Development Kit: SDK) or the like may be embodied as a software product.

While embodiments of the present invention have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will readily occur to those skilled in the art without departing from the scope and spirit of each described embodiment. The choice of terminology used herein is such that it best interprets the principles, practical applications, or improvements to the technology in the marketplace of each embodiment, or that others of ordinary skill in the art may recognize each implementation disclosed herein. The purpose is to help you understand the examples.

In an embodiment of the present application, a first image of an escalator is obtained, region detection is performed on the first image, a first region of the escalator in the first image is determined, and a second region corresponding to the first region is determined. performing state identification of an escalator on a region image to determine at least one state identification result of the escalator, wherein the state identification result includes whether the escalator is in an empty state or in a non-empty state; A state of the escalator is determined based on the at least one state identification result. In this way, the accuracy rate of escalator operation status identification can be improved, and a complete and stable set of smart escalator solutions can be formed, which can be applied to all current public escalator scenes. can.

According to embodiments of the present application, detecting a location area of an escalator in an image, identifying at least one status identification result based on the area image of the escalator, and determining an empty status of the escalator based on the at least one status identification result. Or the non-empty status can be determined, the accuracy of locating the area of the escalator can be improved, and the accuracy rate of identifying the running status of the escalator can be improved.
For example, the present application provides the following items.
(Item 1)
An image detection method comprising:
obtaining a first image of the escalator;
performing region detection on the first image to determine a first region of the escalator in the first image;
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator, wherein the state identification result indicates that the escalator is in an empty state; including being or being in a non-idle state;
determining a state of the escalator based on the at least one state identification result.
(Item 2)
Determining the state of the escalator based on the at least one state identification result comprises:
determining a state discrimination value of the escalator based on a plurality of state identification results and weights of the plurality of state identification results when there are a plurality of state identification results;
determining that the escalator is in an empty state when the state discrimination value is greater than or equal to a first threshold.
The method of item 1.
(Item 3)
The state identification result includes a first state identification result,
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator;
performing a classification process on the first area image to obtain a first state identification result of the escalator.
3. The method of item 1 or 2.
(Item 4)
the state identification result includes a second state identification result;
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator;
dividing the first area image into a background area and a foreground area where the escalator is located;
adjusting the pixel values of the background area to obtain an adjusted second area image;
performing a classification process on the second area image to obtain a second state identification result of the escalator.
The method according to any one of items 1-3.
(Item 5)
The state identification result includes a third state identification result,
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator;
performing pixel matching on the first area image and a predetermined reference image to determine a proportion of a matching area between the first area image and the reference image, wherein the reference image is in an empty state; including an area image corresponding to the escalator;
determining that the third state identification result is that the escalator is in an empty state if the ratio of the matching area is greater than or equal to a second threshold.
The method according to any one of items 1-4.
(Item 6)
The state identification result includes a fourth state identification result,
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator;
performing a first target detection on the first area image to determine if a first target is present in the first area image;
determining that the fourth state identification result is that the escalator is in an empty state if the first target does not exist in the first area image.
The method according to any one of items 1-5.
(Item 7)
The method includes:
and transmitting an escalator stop signal when the escalator is in an empty state, wherein the escalator stop signal is for instructing the escalator to stop running. do
The method according to any one of items 1-6.
(Item 8)
The method includes:
sending an escalator start signal when the escalator is in a non-empty state and the escalator has been stopped running, the escalator start signal is for instructing the escalator to run; characterized by further comprising
The method of any one of items 1-7.
(Item 9)
The method includes:
performing second target detection on the first image to determine a third region of the second target in the first image;
determining a detection result of the second target based on the positional relationship between the first area and the third area, wherein the detection result is that the second target is positioned on the escalator; or and not on the escalator.
The method of any one of items 1-8.
(Item 10)
Determining the detection result of the second target based on the positional relationship between the first area and the third area includes:
determining that the detection result is that the second target is positioned on the escalator if an area ratio between the fourth area and the third area is greater than or equal to a third threshold; The area includes a common area of the first area and the third area.
The method of item 9.
(Item 11)
The second goal includes prohibited items on escalators, and the method includes:
Further comprising transmitting warning information when the second target is located on the escalator.
A method according to item 9 or 10.
(Item 12)
An image detection device,
an image acquisition module configured to acquire a first image of an escalator;
a region detection module configured to perform region detection on the first image to determine a first region of the escalator in the first image;
a state identification module configured to perform state identification of an escalator on a first region image corresponding to the first region and determine at least one state identification result of the escalator, wherein the state identification result is , the escalator being in an empty state or in a non-empty state; and
a state determination module configured to determine a state of the escalator based on the at least one state identification result.
(Item 13)
The state determination module includes:
a discriminant value determination sub-module configured to determine a state discriminant value of the escalator based on a plurality of state identification results and weights of the plurality of state identification results when there are a plurality of state identification results;
a state determination sub-module configured to determine that the escalator is in an empty state when the state discrimination value is greater than or equal to a first threshold.
13. Apparatus according to item 12.
(Item 14)
The state identification results include a first state identification result, and the state identification module comprises:
A first result determination sub-module configured to perform a classification process on the first region image to obtain a first state identification result of the escalator.
14. Apparatus according to item 12 or 13.
(Item 15)
The state identification results include a second state identification result, and the state identification module comprises:
a segmentation sub-module configured to segment the first area image to divide the first area image into a background area and a foreground area where the escalator is located;
a pixel adjustment sub-module configured to adjust pixel values of the background area to obtain an adjusted second area image;
a second result determination sub-module configured to perform a classification process on the second area image to obtain a second state identification result of the escalator.
Apparatus according to any one of items 12-14.
(Item 16)
The state identification results include a third state identification result, and the state identification module comprises:
a pixel matching sub-module configured to perform pixel matching on the first area image and a predetermined reference image to determine a ratio of matching areas between the first area image and the reference image; a pixel matching sub-module, wherein the reference image includes an area image corresponding to the escalator in an empty state;
a third result determination sub-module configured to determine that the third state identification result is that the escalator is in an empty state if the proportion of the matching area is greater than or equal to a second threshold. characterized by
Apparatus according to any one of items 12-15.
(Item 17)
The state identification results include a fourth state identification result, and the state identification module comprises:
a detection sub-module configured to perform a first target detection on the first area image and determine whether a first target is present in the first area image;
a fourth result determination sub-module configured to determine that the fourth state identification result is that the escalator is in an empty state if no first target is present in the first area image. characterized by
Apparatus according to any one of items 12-16.
(Item 18)
The device comprises:
a stop signal transmission module configured to transmit an escalator stop signal when the escalator is in an empty state, wherein the escalator stop signal is for instructing the escalator to stop running; characterized by further comprising a stop signal transmission module
Apparatus according to any one of items 12-17.
(Item 19)
The apparatus has an activation signal transmission module configured to transmit an escalator activation signal when the escalator is in a non-empty state and the escalator has already stopped running, the escalator activation signal comprising: further comprising an activation signal transmission module for instructing the escalator to run
Apparatus according to any one of items 12-18.
(Item 20)
The device comprises:
a target detection module configured to perform a second target detection on said first image to determine a third region of a second target in said first image;
A detection result determination module configured to determine a detection result of the second target based on the positional relationship between the first area and the third area, wherein the detection result is obtained by determining whether the second target is a detection result determination module including located on the escalator or not located on the escalator.
20. Apparatus according to any one of items 12-19.
(Item 21)
The detection result determination module is
A determination sub-module configured to determine that the detection result is that the second target is located on the escalator if the area ratio of the fourth area and the third area is greater than or equal to a third threshold. wherein the fourth area comprises a decision sub-module including a common area of the first area and the third area.
21. Apparatus according to item 20.
(Item 22)
The second goal includes prohibited items on an escalator, and the device:
further comprising a warning information transmission module configured to transmit warning information when the second target is located on the escalator;
22. Apparatus according to item 20 or 21.
(Item 23)
an electronic device,
a processor;
a memory configured to store instructions executable by the processor;
An electronic device, wherein the processor is configured to invoke instructions stored in the memory to perform the method of any one of items 1-11.
(Item 24)
A method according to any one of items 1 to 11, wherein computer program instructions are stored on said computer readable storage medium, and when said computer program instructions are executed by a processor. A computer-readable storage medium for realizing
(Item 25)
12. A computer program, said computer program comprising computer readable code, and when said computer readable code is executed in said electronic device, a processor in said electronic device performs the method according to any one of items 1-11. A computer program that runs a

Claims (25)

An image detection method comprising:
obtaining a first image of the escalator;
performing region detection on the first image to determine a first region of the escalator in the first image;
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator, wherein the state identification result indicates that the escalator is in an empty state; including being or being in a non-idle state;
determining a state of the escalator based on the at least one state identification result. Determining the state of the escalator based on the at least one state identification result comprises:
determining a state discrimination value of the escalator based on a plurality of state identification results and weights of the plurality of state identification results when there are a plurality of state identification results;
2. The method of claim 1, comprising determining that the escalator is in an empty state if the state discrimination value is greater than or equal to a first threshold. The state identification result includes a first state identification result,
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator;
3. The method of claim 1 or 2, comprising performing a classification process on the first area image to obtain a first state identification result of the escalator. the state identification result includes a second state identification result;
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator;
dividing the first area image into a background area and a foreground area where the escalator is located;
adjusting the pixel values of the background area to obtain an adjusted second area image;
performing a classification process on the second area image to obtain a second state identification result of the escalator. The state identification result includes a third state identification result,
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator;
performing pixel matching on the first area image and a predetermined reference image to determine a proportion of a matching area between the first area image and the reference image, wherein the reference image is in an empty state; including an area image corresponding to the escalator;
determining that the third state identification result is that the escalator is in an empty state if the proportion of the matching area is greater than or equal to a second threshold. The method according to any one of the above. The state identification result includes a fourth state identification result,
performing state identification of an escalator on a first region image corresponding to the first region and determining at least one state identification result of the escalator;
performing a first target detection on the first area image to determine if a first target is present in the first area image;
and determining that the fourth state identification result is that the escalator is in an empty state if no first target is present in the first area image. The method according to any one of the above. The method includes:
and transmitting an escalator stop signal when the escalator is in an empty state, wherein the escalator stop signal is for instructing the escalator to stop running. A method according to any one of claims 1-6. The method includes:
sending an escalator start signal when the escalator is in a non-empty state and the escalator has been stopped running, the escalator start signal is for instructing the escalator to run; A method according to any one of claims 1 to 7, further comprising: The method includes:
performing second target detection on the first image to determine a third region of the second target in the first image;
determining a detection result of the second target based on the positional relationship between the first area and the third area, wherein the detection result is that the second target is positioned on the escalator; or The method of any one of claims 1-8, further comprising: not located on the escalator. Determining the detection result of the second target based on the positional relationship between the first area and the third area includes:
determining that the detection result is that the second target is positioned on the escalator if an area ratio between the fourth area and the third area is greater than or equal to a third threshold; 10. The method of claim 9, wherein the area comprises a common area of the first area and the third area. The second goal includes prohibited items on escalators, and the method includes:
11. A method according to claim 9 or 10, further comprising transmitting warning information if the second target is located on the escalator. An image detection device,
an image acquisition module configured to acquire a first image of an escalator;
a region detection module configured to perform region detection on the first image to determine a first region of the escalator in the first image;
a state identification module configured to perform state identification of an escalator on a first region image corresponding to the first region and determine at least one state identification result of the escalator, wherein the state identification result is , the escalator being in an empty state or in a non-empty state; and
a state determination module configured to determine a state of the escalator based on the at least one state identification result. The state determination module includes:
a discriminant value determination sub-module configured to determine a state discriminant value of the escalator based on a plurality of state identification results and weights of the plurality of state identification results when there are a plurality of state identification results;
13. The apparatus of claim 12, comprising a state determination sub-module configured to determine that the escalator is in an empty state if the state discrimination value is greater than or equal to a first threshold. The state identification results include a first state identification result, and the state identification module comprises:
14. Apparatus according to claim 12 or 13, comprising a first result determination sub-module configured to perform a classification process on the first area image to obtain a first state identification result of the escalator. . The state identification results include a second state identification result, and the state identification module comprises:
a segmentation sub-module configured to segment the first area image to divide the first area image into a background area and a foreground area where the escalator is located;
a pixel adjustment sub-module configured to adjust pixel values of the background area to obtain an adjusted second area image;
a second result determination sub-module configured to perform a classification process on the second area image to obtain a second state identification result of the escalator. A device according to any one of the preceding clauses. The state identification results include a third state identification result, and the state identification module comprises:
a pixel matching sub-module configured to perform pixel matching on the first area image and a predetermined reference image to determine a ratio of matching areas between the first area image and the reference image; a pixel matching sub-module, wherein the reference image includes an area image corresponding to the escalator in an empty state;
a third result determination sub-module configured to determine that the third state identification result is that the escalator is in an empty state if the proportion of the matching area is greater than or equal to a second threshold. A device according to any one of claims 12 to 15, characterized in that: The state identification results include a fourth state identification result, and the state identification module comprises:
a detection sub-module configured to perform a first target detection on the first area image and determine whether a first target is present in the first area image;
a fourth result determination sub-module configured to determine that the fourth state identification result is that the escalator is in an empty state if no first target is present in the first area image. A device according to any one of claims 12 to 16, characterized in that: The device comprises:
a stop signal transmission module configured to transmit an escalator stop signal when the escalator is in an empty state, wherein the escalator stop signal is for instructing the escalator to stop running; Device according to any one of claims 12 to 17, characterized in that it further comprises a stop signal transmission module. The apparatus has an activation signal transmission module configured to transmit an escalator activation signal when the escalator is in a non-empty state and the escalator has already stopped running, the escalator activation signal comprising: Apparatus according to any one of claims 12 to 18, further comprising an activation signal transmission module for instructing the escalator to run. The device comprises:
a target detection module configured to perform a second target detection on said first image to determine a third region of a second target in said first image;
A detection result determination module configured to determine a detection result of the second target based on the positional relationship between the first area and the third area, wherein the detection result is obtained by determining whether the second target is 20. The apparatus according to any one of claims 12 to 19, further comprising a detection result determination module, including located on the escalator or not located on the escalator. The detection result determination module is
A determination sub-module configured to determine that the detection result is that the second target is located on the escalator if the area ratio of the fourth area and the third area is greater than or equal to a third threshold. 21. The apparatus of claim 20, wherein said fourth area comprises a determining sub-module including an intersection area of said first area and said third area. The second goal includes prohibited items on escalators, and the device:
22. Apparatus according to claim 20 or 21, further comprising a warning information sending module configured to send warning information when said second target is located on said escalator. an electronic device,
a processor;
a memory configured to store instructions executable by the processor;
An electronic device, wherein the processor is configured to invoke instructions stored in the memory to perform the method of any one of claims 1-11. A computer readable storage medium having computer program instructions stored thereon, when the computer program instructions are executed by a processor, according to any one of claims 1 to 11. A computer-readable storage medium that implements a method. A computer program product, said computer program product comprising computer readable code, and when said computer readable code is executed in said electronic device, a processor in said electronic device executes the program according to any one of claims 1 to 11. A computer program that implements a method.

Images