CN114037949A - Object detection method - Google Patents

Abstract

The present specification provides an object detection method applied to a robot, including: acquiring a current environment image of a target environment according to a first shooting module of the robot; determining position information of an initial detection object under the condition that the current environment image is determined to contain the initial detection object; and acquiring a current area image of the position information according to a second shooting module of the robot, and determining whether a target detection object exists in the target environment currently according to the current area image.

Description

Object detection method

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an object detection method.

Background

With the rapid development of internet technology and artificial intelligence technology, many organizations will perform automated operation and maintenance work in data centers for storing private data through robots in order to avoid disclosure of private data such as user information, privacy technology, business secrets, and the like. However, in practical applications, when the robot performs specific actions such as opening and closing a door in the operation and maintenance process, the robot cannot detect whether security risk personnel exist in the surrounding environment, so that the personnel with security risk may enter the data center following the robot, and privacy data may be leaked.

Disclosure of Invention

In view of this, the embodiments of the present specification provide an object detection method. The present specification also relates to a method for detecting a risk object, a robot, an object detection apparatus, a risk object detection apparatus, a computing device, a computer-readable storage medium, and a computer program, so as to solve the technical drawbacks of the prior art.

According to a first aspect of embodiments herein, there is provided an object detection method applied to a robot, including:

acquiring a current environment image of a target environment according to a first shooting module of the robot;

determining position information of an initial detection object under the condition that the current environment image is determined to contain the initial detection object;

and acquiring a current area image of the position information according to a second shooting module of the robot, and determining whether a target detection object exists in the target environment currently according to the current area image.

According to a second aspect of the embodiments of the present specification, there is provided a risk object detection method applied to an operation and maintenance robot in a data center, including:

acquiring a current environment image of the data center according to a first camera of the operation and maintenance robot;

determining position information of a potential risk object in the case that the current environment image is determined to contain the potential risk object;

acquiring a current area image of the position information according to a second camera of the operation and maintenance robot;

in the event that a target risk object is currently present in the data center as determined from the current region image, generating and issuing an object risk notification based on the target risk object.

According to a third aspect of embodiments herein, there is provided a robot including a robot body, a robot arm mounted on the robot body, a first photographing module, and a second photographing module, wherein,

the second shooting module is installed on the robot body, and the second shooting module is installed on the mechanical arm; the robot implements the steps of any of the object detection method and the risk object detection method when performing object detection.

According to a fourth aspect of embodiments herein, there is provided an object detection apparatus applied to a robot, including:

a first acquisition module configured to acquire a current environment image of a target environment according to a first photographing module of the robot;

a first determination module configured to determine position information of an initial detection object in a case where it is determined that the initial detection object is included in the current environment image;

and the second determination module is configured to acquire a current area image of the position information according to a second shooting module of the robot, and determine whether a target detection object currently exists in the target environment according to the current area image.

According to a fifth aspect of the embodiments of the present specification, there is provided a risk object detection apparatus applied to an operation and maintenance robot of a data center, including:

the first acquisition module is configured to acquire a current environment image of the data center according to a first camera of the operation and maintenance robot;

a first determination module configured to determine location information of a risk potential object if it is determined that the risk potential object is included in the current environment image;

the second acquisition module is configured to acquire a current area image of the position information according to a second camera of the operation and maintenance robot;

a second determination module configured to generate and issue an object risk notification based on a target risk object in the data center if it is determined from the current region image that the target risk object is currently present.

According to a sixth aspect of embodiments herein, there is provided a computing device comprising:

a memory and a processor;

the memory is for storing computer-executable instructions and the processor is for executing the computer-executable instructions which, when executed by the processor, implement the steps of any of the object detection methods and the risk object detection method.

According to a seventh aspect of embodiments herein, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement any of the object detection methods and the steps of the risk object detection method.

According to an eighth aspect of embodiments herein, there is provided a computer program, wherein the computer program, when executed in a computer, causes the computer to perform the steps of any of the object detection methods and the risk object detection method.

The object detection method provided by the specification is applied to a robot and comprises the steps of obtaining a current environment image of a target environment according to a first shooting module of the robot; determining position information of an initial detection object under the condition that the current environment image is determined to contain the initial detection object; and acquiring a current area image of the position information according to a second shooting module of the robot, and determining whether a target detection object exists in the target environment currently according to the current area image.

Specifically, according to the object detection method, under the condition that the robot determines that the current environment image acquired by the first shooting module contains the initial detection object, the position information of the initial detection object is determined, the current area image of the position information is acquired from the target environment through the second shooting module, and whether the target detection object exists in the target environment is further determined according to the current area image, so that whether the target detection object exists in the target environment is rapidly and accurately detected in real time, and the problem of data leakage caused by the target detection object is further avoided.

Drawings

Fig. 1 is a flowchart of an object detection method provided in an embodiment of the present specification;

FIG. 2 is a flowchart illustrating a method for detecting an object according to an embodiment of the present disclosure, wherein the method determines a suspicious object based on a current environment image;

fig. 3 is a flowchart illustrating a method for detecting a suspicious pedestrian based on a current region image according to an embodiment of the present disclosure;

fig. 4 is a processing flow chart of an object detection method applied in a suspicious target scene in a robot detection surrounding environment according to an embodiment of the present specification;

FIG. 5 is a flowchart of a risk object detection method provided in an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of an object detection apparatus according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of a risk object detection apparatus according to an embodiment of the present disclosure;

fig. 8 is a block diagram of a computing device according to an embodiment of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.

The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

First, the noun terms to which one or more embodiments of the present specification relate are explained.

The anti-following function: when the robot enters and exits a room, people are prevented from following behind the robot or entering the room from the front through gaps, and if the situation occurs, the robot needs to report early warning in time, so that loss is avoided.

Pedestrian detection problem: the main purpose of the visual inspection problem is to determine the number, location and corresponding posture of pedestrians in a camera.

Dynamic target detection algorithm: a visual detection algorithm is mainly applied to the field of video image processing, and the main realization method is to find a changed area in a picture by comparing historical data so as to determine the area of a dynamic target.

And (3) target detection algorithm: compared with deep learning, the visual detection algorithms are mainly used for manually extracting visual features in pictures by researchers, establishing corresponding statistical distribution and obtaining detection results according to different classification methods.

UVC: a protocol standard defined for USB video capture devices.

KNN model: a K nearest neighbor (KNN, K-nearest neighbor) classification algorithm.

R-CNN: Region-CNN, is a detection algorithm that applies deep learning to target detection.

Yolo: an object recognition and location algorithm based on a deep neural network.

ImageNet dataset: a data set for training a visual object recognition algorithm.

The data center comprises: is a globally collaborative network of specific devices used to deliver, accelerate, present, compute, store data information over the internet network infrastructure.

With the rapid development of internet technology and artificial intelligence technology, many organizations will perform automated operation and maintenance work in data centers for storing private data through robots in order to avoid disclosure of private data such as user information, privacy technology, business secrets, and the like. Or, in order to ensure that important production equipment or innovative production equipment is not damaged and leaked by illegal personnel, the robot is also adopted to perform automatic operation and maintenance work in a production workshop.

And as the main function of the robot is to execute automatic operation and maintenance work in a modern data center or a production workshop, the whole process of the operation and maintenance operation is completely automatic, wherein the whole process comprises the action of entering a production booth (machine room) of the production workshop or the data center.

The manner of entering the production plant or the machine room provided in the present specification may include:

(1) the robot sends a door opening request to a control system of a production workshop or a machine room, the control system automatically opens a door of the production workshop or the machine room through a door machine, the robot enters the production workshop or the machine room through the door, the robot sends a door closing request to the control system, and the control system automatically closes the door through the door machine.

(2) And triggering the control system to verify the action before the robot runs to the door of the production workshop or the machine room, opening the door of the production workshop or the machine room when the robot is determined to enter the production workshop or the machine room through the verification action, and closing the door when the robot enters the production workshop or the machine room.

Based on this, when the robot enters a production workshop or a machine room, a person with safety risk may enter the production workshop or the machine room following the robot, so as to cause certain potential safety hazard; therefore, the embodiments of the present specification provide two solutions to the above problems. The first scheme is as follows: the robot detects pedestrians in the camera view field by using a deep learning method, and many classical neural network models such as Yolo and R-CNN can be used for article detection in the field of deep learning. Based on the ImageNet dataset, these networks can provide relatively high pedestrian detection accuracy, but this approach has two major problems;

1. the above networks are proposed to solve the general visual target detection problem, and a large amount of data is required to be retrained to ensure the performance. The scene specificity of the internal environment of the data center is relatively strong, only operation and maintenance personnel, cabinets, servers and other related objects are targets visible in the visual field of the camera, and the objects are limited by the confidentiality property of the data center and are difficult to collect in large quantities, so that a deep learning model with high precision is difficult to obtain for the internal environment of the data center;

2. the update frequency of the current camera is about 30Hz, the deduction of the model needs to be accelerated by a GPU with stronger calculation force to achieve a real-time effect, and the cost of the robot is increased. When the mechanism is used for reducing the cost, if a GPU (graphic processing unit) on a robot manual control machine is cancelled, the speed of performing deep learning calculation by only using a CPU (central processing unit) is difficult to achieve real time, and the performance of a detection result is seriously influenced.

Scheme II: visual features such as SIFT, HOG, etc. may also be used for pedestrian detection. The calculation and extraction of the visual features do not need to use a neural network, and a relatively mature machine learning algorithm is used for acceleration to realize real-time detection on a CPU (central processing unit), but the scheme has relatively high requirements on illumination conditions and picture quality, and the algorithm has relatively high false alarm rate for some pictures with relatively large distortion, such as pictures of a panoramic camera or a panoramic camera, and is unacceptable for products.

Based on this, the object detection method provided by the present specification is applied to a robot for realizing automated operation and maintenance, and the object detection method identifies suspicious people (people with safety risks) by using a plurality of cameras configured on the robot, combines a plurality of visual detection algorithms, corrects the relative positions of the plurality of cameras, completes real-time detection according to a final correction result, and realizes that the robot monitors the surrounding environment all the time from the time of opening a door to the time of completely closing the door, thereby preventing people with safety risks from entering a production package in the process of robot entering and exiting, and avoiding the occurrence of potential safety hazards.

Specifically, in the present specification, an object detection method is provided, and the present specification relates to a risk object detection method, a robot, an object detection apparatus, a risk object detection apparatus, a computing device, a computer-readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

Referring to fig. 1, fig. 1 shows a flowchart of an object detection method provided in an embodiment of the present specification, where the object detection method is applied to a robot, and specifically includes the following steps:

step 102: and acquiring a current environment image of a target environment according to the first shooting module of the robot.

In practical application, the object detection method provided by the specification is applied to a robot for realizing automatic operation and maintenance, and can be applied to different scenes according to different application scenes of the robot; for example, when the robot is applied to a data center, the object detection method can detect the target detection object in real time, so that the data leakage caused by illegal implementation of the target detection object in the data center is avoided. When the robot is applied to an automatic production workshop, the object detection method can detect the target detection object in real time, so that the target detection object is prevented from illegally entering the automatic production workshop to damage equipment in the production workshop or prevent innovative production equipment in the production workshop from being revealed. In practical application, the robot comprises a robot body and a mechanical arm installed on the robot body.

Wherein the target detection object can be understood as suspicious personnel capable of causing data leakage or destroying equipment in the target environment; or suspicious devices, such as drones, mobile candid devices, etc., that can cause data leakage or damage to devices within the target environment. Correspondingly, the target environment may be understood as the environment in which the robot works, including but not limited to corridors, the interior of the machine room, and machine room entrances, etc.

The first shooting module can be understood as a first camera configured on the robot, for example, the first shooting module can be a panoramic camera or a panoramic camera. In practical application, in order to reduce the processing pressure of the CPU of the robot, the first shooting module may employ a camera with a lower frame rate, so that the robot can perform fast 360-degree panoramic shooting on a target environment in real time through the camera, and an image shot by the camera can be processed quickly by the CPU of the robot without occupying too much CPU computing resources. Correspondingly, the current environment image may be understood as a current environment image of the target environment acquired by the first photographing module, for example, an environment image of a current corridor environment photographed by a panoramic camera configured on the robot.

For example, in a scenario where the object detection method is applied to a robot that performs automatic maintenance on a data center, further description is made on obtaining a current environment image of a target environment according to a first shooting module.

Under the condition that the target environment is the machine room entrance environment, the robot can shoot the environment near a door (machine room entrance) of the machine room through the panoramic camera arranged on the robot in the door opening and closing process by executing the door opening and closing operation, so that the current environment image of the machine room entrance is obtained, and whether suspicious personnel enter the machine room along with the robot is determined in the door opening and closing process of the robot based on the current environment image of the machine room entrance.

Under the condition that the target environment is the environment inside the machine room, the robot can shoot the environment inside the machine room through the panoramic camera arranged in the robot in the process of executing automatic maintenance operation inside the machine room, so that the current environment image inside the machine room is obtained, and whether suspicious personnel are in the machine room or not can be determined based on the current environment image inside the machine room subsequently.

Under the condition that the target environment is the unmanned channel environment, in the process that the robot executes automatic maintenance operation in the unmanned channel, the environment of the unmanned channel is shot through a panoramic camera arranged on the robot, and therefore the current environment image of the unmanned channel is obtained. Therefore, whether suspicious people exist in the unmanned passage can be determined subsequently based on the current environment image of the unmanned passage.

Step 104: in a case where it is determined that an initial detection object is included in the current environment image, position information of the initial detection object is determined.

Wherein, the initial detection object can be understood as a potential suspicious person or a potential suspicious device determined by the robot from the current environment image. The position information of the initial detection object may be understood as a position of the initial detection object in the target environment.

Further, in the case that it is determined that the current environment image includes the initial detection object, determining the position information of the initial detection object specifically includes steps a1 to a 2.

Step A1: and detecting the current environment image according to a first detection model to obtain a first detection result of the current environment image.

The first detection model may be understood as a model capable of detecting whether the current environment image includes an initial detection object, for example, a dynamic target detection algorithm, and correspondingly, the first detection result may be understood as a detection result indicating whether the initial detection object exists in the image, and in practical applications, the detection result may be a value within a preset interval, for example, any value within an interval [0,1] or any value within an interval [0,100 ].

Specifically, after the robot acquires a current environment image of the target environment through the first shooting module, the current environment image can be input into the first detection model, and the current environment image is processed based on the first detection module, so that a first detection result representing that an initial detection object exists in the current environment image is output.

In practical application, the first detection result obtained by the first detection module only through the current environment image of the target environment may not be accurate, and therefore in the embodiment of the present specification, by obtaining the historical environment image of the target environment, the first detection model can determine the first result according to the historical environment image and the current environment image, so as to improve the accuracy of subsequently determining the initial detection object. The specific implementation is as follows.

The detecting the current environment image according to the first detection model to obtain a first detection result of the current environment image includes:

acquiring a historical environment image of the target environment, wherein the historical environment image is an environment image of the target environment acquired by the first shooting module in a historical manner;

and inputting the historical environment image and the current environment image into a first detection model to obtain a first detection result of the current environment image.

In practical applications, acquiring the historical environment image of the target environment may be understood as acquiring the historical environment image corresponding to the target environment from a historical image storage module in the robot. In practical application, when the robot determines that the initial detection object does not exist in the target environment according to the first detection result, the current environment image is used as a historical environment image and is stored in the historical image storage module, so that the accurate first detection result can be determined based on the historical environment image. The historical image storage module can be understood as a historical image cache for storing historical environment images.

According to the above example, after the robot obtains the current environment image of the machine room entrance through the panoramic camera, the historical environment image of the machine room entrance is obtained from the historical picture cache of the robot, and the historical environment image is the image of the machine room entrance shot by the robot through the panoramic camera.

After determining the historical environment image, inputting the historical environment image and the current environment image into a dynamic target detection algorithm, and obtaining a first detection result representing whether potential suspicious personnel exist in the current environment image through the algorithm.

In the embodiment of the present specification, a historical environment image of a target environment is obtained, and a first detection result of a current environment image is obtained by inputting the historical environment image and the current environment image into a first detection model, so that accuracy of the first detection result is improved, and whether an initial detection object exists in the current environment image is accurately and quickly determined based on the first detection result subsequently.

Further, the inputting the historical environment image and the current environment image into a first detection model to obtain a first detection result of the current environment image includes:

inputting the historical environment image and the current environment image into a first detection model, detecting the historical environment image and the current environment image through an object detection layer of the first detection model, and obtaining a confidence coefficient that the current environment image contains a suspected initial detection object;

determining the area information of the suspected initial detection object under the condition that the confidence coefficient is greater than or equal to a preset confidence coefficient threshold value;

and detecting the current environment image according to the area information of the suspected initial detection object through an area detection layer of the first detection model to obtain a first detection result of the current environment image.

The object detection layer may be understood as a detection layer in the first detection model, which is used for determining a confidence of a suspected initial detection object; in practice, the object detection layer may be a pre-trained neural network model, such as a KNN model. Correspondingly, the confidence of the suspected initial detection object can be understood as a value representing that the suspected initial detection object exists in the current environment image, for example, any value in the [0,1] interval and any value in the [0,100] interval.

In practical application, due to the influence of factors such as light change of target environments such as a machine room and a corridor, or noise of a panoramic camera, images of suspected suspicious people may exist in a panoramic environment image shot by the panoramic camera, and the images may be determined as potential suspicious people by an object detection layer; therefore, in order to improve the detection result of the first detection model, the object determined according to the detection result of the object detection layer is used as a suspected initial detection object, and the area of the suspected initial detection object is more accurately judged through the interval detection layer, so that the first detection result representing whether the initial detection object exists in the current environment image is accurately obtained.

The preset confidence threshold may be set according to an actual application scenario, which is not specifically set in the present specification, for example, when the confidence is any value in the range of [0,1], the preset confidence threshold may be a value of 0.8. The region information may be understood as information of a region where the suspected initial detection object is located in the current environment image. The region detection layer can be understood as a detection layer used for determining connectivity of a suspected initial detection object in a region in the first detection model; in practical application, the region detection layer may be any algorithm capable of determining whether the regions to which the suspected initial detection objects in the current environment image belong are completely connected.

According to the above example, after the robot determines the current environment image and the historical environment image of the machine room entrance, the robot inputs the current environment image and the historical environment image into the dynamic target detection model, and the object detection layer based on the dynamic target detection model detects the current environment image and the historical environment image, so as to output a confidence coefficient representing that potential suspicious personnel may be contained in the current environment image.

After the confidence coefficient is determined, the robot determines whether the confidence coefficient is greater than or equal to a preset confidence coefficient threshold, for example, if the confidence coefficient is 0.9 and the preset confidence coefficient threshold is 0.8, it may be determined that the confidence coefficient is greater than or equal to the preset confidence coefficient threshold; and under the condition that the confidence coefficient is greater than or equal to a preset confidence coefficient threshold value, the robot determines the region information of the potential suspicious personnel from the current environment image.

After the regional information of the potential suspicious people is determined, determining the region of the potential suspicious people from the current environment image based on the regional information through a regional detection layer of a dynamic target detection model, detecting whether the potential suspicious people in the region are connected or not, and outputting the detection result as a first detection result.

In practical application, due to the influence of factors such as light at the entrance of a machine room or noise of a current environment image, a shadow image appears in the current environment image, and when an object detection layer detects the shadow image, the shadow image may serve as a potential suspicious target; the potentially suspicious object may be a potentially suspicious device, a potentially suspicious person.

Therefore, in order to ensure the accuracy of the detection result of the dynamic target detection model, the target detected by the object detection layer is determined as a potential suspicious target, the region where the potential suspicious target in the current environment image is located is determined, whether the potential suspicious target in the region is connected or not is detected through the region detection layer, and the detection result indicating whether the potential suspicious target in the region is complete or connected or not is output.

Step A2: and determining the position information of the initial detection object when the current environment image is determined to contain the initial detection object according to the first detection result.

According to the above example, after the detection result of the region detection layer is output as the first detection result, if it is determined that the potential suspicious people in the region are complete and connected according to the first detection result, it is determined that the current environment image includes the potential suspicious people, that is, the potential suspicious people are around the robot. And obtain location information of the potentially suspect person in the target environment.

If the suspected suspicious personnel in the area are incomplete and not communicated, determining that the current environment image does not contain the suspicious personnel; the robot continues to obtain the current environment image of the machine room entrance in real time through the panoramic camera.

Further, the determining the position information of the initial detection object includes:

determining the area information of the initial detection object in the current environment image, and determining the corresponding relation between the current environment image and the target environment;

and determining the position information of the initial detection object in the target environment according to the corresponding relation between the current environment image and the target environment and the area information of the initial detection object.

The corresponding relationship between the current environment image and the target environment can be understood as the corresponding relationship between the two-dimensional current environment image and the three-dimensional target environment.

According to the above example, when the potential suspicious people contained in the current environment image of the machine room entrance are determined according to the first detection result output by the dynamic target detection algorithm, the region information of the potential suspicious people in the current environment image is determined, and the corresponding relation between the current environment image and the environment of the machine room entrance is determined; and determining corresponding position information of the area information of the potential suspicious personnel in the environment of the entrance of the machine room based on the corresponding relation.

In the embodiment of the present specification, based on the corresponding relationship between the current environment image and the target environment, the position information of the initial detection object in the target environment can be determined according to the area information of the initial detection object in the current environment image. The second shooting module of the subsequent robot can shoot the image of the position information conveniently.

Further, before the detecting the current environment image according to the first detection model and obtaining the first detection result of the current environment image, the method further includes:

and carrying out image segmentation and denoising processing on the current environment image to obtain the processed current environment image.

Along with the above example, because the current environment image shot by the robot through the panoramic camera is large or the number of elements contained in the current environment image is too large, the current environment image needs to be segmented, so as to obtain a plurality of segmented current environment images or the current environment image with redundant elements removed, and the subsequent robot can save the computing resources of the CPU of the robot by processing the segmented current environment image.

Meanwhile, denoising the segmented panoramic image to obtain a processed current environment image, and processing the denoised current environment image by the follow-up robot, so that the accuracy of a follow-up first detection result is further improved.

The object detection method provided by the specification is applied to a robot, the robot shoots a current environment image of a machine room entrance through a panoramic camera, and determines a suspicious target through detecting the current environment image, wherein the suspicious target can be understood as an initial detection object in the embodiment, such as a suspicious person and suspicious equipment; fig. 2 is a flowchart illustrating a process of determining a suspicious target by detecting the current environment image by the robot, where fig. 2 is a flowchart of determining the suspicious target based on the current environment image in an object detection method provided in an embodiment of the present specification. As shown in fig. 2, the robot performs 360-degree panoramic shooting through a panoramic camera arranged on the robot, after a panoramic picture is obtained, the panoramic picture is subjected to picture segmentation processing, the calculation pressure of a CPU of the robot is reduced, and the noise of the segmented panoramic picture is removed, so that whether a potential detection object (i.e., a suspected target) exists in subsequent judgment is further improved. The panoramic picture may be understood as the current environment image in the above embodiments.

After the pictures are subjected to denoising processing, the pictures are added into a picture cache, historical environment images of a machine room entrance are obtained from the picture cache, and whether suspicious targets exist in the current environment images or not is judged by performing dynamic target detection, connected region analysis and other processing on the current environment images and the historical environment images. The dynamic target detection, connected region analysis, and other processes may be implemented by a pre-trained model, or may be implemented by a pre-programmed tool such as a script or a program that is capable of performing dynamic target detection and connected region analysis.

Furthermore, the current environment image shot by the panoramic camera deployed on the robot has the characteristics of large distortion, low resolution and the like; although the robot can rapidly process the current environment image, and the computing resources of the CPU of the robot are not excessively occupied; however, due to the influence of factors such as large distortion and low resolution of the current environment image, although the current environment image is segmented, denoised and subjected to algorithm detection, the detection result based on the current environment image has insufficient accuracy and cannot meet the actual application requirements.

Therefore, after the robot determines that the robot includes the suspicious target based on the current environment image, the suspicious target is taken as a potential suspicious target (i.e., an initial detection object), and the second camera deployed on the robot shoots an area of the potential suspicious target to obtain a high-definition image including the potential suspicious target, and further, based on the high-definition image, whether the suspicious target really exists at the entrance of the machine room is accurately determined, which is described in the following specific manner.

Step 106: and acquiring a current area image of the position information according to a second shooting module of the robot, and determining whether a target detection object exists in the target environment currently according to the current area image.

The second shooting module can be understood as a first camera arranged on the robot, and the camera can shoot high-definition images, such as a code scanning camera; therefore, the robot can accurately identify whether the target detection object exists in the target environment according to the high-definition image. The current region image may be understood as position information of an initial detection object, a current image of a corresponding region in the target environment.

In practical application, after the robot determines that the current environment image includes a potential detection object, because the detection result based on the current environment image has insufficient accuracy and cannot meet the requirement of practical application, the initial detection object is not necessarily a suspicious target, and therefore, the robot triggers a code scanning camera deployed on the robot to shoot the current position area of the initial detection object, so that whether the suspicious target (namely the target detection object) exists in the target environment can be determined more accurately based on the shot current area image.

The robot triggers the code scanning camera deployed on the robot to perform shooting, and can set according to actual application scenes, for example, when the code scanning camera is located on a mechanical arm of the robot, the robot can trigger the mechanical arm to move to an area corresponding to the position information of the initial detection object, and shoot the area through the code scanning camera on the mechanical arm, so that a current area image of the area is shot.

Or when the code scanning camera is positioned on the body of the robot, the robot can trigger the robot to move or rotate, so that the code scanning camera is aligned to the area corresponding to the position information of the initial detection object, and the current area image of the area is shot.

Further, the determining whether a target detection object currently exists in the target environment according to the current region image includes steps B1 to B2:

step B1: and detecting the current area image according to a second detection model to obtain a second detection result of the current area image.

The second detection model may be understood as a model capable of detecting whether a target detection object currently exists in the target environment based on the current region image, for example, a visual detection algorithm; correspondingly, the second detection result can be understood as a value representing whether the target detection object really exists in the target environment currently, for example, any value in the [0,1] interval and any value in the [0,100] interval.

Specifically, after the robot captures a current area image of the position information of the initial detection object through the second capturing module, the current area image is input into the second detection model, and the current area image is detected based on the second detection model, so that a second detection result representing whether a target detection object exists in the target environment at present is output.

In practical application, the robot detects the current area image according to a second detection model to obtain a second detection result of the current area image, and the method comprises the following steps:

acquiring visual features of the current area image;

and inputting the visual characteristics of the current area image into a second detection model to obtain a second detection result of the current area image.

The acquiring of the visual features of the current region image can be understood as that the robot performs feature extraction on the current region image through a feature acquisition algorithm, so as to acquire the visual features corresponding to the current region image. The feature acquisition algorithm can be understood as any algorithm capable of realizing visual feature extraction on an image.

Along the above example, the object detection method can be applied to a scene in which the robot detects suspicious pedestrians in the surrounding environment, wherein the initial detection object can be a potential suspicious pedestrian; the target detection object may be a real suspicious pedestrian. Referring to fig. 3, fig. 3 is a flowchart illustrating a method for determining a suspicious pedestrian based on a current region image in an object detection method according to an embodiment of the present disclosure. Because the robot determines that the suspicious pedestrian has insufficient accuracy and cannot meet the actual application requirements based on the current environment image, the object detection method provided by the specification determines that the suspicious target is used as a potential suspicious pedestrian (namely an initial detection object) based on the current environment image, the robot moves to the region corresponding to the position information of the potential suspicious pedestrian at the entrance of the house by triggering the mechanical arm, and after acquiring the picture of the region according to a code scanning camera (code scanning UVC) on the mechanical arm, the robot performs denoising processing on the picture, and extracts the visual features in the current region image by a feature acquisition algorithm, so that the visual feature distribution of the current region image is acquired. And inputting the visual features into a visual detection model for pedestrian classification detection, so as to obtain a second detection result (namely a pedestrian detection result) representing whether the current region image really has suspicious pedestrians.

As shown in fig. 3, the robot needs to add the visual features to the feature cache while inputting the visual features into the visual detection model for pedestrian classification detection. The robot can adjust the parameters of the visual detection model based on the visual features in the feature cache, and the performance of the visual detection model can be optimized in real time.

In practical application, before inputting the visual features of the image of the current region into the second detection model to obtain the second detection result of the image of the current region, the visual features of the image of the historical region are also required to be obtained, the visual features of the image of the historical region are used as training data to train the second detection model, and in the training process, parameters of the second detection model (single-step detection algorithm) are optimized, so that the detection accuracy and speed of the overall scheme of the object detection method provided by the specification are improved.

In the object detection method provided in this specification, after the visual features of the current area image are processed by the second detection model to obtain the second detection result, in the embodiment of the specification, a manner of determining the second detection result of the current area image by the object area of the suspected target detection object and the object area of the initial detection object is further provided, which is specifically as follows.

The obtaining of the second detection result of the current region image includes:

acquiring a suspected target detection object in the current area image output by the second detection model, and acquiring an object area of the suspected target detection object;

acquiring an object area of the initial detection object in the current environment image;

and determining a second detection result of the current area image according to the object area of the suspected target detection object and the object area of the initial detection object.

In the case that the target detection object is a real suspicious target, the suspected target detection object may be understood as an object of the suspected target detection object in the current area image, for example, an object that may be a real suspicious person in the current area image. The object region of the initial detection object may be understood as a region where the initial detection object is located in the current environment image. The object region of the suspected target detection object may be understood as a region where the suspected target detection object is located in the current region image.

In the above example, after the robot inputs the obtained current region image to the visual inspection model, the visual inspection model processes the current region image, thereby outputting an object in the current region image that may be a real suspicious person. Then the robot determines the area where the object which is probably the real suspicious person is located from the current area image through the area determination algorithm.

And determining the region where the potential suspicious person is located from the current environment image through the region determination algorithm. The region determining algorithm may be any algorithm capable of determining a region in which an object is located in an image.

After the determination of the area is completed, the robot determines a second detection result representing whether the suspicious person really exists in the current area image or not based on the area where the potential suspicious person is located and the area where the object possibly being the real suspicious person is located.

Further, the determining a second detection result of the current region image according to the object region of the suspected target detection object and the object region of the initial detection object includes:

projecting the object area of the suspected target detection object to the current environment image according to the relative position relationship between the first shooting module and the second shooting module;

determining an overlapping area of an object region of the suspected target detection object and an object region of the initial detection object in the current environment image;

determining a region area of a target region of the initial detection object;

and determining the ratio of the overlapping area to the area of the region as a second detection result of the current region image.

Along the above example, the area where the initial detection object is located, and the area where the object, which may be a real suspicious person, is located, may be rectangular areas.

After the rectangular area of the initial detection object in the current environment image is determined, the rectangular area of the object which is probably a real suspicious person is projected to the current environment image shot by the panoramic camera according to the relative position relation between the panoramic camera and the code scanning camera.

Then, the robot determines the rectangular region where the object which is possibly the real suspicious person is located and the overlapping area of the rectangular region where the potential suspicious person is located in the current environment image; and determining the area of the rectangular region where the potential suspicious person is located in the current environment image. And calculating the ratio of the overlapping area to the area of the region, and determining the ratio as a second detection result for representing whether the suspicious person really exists in the current region image. It is convenient to subsequently determine whether the target detection object currently exists in the target environment based on whether the ratio is greater than or equal to a preset ratio threshold.

In the examples of this specification. The object detection method is described by taking the area where the initial detection object is located and the area where the object which may be a real suspicious person is located as rectangular areas as examples; in practical applications, the area where the initial detection object is located and the area where the suspected suspicious object is located may be in any shape, such as a rectangular area, a circular area, and the like.

Further, before the detecting the current area image according to the second detection model and obtaining the second detection result of the current area image, the method includes:

and denoising the current region image to obtain the processed current region image.

Along with the above example, because the current area image shot by the robot through the code scanning camera may have noise, the robot performs denoising processing on the current area image before detecting the current area image to obtain a processed current area image, and the subsequent robot performs detection on the denoised current area image, thereby further improving the accuracy of the subsequent second detection result.

Step B2: and determining whether a target detection object exists in the target environment currently according to the second detection result.

Specifically, when the second detection result is a numerical value representing whether a target detection object currently exists in the target environment, the robot determines whether the second detection result is greater than or equal to a preset determination threshold value after determining the second detection result, if so, determines that a real target detection object currently exists in the target environment, and if not, determines that the target detection object currently does not exist in the target environment. The preset judgment threshold may be set according to an actual application scenario, for example, a value of 0.8.

And under the condition that the second detection result is determined based on the ratio of the overlapping area to the area, judging whether the second detection result is larger than or equal to a preset ratio threshold, if so, determining that a real target detection object currently exists in the target environment, and if not, determining that the target detection object currently does not exist in the target environment. The preset ratio threshold may be set according to an actual application scenario, for example, the preset ratio threshold may be 80% of the ratio when the ratio is any ratio in a range from 0% to 100%.

In the object detection method provided in the embodiment of the present specification, when it is determined that the current environment image acquired by the first photographing module includes an initial detection object, the position information of the initial detection object is determined, the current area image of the position information is acquired from the target environment through the second photographing module, and whether the target detection object currently exists in the target environment is further determined according to the current area image, so that whether the target detection object exists in the target environment is quickly and accurately detected in real time, and the problem of data leakage caused by the target detection object is further avoided.

Further, under the condition that the robot determines that the suspicious target exists at the entrance of the machine room at present, in order to avoid that the suspicious target enters the machine room along with the robot, the robot needs to warn relevant personnel that suspicious personnel exist in the target environment in time, and the specific mode is as follows.

After determining whether a target detection object currently exists in the target environment according to the current region image, the method further includes:

and under the condition that the current target detection object exists in the target environment according to the current region image, generating and sending an object detection notice according to the target detection object.

The object detection notification may be understood as a notification that informs the relevant person to process the suspicious object or warns the relevant person of the presence of the suspicious object in the object environment.

Along with the above example, when the robot determines that suspicious people exist at the entrance of the machine room, the suspicious people may enter the machine room along with the robot when the robot performs door opening, door closing and other actions to enter and exit the machine room. Therefore, the robot generates and sends the object detection notice according to the suspicious target, and in practical application, the object detection notice can be sent to the monitoring terminal, so that the existence of suspicious personnel at the entrance of the machine room of related personnel is notified. Alternatively, the robot sends an object detection notification to an alarm device, which triggers an alarm function based on the received object detection notification, thereby informing the relevant personnel that there is suspicious personnel at the entrance of the machine room.

In specific implementation, when it is determined that the target detection object does not exist in the target environment currently according to the current area image, the robot continues to acquire the current environment image of the target environment based on the first photographing module, and determines whether the current environment image includes the initial detection object.

In the embodiment of the present specification, in a case where it is determined from the current region image that the target detection object currently exists in the target environment, the object detection notification is generated and transmitted from the target detection object. Therefore, the suspicious target in the target environment is informed to the related personnel, the problem of data leakage is further avoided, and the safety of the data is ensured.

The following description will further describe the object detection method with reference to fig. 4 by taking an application of the object detection method provided in this specification in a scenario where a robot detects a suspicious object in a surrounding environment as an example. Fig. 4 shows a processing flowchart of an object detection method applied to a suspicious target scene in a robot detection surrounding environment according to an embodiment of the present specification, and based on the flowchart, the object detection method provided by the present specification provides an anti-trail detection algorithm that uses data of a plurality of cameras for identification and can perform real-time calculation. The algorithm combines various visual detection algorithms, corrects by using the relative positions of a plurality of cameras, and completes detection according to the final correction result. The method specifically comprises the following steps:

step 402: the panoramic camera acquires a picture.

Specifically, the robot can perform 360-degree panoramic shooting on the surrounding environment in real time according to a panoramic camera arranged on the robot, so that an environment picture of the surrounding environment is obtained.

The surrounding environment may be understood as the target environment in the above-described embodiment, and the environment picture may be understood as the current environment image in the above-described embodiment.

Step 404: and (4) detecting a dynamic target.

Specifically, after the robot acquires the environment picture, the robot detects the environment picture based on a dynamic target detection algorithm to acquire a dynamic target detection result.

Here, the target detection result may be understood as the second detection result in the above embodiment. The dynamic object detection algorithm may be a background subtraction algorithm or an optical flow algorithm.

Step 406: and judging whether the suspicious target exists or not.

Specifically, the robot determines whether there is a suspicious target in the surrounding environment based on the detection result of the dynamic target, if so, step 408 is executed, and if not, the robot continues to detect the environment picture acquired by the panoramic camera based on the dynamic target detection algorithm.

Step 408: and triggering the code scanning camera.

Specifically, the robot triggers one or more code scanning cameras to start under the condition that suspicious targets exist in the surrounding environment, when one code scanning camera is arranged on the mechanical arm of the robot, the robot moves the mechanical arm and aligns the code scanning camera on the mechanical arm to the area of the suspicious targets when the code scanning camera is triggered.

In the case of two scanning cameras, one code scanning camera may be disposed on a mechanical arm of the robot, and the scanning camera is moved to an arbitrary direction based on the mechanical arm; and the code scanning camera is arranged in front of the body of the robot and can cover the area in front of the robot. When the code scanning camera is triggered, the robot aligns the code scanning camera to the area of the suspicious target by moving the mechanical arm or adjusting the angle of the vehicle body.

Step 410: and acquiring the picture.

Specifically, after the robot aligns the code scanning cameras to the area of the suspicious target, one or more code scanning cameras shoot the area, so that high-definition pictures of one or more visual angles of the area are obtained.

Step 412: detection is based on visual features.

Specifically, after the robot obtains the high-definition pictures, the robot obtains visual features of the high-definition pictures, and detects the high-definition pictures based on the visual features through a visual detection model to obtain detection results of the high-definition pictures at one or more visual angles.

Step 414: and (5) multi-view detection and proofreading.

Under the condition that a plurality of code scanning cameras are arranged, detection and verification are carried out on the detection results of the high-definition pictures with the plurality of visual angles, and therefore the proofreading confidence coefficient of whether suspicious targets exist in the surrounding environment is obtained.

In practical application, when one code scanning camera is used, multi-view detection verification is not performed, and a result detected by a visual detection model based on visual features is directly used as a proofreading confidence coefficient of whether a suspicious target exists in the surrounding environment.

Step 416: and judging the proof of confidence.

Specifically, whether the proofreading confidence coefficient is higher than or equal to a preset confidence coefficient threshold value is judged, if so, step 418 is executed, and if not, the robot continues to detect the environment picture acquired by the panoramic camera based on a dynamic target detection algorithm.

Step 418: and outputting a detection result.

Specifically, the robot outputs a detection result indicating that a suspicious object exists in the surrounding environment.

The object detection method provided by the present specification may further include that when a person approaches the periphery of the robot, the robot performs dynamic object detection through the panoramic camera first, and the dynamic object detection may be implemented based on a background subtraction algorithm or an optical flow algorithm to find a suspicious rectangular region with the highest confidence (x1, y1, w1, h 1). Then a plurality of code scanning cameras of the robot are started, whether pedestrians exist in the visual field of the cameras is checked, if not, the current detection result is a false alarm, and dynamic target detection is continued;

if so, the detected suspicious rectangular region is denoted as (x2, y2, w2, h 2). According to the relative position relationship between the two cameras, a rectangular area of the code scanning camera can be projected into the panoramic camera, the projection is a plane quadrangle, then the ratio of the overlapping area of the two areas to the original area is set as a confidence coefficient, a detection result with the confidence coefficient higher than or equal to a preset confidence coefficient threshold value is set as a suspicious target, and otherwise, dynamic target detection is continuously executed.

According to the object detection method provided by the specification, data in a plurality of cameras are used for mutual proofreading, so that the resistance of a detection result to noise and illumination conditions of the cameras is improved, the whole scheme is relatively insensitive to the external environment, and the detection precision of the whole scheme is enhanced; because the resolution ratio of the panoramic camera is low, the requirement on computing power is low when dynamic target search is carried out through a panoramic picture, the computation amount of a code scanning camera detection algorithm is relatively large, different visual detection algorithms are selected by the object detection method through the picture characteristics of different cameras, the detection speed of the whole scheme is improved through reasonable step scheduling, the whole computing power consumption of the scheme is reduced, and the detection speed of the detection algorithm is improved; and the dynamic target detection and the algorithm based on the visual characteristics are subjected to parameter tuning, so that the robot can run on a CPU of the robot in real time, and the precision meeting the product requirements can be ensured.

Referring to fig. 5, fig. 2 is a flowchart of a risk object detection method provided in an embodiment of the present specification; the risk object detection method is applied to an operation and maintenance robot of a data center, and specifically comprises the following steps:

step 502: and acquiring a current environment image of the data center according to the first camera of the operation and maintenance robot.

Step 504: determining location information for a potentially risky object if it is determined that the potentially risky object is included in the current environmental image.

Step 506: and acquiring a current area image of the position information according to a second camera of the operation and maintenance robot.

Step 508: in the event that a target risk object is currently present in the data center as determined from the current region image, generating and issuing an object risk notification based on the target risk object.

The operation and maintenance robot can be understood as a robot which performs tasks such as equipment operation maintenance and cleaning in a data center. In practical applications, the operation and maintenance robot includes a robot body and a mechanical arm mounted on the robot body, such as the robot in the above embodiments.

The first camera may be understood as a camera deployed on the operation and maintenance robot, for example, a panoramic camera, and the camera can acquire, in real time, fast, and 360 degrees without dead angles, a current environment image of a data center where the operation and maintenance robot works. In practical application, the first camera may be configured on a robot body of the operation and maintenance robot. Such as the first capture module in the embodiments described above.

The second camera may be understood as one or more cameras deployed on the operation and maintenance robot, such as a code scanning camera, which, when triggered, can acquire a high-definition image of a local area of a data center on which the operation and maintenance robot works. In practical application, the first camera may be disposed on a robot body of the operation and maintenance robot, or may be disposed on a robot arm of the robot. Such as the second capture module in the above-described embodiment.

The target risk object can be understood as a risk person or risk equipment causing leakage of data of the data center or causing damage to equipment of the data center. Such as the target detection object in the above-described embodiments.

The potential risk object may be understood as a risk person or risk equipment that may cause leakage of data of the data center or damage to equipment of the data center. Such as the initial detection object in the above-described embodiment.

During specific implementation, when the operation and maintenance robot performs operation and maintenance on the data center, 360-degree panoramic shooting is performed on the environment of the data center according to the panoramic camera configured on the operation and maintenance robot, so that an environment picture of the data center is obtained. After the operation and maintenance robot obtains the environment picture, the operation and maintenance robot detects the environment picture based on a dynamic target detection algorithm to obtain a dynamic target detection result representing whether a potential risk object exists in the environment picture.

And the operation and maintenance robot judges whether a potential risk object exists in the data center based on the dynamic target detection result, and if not, the operation and maintenance robot continues to detect the environment picture acquired by the panoramic camera based on a dynamic target detection algorithm. If so, the operation and maintenance robot triggers one or more code scanning cameras to start, when one code scanning camera is arranged on the mechanical arm of the operation and maintenance robot, the operation and maintenance robot moves the mechanical arm to align the code scanning camera on the mechanical arm to the area of the potential risk object in the data center under the condition that the code scanning camera is triggered.

Under the condition that two scanning cameras are arranged, one code scanning camera can be configured on a mechanical arm of the operation and maintenance robot, and the scanning camera is moved to any direction based on the mechanical arm; the code scanning camera is arranged in front of the body of the operation and maintenance robot and can cover the area in front of the operation and maintenance robot. When the code scanning camera is triggered, the operation and maintenance robot aligns the code scanning camera to the area of the potential risk object in the data center by moving the mechanical arm or adjusting the angle of the vehicle body.

After the code scanning cameras are aligned to the areas of the potential risk objects, the operation and maintenance robot shoots the areas through one or more code scanning cameras, and therefore high-definition pictures of one or more visual angles of the areas are obtained. After the operation and maintenance robot obtains the high-definition pictures, the visual characteristics of the high-definition pictures are obtained, detection is carried out on the basis of the visual characteristics through a visual detection model, and detection results of the high-definition pictures at one or more visual angles are obtained. In practical application, under the condition that a plurality of code scanning cameras are arranged, detection and verification are carried out on the detection results of high-definition pictures with a plurality of visual angles, so that the proofreading confidence coefficient of whether a target risk object exists in a data center is obtained. And under the condition that one code scanning camera is adopted, multi-view detection and verification are not carried out, and the result of detection based on visual characteristics through the visual detection model is directly used as the proofreading confidence coefficient of whether the target risk object exists in the data center or not.

And the operation and maintenance robot judges whether the proofreading confidence coefficient is higher than or equal to a preset confidence coefficient threshold value, and if so, the operation and maintenance robot outputs a detection result indicating that a target risk object exists in the data center. If the current target is lower than the target threshold value, the robot continues to detect the environmental picture acquired by the panoramic camera based on a dynamic target detection algorithm.

The risk object detection method provided by the specification is applied to an operation and maintenance robot of a data center, the operation and maintenance robot acquires a current environment image of the data center through a first camera, determines position information of a potential risk object under the condition that the current environment image contains the potential risk object, acquires a current region image of the position information from the data center through a second camera, and generates an object risk notification under the condition that the current region image determines that the data center currently has a target risk object, so that whether the risk object exists in the data center is rapidly and accurately detected in real time, related personnel are informed to process the risk object, and the problem of data leakage caused by the risk object is further avoided.

The above is a schematic scheme of a risk object detection method of this embodiment. It should be noted that the technical solution of the risk object detection method and the technical solution of the object detection method belong to the same concept, and details that are not described in detail in the technical solution of the risk object detection method can be referred to the description of the technical solution of the object detection method.

An embodiment of the present disclosure further provides a robot, which includes a robot body, a mechanical arm mounted on the robot body, a first photographing module, and a second photographing module, wherein,

the second shooting module is installed on the robot body, and the second shooting module is installed on the mechanical arm; the robot implements the steps of any of the object detection method and the risk object detection method when performing object detection.

The above is a schematic solution of a robot of the present embodiment. It should be noted that the technical solution of the robot belongs to the same concept as the technical solution of the object detection method, and details of the technical solution of the robot, which are not described in detail, can be referred to the description of the technical solution of the object detection method.

Corresponding to the above method embodiment, the present specification further provides an embodiment of an object detection apparatus, and fig. 6 shows a schematic structural diagram of an object detection apparatus provided in an embodiment of the present specification. As shown in fig. 6, the apparatus is applied to a robot, and includes:

a first obtaining module 602 configured to obtain a current environment image of a target environment according to a first photographing module of the robot;

a first determining module 604 configured to determine position information of an initial detection object in a case where it is determined that the initial detection object is included in the current environment image;

a second determining module 606 configured to obtain a current area image of the position information according to a second photographing module of the robot, and determine whether a target detection object currently exists in the target environment according to the current area image.

Optionally, the first determining module 604 is further configured to:

detecting the current environment image according to a first detection model to obtain a first detection result of the current environment image;

and determining the position information of the initial detection object when the current environment image is determined to contain the initial detection object according to the first detection result.

Optionally, the first determining module 604 is further configured to:

acquiring a historical environment image of the target environment, wherein the historical environment image is an environment image of the target environment acquired by the first shooting module in a historical manner;

and inputting the historical environment image and the current environment image into a first detection model to obtain a first detection result of the current environment image.

Optionally, the first determining module 604 is further configured to:

inputting the historical environment image and the current environment image into a first detection model, detecting the historical environment image and the current environment image through an object detection layer of the first detection model, and obtaining a confidence coefficient that the current environment image contains a suspected initial detection object;

determining the area information of the suspected initial detection object under the condition that the confidence coefficient is greater than or equal to a preset confidence coefficient threshold value;

and detecting the current environment image according to the area information of the suspected initial detection object through an area detection layer of the first detection model to obtain a first detection result of the current environment image.

Optionally, the first determining module 604 is further configured to:

determining the area information of the initial detection object in the current environment image, and determining the corresponding relation between the current environment image and the target environment;

and determining the position information of the initial detection object in the target environment according to the corresponding relation between the current environment image and the target environment and the area information of the initial detection object.

Optionally, the first determining module 604 is further configured to:

and carrying out image segmentation and denoising processing on the current environment image to obtain the processed current environment image.

Optionally, the second determining module 606 is further configured to:

detecting the current area image according to a second detection model to obtain a second detection result of the current area image;

and determining whether a target detection object exists in the target environment currently according to the second detection result.

Optionally, the second determining module 606 is further configured to:

acquiring visual features of the current area image;

and inputting the visual characteristics of the current area image into a second detection model to obtain a second detection result of the current area image.

Optionally, the second determining module 606 is further configured to:

and denoising the current region image to obtain the processed current region image.

Optionally, the second determining module 606 is further configured to:

acquiring a suspected target detection object in the current area image output by the second detection model, and acquiring an object area of the suspected target detection object;

acquiring an object area of the initial detection object in the current environment image;

and determining a second detection result of the current area image according to the object area of the suspected target detection object and the object area of the initial detection object.

Optionally, the second determining module 606 is further configured to:

projecting the object area of the suspected target detection object to the current environment image according to the relative position relationship between the first shooting module and the second shooting module;

determining an overlapping area of an object region of the suspected target detection object and an object region of the initial detection object in the current environment image;

determining a region area of a target region of the initial detection object;

and determining the ratio of the overlapping area to the area of the region as a second detection result of the current region image.

Optionally, the object detection apparatus further includes a notification module configured to:

and under the condition that the current target detection object exists in the target environment according to the current region image, generating and sending an object detection notice according to the target detection object.

In the object detection device provided in the embodiment of the present specification, when it is determined that the current environment image acquired by the first photographing module includes an initial detection object, position information of the initial detection object is determined, a current area image of the position information is acquired from a target environment through the second photographing module, and whether a target detection object currently exists in the target environment is further determined according to the current area image, so that whether the target detection object exists in the target environment is quickly and accurately detected in real time, and a problem of data leakage caused by the target detection object is further avoided.

The above is a schematic configuration of an object detection apparatus of the present embodiment. It should be noted that the technical solution of the object detection apparatus and the technical solution of the object detection method belong to the same concept, and for details that are not described in detail in the technical solution of the object detection apparatus, reference may be made to the description of the technical solution of the object detection method.

Corresponding to the above method embodiment, the present specification further provides an embodiment of a risk object detection device, and fig. 7 shows a schematic structural diagram of a risk object detection device provided in an embodiment of the present specification. As shown in fig. 7, the apparatus is applied to an operation and maintenance robot, and includes:

a first obtaining module 702, configured to obtain a current environment image of the data center according to a first camera of the operation and maintenance robot;

a first determining module 704 configured to determine location information of a risk potential object if it is determined that the risk potential object is included in the current environment image;

a second obtaining module 706 configured to obtain a current area image of the position information according to a second camera of the operation and maintenance robot;

a second determination module 708 configured to generate and issue an object risk notification based on a target risk object in the data center if it is determined from the current region image that the target risk object is currently present.

The risk object detection device provided in the embodiment of the present specification is applied to an operation and maintenance robot, the operation and maintenance robot acquires a current environment image of a data center through a first camera, determines position information of a potential risk object under the condition that the current environment image includes the potential risk object, acquires a current region image of the position information from the data center through a second camera, and further determines whether a target risk object exists in the data center according to the current region image, so that whether a risk object exists in the data center is quickly and accurately detected in real time, and the problem of data leakage caused by the risk object is further avoided.

The above is a schematic scheme of a risk object detection apparatus of the present embodiment. It should be noted that the technical solution of the risk object detection apparatus and the technical solution of the object detection method belong to the same concept, and details that are not described in detail in the technical solution of the risk object detection apparatus can be referred to the description of the technical solution of the object detection method.

FIG. 8 illustrates a block diagram of a computing device 800, according to one embodiment of the present description. The components of the computing device 800 include, but are not limited to, memory 810 and a processor 820. The processor 820 is coupled to the memory 810 via a bus 830, and the database 850 is used to store data.

Computing device 800 also includes access device 840, access device 840 enabling computing device 800 to communicate via one or more networks 860. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 840 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.

In one embodiment of the present description, the above-described components of computing device 800, as well as other components not shown in FIG. 8, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 8 is for purposes of example only and is not limiting as to the scope of the description. Those skilled in the art may add or replace other components as desired.

Computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 800 may also be a mobile or stationary server.

Wherein the processor 820 is configured to execute computer-executable instructions that, when executed by the processor 820, implement any of the object detection methods and the steps of the risk object detection method.

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the object detection method belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the object detection method.

An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement any of the object detection methods and the steps of the risk object detection method.

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the object detection method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the object detection method.

An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer program causes the computer to execute any of the object detection method and the steps of the risk object detection method.

The above is an illustrative scheme of a computer program of the present embodiment. It should be noted that the schematic plan of the computer program and the technical plan of the object detection method described above belong to the same concept, and details that are not described in detail in the schematic plan of the computer program can be referred to the description of the technical plan of the object detection method described above.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present disclosure is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present disclosure. Further, those skilled in the art should also appreciate that the embodiments described in this specification are preferred embodiments and that acts and modules referred to are not necessarily required for this description.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the specification and its practical application, to thereby enable others skilled in the art to best understand the specification and its practical application. The specification is limited only by the claims and their full scope and equivalents.

Claims (14)

1. An object detection method applied to a robot includes:

acquiring a current environment image of a target environment according to a first shooting module of the robot;

determining position information of an initial detection object under the condition that the current environment image is determined to contain the initial detection object;

and acquiring a current area image of the position information according to a second shooting module of the robot, and determining whether a target detection object exists in the target environment currently according to the current area image.

2. The object detection method according to claim 1, said determining, in a case where it is determined that an initial detection object is included in the current environment image, position information of the initial detection object, comprising:

detecting the current environment image according to a first detection model to obtain a first detection result of the current environment image;

and determining the position information of the initial detection object when the current environment image is determined to contain the initial detection object according to the first detection result.

3. The object detection method according to claim 2, wherein the detecting the current environment image according to the first detection model to obtain the first detection result of the current environment image comprises:

acquiring a historical environment image of the target environment, wherein the historical environment image is an environment image of the target environment acquired by the first shooting module in a historical manner;

and inputting the historical environment image and the current environment image into a first detection model to obtain a first detection result of the current environment image.

4. The object detection method according to claim 3, wherein the inputting the historical environment image and the current environment image into a first detection model to obtain a first detection result of the current environment image comprises:

inputting the historical environment image and the current environment image into a first detection model, detecting the historical environment image and the current environment image through an object detection layer of the first detection model, and obtaining a confidence coefficient that the current environment image contains a suspected initial detection object;

determining the area information of the suspected initial detection object under the condition that the confidence coefficient is greater than or equal to a preset confidence coefficient threshold value;

and detecting the current environment image according to the area information of the suspected initial detection object through an area detection layer of the first detection model to obtain a first detection result of the current environment image.

5. The object detection method of claim 1, said determining location information of the initial detection object, comprising:

determining the area information of the initial detection object in the current environment image, and determining the corresponding relation between the current environment image and the target environment;

and determining the position information of the initial detection object in the target environment according to the corresponding relation between the current environment image and the target environment and the area information of the initial detection object.

6. The object detection method of claim 1, said determining from the current region image whether a target detection object is currently present in the target environment, comprising:

detecting the current area image according to a second detection model to obtain a second detection result of the current area image;

and determining whether a target detection object exists in the target environment currently according to the second detection result.

7. The object detection method according to claim 6, wherein the detecting the current region image according to the second detection model to obtain the second detection result of the current region image comprises:

acquiring visual features of the current area image;

and inputting the visual characteristics of the current area image into a second detection model to obtain a second detection result of the current area image.

8. The object detection method according to claim 6, wherein the obtaining of the second detection result of the current region image includes:

acquiring a suspected target detection object in the current area image output by the second detection model, and acquiring an object area of the suspected target detection object;

acquiring an object area of the initial detection object in the current environment image;

and determining a second detection result of the current area image according to the object area of the suspected target detection object and the object area of the initial detection object.

9. The object detection method according to claim 8, wherein the determining a second detection result of the current region image according to the object region of the suspected target detection object and the object region of the initial detection object includes:

projecting the object area of the suspected target detection object to the current environment image according to the relative position relationship between the first shooting module and the second shooting module;

determining an overlapping area of an object region of the suspected target detection object and an object region of the initial detection object in the current environment image;

determining a region area of a target region of the initial detection object;

and determining the ratio of the overlapping area to the area of the region as a second detection result of the current region image.

10. The object detection method according to any one of claims 1 to 9, further comprising, after determining whether a target detection object is currently present in the target environment from the current region image:

and under the condition that the current target detection object exists in the target environment according to the current region image, generating and sending an object detection notice according to the target detection object.

11. A risk object detection method is applied to an operation and maintenance robot of a data center and comprises the following steps:

acquiring a current environment image of the data center according to a first camera of the operation and maintenance robot;

determining position information of a potential risk object in the case that the current environment image is determined to contain the potential risk object;

acquiring a current area image of the position information according to a second camera of the operation and maintenance robot;

in the event that a target risk object is currently present in the data center as determined from the current region image, generating and issuing an object risk notification based on the target risk object.

12. A robot comprises a robot body, a mechanical arm arranged on the robot body, a first shooting module and a second shooting module, wherein,

the second shooting module is installed on the robot body, and the second shooting module is installed on the mechanical arm; the robot realizes the steps of the object detection method according to any one of claims 1 to 10 and the risk object detection method according to claim 11 when performing object detection.

13. A computing device, comprising:

a memory and a processor;

the memory is for storing computer executable instructions and the processor is for executing the computer executable instructions which, when executed by the processor, perform the steps of the object detection method of any one of claims 1 to 10 and the risk object detection method of claim 11.

14. A computer readable storage medium storing computer executable instructions which, when executed by a processor, implement the steps of the object detection method of any one of claims 1 to 10 and the risk object detection method of claim 11.

Images