CN113450534A - Device and method for detecting approach of children to dangerous goods - Google Patents

Abstract

The embodiment of the application provides a device and a method for detecting that a child is close to a dangerous article. The camera module comprises a color camera and a depth camera, and is used for shooting RGB images and depth images corresponding to the RGB images; the controller is configured to: detecting whether children and dangerous goods exist in the RGB image; if both the children and the dangerous goods exist, calculating the approaching distance of the children and the dangerous goods according to the depth image; judging whether the approach distance is smaller than the safety distance; and if the approaching distance is less than the safe distance, sending a control signal for alarming. This application utilizes RGB image to carry out children and detects and hazardous articles and detect, when detecting out hazardous articles, judges based on the distance of degree of depth image between children and the hazardous articles, sends the control signal who carries out the warning when this distance is less than safe distance, when having solved unmanned nurse children, is difficult to discover children and is close to the problem of hazardous articles, through timely warning, is favorable to ensureing children's safety.

Description

Device and method for detecting approach of children to dangerous goods

Technical Field

The application relates to the technical field of child monitoring devices, in particular to a device and a method for detecting that a child is close to a dangerous article.

Background

With the development of the times, every family pays more and more attention to the growth and development of children, and the child care gradually becomes one of the core problems in the family. The young children can approach and take dangerous objects such as scissors, cutters, lighters and the like at any time due to the curiosity of the young children and the unknown property of the objects, and certain damage can be caused to the young children. Parents cannot keep the children around for 24 hours without time and ensure that the children do not contact dangerous goods within the activity range. Many parents choose to install the mode of surveillance camera head and carry out remote guardianship to children in at home, however, the surveillance camera head only can play the monitoring effect when the head of a family looks over, and the head of a family does not have the time to look over the surveillance camera head always, can't carry out the guardianship of full time quantum to children.

Disclosure of Invention

The application provides a device and a method for detecting that a child is close to a dangerous article, and aims to solve the problem of monitoring the child in a full time period.

In a first aspect, the present application provides a device for detecting the proximity of a child to a hazardous material, the device comprising: a camera module and a controller, wherein,

the camera module comprises a color camera and a depth camera, and is used for shooting RGB images and depth images corresponding to the RGB images;

the controller is in communicative connection with the camera module, the controller configured to: detecting whether children and dangerous goods exist in the RGB image; if children and dangerous goods exist in the RGB image, calculating the approaching distance of the children and the dangerous goods according to the depth image; judging whether the approach distance is smaller than a safety distance; and if the approaching distance is less than the safe distance, sending a control signal for alarming.

In a second aspect, the present application provides a method of detecting the proximity of a child to a hazardous material, the method comprising:

shooting an RGB image and a depth image corresponding to the RGB image;

detecting whether children and dangerous goods exist in the RGB image;

if children and dangerous goods exist in the RGB image, calculating the approaching distance of the children and the dangerous goods according to the depth image;

judging whether the approach distance is smaller than a safety distance;

and if the approaching distance is less than the safe distance, sending a control signal for alarming.

In a third aspect, the present application provides a method of detecting the proximity of a child to a hazardous material, the method comprising:

receiving an RGB image and a depth image corresponding to the RGB image;

detecting whether children and dangerous goods exist in the RGB image;

if children and dangerous goods exist in the RGB image, calculating the approaching distance of the children and the dangerous goods according to the depth image;

judging whether the approach distance is smaller than a safety distance;

and if the approaching distance is less than the safe distance, sending a control signal for alarming.

The device and the method for detecting the approach of the children to the dangerous goods have the advantages that:

this application is through the depth map that shoots indoor RGB image and correspond, utilizes RGB image to carry out children and detects and hazardous articles detect, when detecting out hazardous articles, judges based on the distance between children and the hazardous articles of depth image pair, reports to the police when this distance is less than safe distance, when having solved unmanned nurse children, is difficult to discover children and is close to the problem of hazardous articles, through in time reporting to the police, is favorable to ensureing children's safety.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without any creative effort.

Fig. 1 is a schematic structural diagram of an apparatus for detecting a child approaching a dangerous goods according to an embodiment of the present disclosure;

FIG. 2 is a schematic view of a framework according to an embodiment of the present disclosure;

fig. 3 is a schematic flow chart illustrating a method for detecting a child approaching a dangerous goods according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of another method for detecting that a child is close to a dangerous goods according to an embodiment of the present disclosure.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In a first aspect, the present embodiment provides a device for detecting a child approaching a dangerous good, which may include a camera module 100, a controller 200 and an alarm 300, as shown in fig. 1.

The camera module 100 includes a color camera 101 and a depth camera 102, wherein the color camera 101 can be used for capturing RGB images, and the depth camera 102 can be used for capturing depth images. The camera module 100 may be selected as a module in which the output color image and the depth image correspond to each other, where the correspondence includes the same time, angle of view, and resolution, the angle of view includes a horizontal angle of view and a vertical angle of view, and the horizontal angle of view of the color camera 101 and the vertical angle of view of the depth camera 102 need to be the same.

The controller 200 may be a processor having data processing capabilities. The controller 200 can be electrically connected with the camera module 100 to form an all-in-one machine, and the pictures shot by the camera module 100 can be obtained through cables; the controller 200 may also be connected to the camera module 100 in a wireless communication manner, and acquire pictures taken by the camera module 100 through a network, for example, the controller 200 may be a remote server or an intelligent terminal, and may control the camera module 100 through the communication network.

The camera module 100 or the all-in-one machine can be installed by a worker at a suitable position in the home of the user to photograph an area to be monitored. For convenience of description, the camera module 100 and the controller 200 are wirelessly connected for example.

After the camera module 100 is installed and started, the RGB image and the depth image are captured and uploaded to the controller 200. The shooting frame rate of the camera module 100 can be set to 30 frames per second, which can meet the detection requirement, and of course, other frame rates can be selected.

For RGB images captured by the color camera 101, the controller 200 may capture N frames of images in succession to form an image sequence, and perform the child detection and the threat detection on each frame of image in the image sequence, where N represents the selected number of captured images, for example, N may be selected to be 10, and for a color camera 101 with a capturing frame rate of 30 frames per second, the RGB images may be captured three times in one second, and 10 frames of RGB images are captured each time for the child detection and the threat detection.

The children detection can be based on human skeleton key point detection technology, such as openposition, human skeleton key point detection is carried out on an RGB image, feature mapping is created for the image through a convolutional neural network, then the image is divided into two branches to be processed, one branch extracts a body part position confidence map through the convolutional neural network, the other branch extracts skeleton key point affinity, the body part position confidence map and the skeleton key point affinity map are analyzed through a greedy inference algorithm, through even matching, associated skeleton key points are found, the associated skeleton key points are connected, and due to the fact that the skeleton key point affinity map has vectority, the associated skeleton key points obtained according to the even matching generally belong to the same person, and all the skeleton key points of the same person are combined into an integral skeleton of the same person. As shown in fig. 2, white dots are skeleton key points, and the skeleton key points detected in this embodiment are: the skeleton comprises a head skeleton key point, a neck skeleton key point, a left shoulder skeleton key point, a left elbow skeleton key point, a left wrist skeleton key point, a right shoulder skeleton key point, a right elbow skeleton key point, a right wrist skeleton key point, a left hip skeleton key point, a left knee skeleton key point, a left ankle skeleton key point, a right hip skeleton key point, a right knee skeleton key point, a right wrist skeleton key point, a left eye skeleton key point, a right eye skeleton key point, a left ear skeleton key point and a right ear skeleton key point, wherein white dots are connected through lines to form the skeleton of the human.

In practical implementation, adults and children can be distinguished according to the skeleton characteristics of the adults and children, for example, the skeleton size ratio of the adults and the skeleton size ratio of the children are different, and the ratio of the head to the body of the children is usually larger than that of the heads and the bodies of the adults. One way to distinguish adults from children is to: calculating the distance between the head skeleton key point and the neck skeleton key point, and recording as a first distance; calculating the distance between the key point of the left ankle and the key point of the left knee bone, and recording as a second distance; compare the ratio of first interval and second interval and children detection threshold value, if this ratio is greater than children and detects the threshold value, can regard this skeleton information to belong to children, if this ratio is less than or equal to children and detects the threshold value, can regard this skeleton information to belong to adult, wherein, children detect the threshold value and can set to 0.5. Of course, adults and children can be distinguished according to other ways, and the embodiment is not illustrated. In addition, if there is no other person at home than the child being monitored, a distinction between adults and children may not be necessary.

For the ith frame image f in the image sequence_iIf the human body exists, detecting the key points of the human skeleton, wherein i belongs to [1, N ]]When a child is detected, the position information (x ') of the key points of the two hand bones of the human body in the RGB image frame is output'_ki，y′_ki) K is 1, 2, wherein (x'_ki，y′_ki) Is a relative coordinate in a pixel coordinate system, wherein the pixel coordinate system is a coordinate system established in the RGB image. The positions of the hand skeleton key points of the frame are mapped to the depth image in combination with the depth image acquired by the depth camera 102, and the depth information H 'of the child hand skeleton key points is acquired'_ki. Finally, outputting the detection result of each frame image { (x {)'_ki，y′_ki，H′_ki) If the human body is an adult or a child, the determination method may refer to the above method for distinguishing an adult from a child, and details thereof are not repeated herein. If the image f_iIf no human body exists, the detection result is not output, which indicates that no human body is detected, the frame of image is not further processed, and the next frame of image is obtained for detecting the children.

The detection of dangerous articles and the detection of children can be carried out synchronously so as to improve the detection efficiency. Hazardous article detection may be based on common article detection technologies such as Fast RCNN, SSD, YOLO series, etc. The scheme takes a Yolov3 algorithm as an example, and the algorithm scheme is as follows: inputting an image, extracting features through a characteristic neural network, outputting the feature image on three scales, extracting the detected target position and type according to the output of the network, and then obtaining final detection information through NMS (Non-Maximum Suppression) operation.

Based on the object detection technology, a large number of dangerous goods image samples of different types are respectively input into the neural network for training, and the finally obtained dangerous goods detection neural network can identify whether the input images contain dangerous goods or not and the type of each dangerous goods if the input images contain the dangerous goods.

And inputting the RGB image into a dangerous article detection neural network, and judging whether dangerous articles exist in the RGB image or not through the pre-trained dangerous article detection neural network. For a frame of RGB image f_iAfter being compressed, the compressed image is input into a dangerous goods detection neural network, and dangerous goods information { (x) detected by the frame of RGB image is output_0ij，y_0ij，w_ij，h_ij，c_ij) J is 0, 1, wherein j is more than or equal to 0, and represents the j-th dangerous goods detected in the current image, x_0ij，y_0ijRespectively representing the i-th frame RGB image f_iDetecting the j-th dangerous goods in the detection frame, wherein the relative coordinate of the center point in the pixel coordinate system, w_ij，h_ijThe size, x, of the width and height of the detection frame with respect to the entire RGB image_0ij，y_0ij，w_ij，h_ij∈[0，1]，c_ijInformation indicating the type of the dangerous material, c_ijE.g. J. For example, assuming that the kinds of dangerous goods detectable by the neural network for detecting dangerous goods are scissors, fruit knives, lighters, etc., the frame RGB image f_iTwo kinds of dangerous goods of scissors, fruit knife exist simultaneously in, then can export two dangerous goods detection information: c. C_i1And c_i2，c_i1Showing scissors, c_i2Showing a fruit knife.

The center point (x) of the dangerous goods_0ij，y_0ij) Mapping to corresponding depth image, and outputting depth value H of center point_ij，H_ijAnd the depth value representing the central point of the jth detection box in the ith frame of depth image grabbed by the depth camera 102.

Calculating the upper left corner (x) of the prediction frame of each dangerous goods under the pixel coordinate system by the following formula_1ij，y_1ij) Lower left corner (x)_2ij，y_2ij) Lower right corner (x)_3ij，y_3ij) The upper right corner (x)_4ij，y_4ij) Relative coordinates of these four vertices:

if x_sijIf x is less than or equal to 0, let x_sij0; if y_sijWhen the value is less than or equal to 0, let y_sij0; if x_sijLet x be more than or equal to 1_sij1 is ═ 1; if y_sijWhen the value is more than or equal to 1, let y_sij1 where s 1.

Available for each hazardous material { (x)_1ij，y_1ij)，(x_2ij，y_2ij)，(x_3ij，y_3ij)，(x_4ij，y_4ij)，H_ijDenotes its position and depth when the image f_iAnd when no dangerous goods exist, the position and the depth are not output, the dangerous goods are not detected, the frame of image is not further processed, and the next frame of image is obtained for dangerous goods detection.

When both the child and the hazardous material are contained in the RGB image, the distance between the child and the hazardous material can be calculated based on the depth image.

The Euclidean distance is used for calculating the distance between the child and the dangerous goods. For image f_iSince the shape of one of the dangerous goods may be different from the shape of the prediction box, there may be no dangerous goods at the four vertices of the prediction box, and the depth value of the center point of the prediction box may be taken as the depth value of the dangerous goods. Taking the minimum value of the distances between the key points of the hand and the four vertexes of the prediction frame of the dangerous goods as the distance between the child and the dangerous goods in the frame image, and calculating the formula as follows:

(2) in the formula, L_ijRepresenting image f_iMiddle child and dangerous goods c_ijThe distance of (c).

Due to the image f_iIn which a plurality of dangerous goods may exist, andtherefore, the distance between the child and each detected dangerous article is calculated, the minimum value is taken as the distance between the child and the nearest dangerous article, the nearest dangerous article is called as a dangerous article to be detected, and the type of the dangerous article nearest to the child is the type of the dangerous article to be detected.

(3) In the formula, L_iRepresenting image f_iThe distance between the child and the nearest dangerous goods, wherein the dangerous goods is the jth detected dangerous goods, and the category is c_ij。

When the distance between the dangerous goods to be detected and the children is far, the safety of the children is difficult to threaten, therefore, whether the distance between the children and the dangerous goods to be detected is smaller than the first distance epsilon or not is judged₁And determining whether the dangerous goods to be detected have threat.

First distance epsilon₁Can be selected to be 50 cm if L_i＜ε₁Then the result of the frame is retained and the result of retaining the frame image is output as follows: (i, L)_i，c_ij) I.e. (few frames, shortest distance, hazardous article category); otherwise, neglecting the frame result, and judging whether the dangerous goods to be detected in the next frame have threat.

The output result of the dangerous goods to be detected of a plurality of frames of images in one image sequence is as follows: { (1, 10, clipper), (3, 9.5, clipper), (4, 10.2, clipper), (5, 9, fruit knife) }, the meanings are as follows: the first frame of the image sequence detected a clipper, the distance between the child's hand and the clipper was 10 centimeters; the first frame detects the scissors, and the distance between the hands of the child and the scissors is 10 centimeters; in the third frame, the scissors are detected, and the distance between the hands of the children and the scissors is 9.5 centimeters; in the fourth frame, the scissors are detected, and the distance between the hands of the children and the scissors is 10.2 centimeters; in the fifth frame, a fruit knife was detected, and the distance between the child's hand and the fruit knife was 9 cm.

As shown in the output result of the image sequence, in an image sequence, the dangerous goods to be detected may have more than one category, and the output result of each frame may be integrated by a multi-frame fusion mechanism decision, so as to determine the dangerous goods to be detected and the category thereof closest to the child.

Suppose that n frame image results are retained in an image sequence, where n₁The frame output result is of type C₁N of₁The shortest distance between the children and the dangerous goods C to be detected in the frame is respectively

Has n₂The frame output result is of type C₂The shortest distance is respectively

Has n_mThe frame output result is of type C_mThe shortest distance is respectively

Wherein n is₁+n₂+…+n_m＝n。

And (3) judging the dangerous goods type closest to the output result by adopting a majority voting rule, namely judging that the majority of the types in the output result is the dangerous goods type closest to the output result, wherein the calculation formula is as follows:

n_α＝max_a{n_α，α＝1，2，…，m} (4)

averaging the distances in the output results of all kinds, and taking the average as the distance between the dangerous goods and the children, wherein the calculation formula is as follows:

when the dangerous goods are close to the children, the alarm can be given to prompt the user to check the dangerous goods. Will be at distance

From a safe distance epsilon₂And comparing to judge whether the distance between the dangerous goods and the children meets the alarm requirement. If it is not

Judgment of children and dangerous goods c_αIf the distance is very short, the dangerous variety information is output, and the alarm 300 is controlled to give an alarm; otherwise, the child is considered to have a certain distance from the dangerous goods, and the image is continuously captured.

Besides using the alarm 300 to send out an alarm, the controller 200 can also generate alarm information, and the alarm information is sent to a mobile communication terminal of a user, such as a smart phone, so that the user can conveniently check the alarm information in time.

To further explain the detection process of a child approaching a dangerous goods, the second aspect of this embodiment shows a method for detecting a child approaching a dangerous goods, and as shown in fig. 3, the method for detecting a child approaching a dangerous goods provided by this embodiment of the present application includes the following steps:

step S110: and shooting the RGB image and the depth image corresponding to the RGB image.

The method comprises the steps of acquiring an RGB image and a depth image corresponding to the RGB image in a user family through a camera module, wherein the shooting frame rate can be 30 frames per second, and each frame of RGB image corresponds to one frame of depth image.

Step S120: and detecting whether children and dangerous goods exist in the RGB image.

The RGB images with preset frame number can be used as an image sequence, each frame of RGB image in the image sequence is detected, and whether the frame of image has RGB images simultaneously or not is detected.

For example, the preset number of frames may be selected to be 10 frames. For one frame of RGB image, whether a human body exists in the RGB image is detected based on a human body skeleton key point detection technology, if the human body exists, whether the identified human body is an adult or a child can be judged according to different skeleton size proportions of the adult and the child, if the human body exists, hand information of the child is output according to a depth image corresponding to the RGB image, wherein the hand information comprises the position and the depth value of a hand skeleton key point, the position is called as a first coordinate, and the depth value is called as a first depth value. And detecting whether the dangerous goods exist in the RGB image or not based on the goods detection technology, if so, outputting positions and depth values of four vertexes of the dangerous goods prediction box according to the depth image corresponding to the RGB image, wherein the position of each vertex is commonly called as a second coordinate, and the depth values of the four vertexes are all the depth values of the center of the dangerous goods prediction box and are called as second depth values.

If no children or no dangerous goods exist in the frame of RGB image, or neither children nor dangerous goods exist, the frame of RGB image and the depth image corresponding to the frame of RGB image are not further analyzed, the next frame of RGB image and the depth image corresponding to the frame of RGB image can be obtained, and whether the children and the dangerous goods exist at the same time or not is detected.

Step S130: and if the children and the dangerous goods exist in the RGB image, calculating the approaching distance of the children and the dangerous goods according to the depth image.

If the child and the dangerous goods exist in one frame of RGB image, for one dangerous goods, calculating distances between the four second coordinates and the first coordinate of the dangerous goods based on the depth image corresponding to the frame of RGB image, wherein the distances can be selected as Euclidean distances, and when the Euclidean distances are calculated, the depth values corresponding to the four second coordinates of the dangerous goods are all the second depth values. And comparing the Euclidean distances between the first coordinate and the four second coordinates respectively, and taking the minimum Euclidean distance as the distance between the hand of the child and the dangerous goods.

And if a plurality of dangerous goods exist in the frame of RGB image, taking the dangerous goods corresponding to the minimum Euclidean distance as the dangerous goods to be detected of the frame of RGB image.

Judging whether the Euclidean distance corresponding to the dangerous goods to be detected is smaller than a first distance or not, and if so, calculating a proximity distance according to the Euclidean distance; if the Euclidean distance is larger than or equal to the first distance, the approach distance is not calculated according to the Euclidean distance, and the Euclidean distance corresponding to the dangerous goods to be detected in the next frame of RGB image can be continuously judged.

For an image sequence, after judging whether dangerous goods used for calculating the approaching distance exist in each frame of image, the category of the dangerous goods used for calculating the approaching distance can be counted, the category of the occurrence times is selected as the dangerous goods corresponding to the image sequence, and the average value of Euclidean distances between the dangerous goods and the key of the hand skeleton of the child is calculated and is used as the approaching distance between the child and the dangerous goods in the image sequence.

Step S140: and judging whether the approaching distance is smaller than the safe distance.

And comparing the approach distance corresponding to the image sequence with the safety distance, and judging whether the approach distance is smaller than the safety distance.

If the approach distance is greater than or equal to the safe distance, continuing to perform the approach distance analysis on the next image sequence.

Step S150: and if the approaching distance is less than the safe distance, sending a control signal for alarming.

And if the approach distance corresponding to the image sequence is less than the safety distance, sending a control signal for alarming, wherein in some embodiments, the control signal can be a signal for controlling an alarm of the alarm, and in other embodiments, the control signal can also be a signal for sending alarm information to the mobile communication terminal to remind a user to check children.

In a third aspect, this embodiment also provides a schematic flow chart of another method for detecting that a child approaches a dangerous good, where the method may be used in a controller of a device for detecting that a child approaches a dangerous good, and may also be used in a control device such as a server with computing data computing capability, as shown in fig. 4, the method for detecting that a child approaches a dangerous good includes the following steps:

step S210: receiving an RGB image and a depth image corresponding to the RGB image.

In some embodiments, the control device may receive an indoor RGB image and a depth image corresponding to the RGB image captured by a camera module of a device for detecting that a child approaches a dangerous good, and in other embodiments, the control device may also receive the indoor RGB image and the depth image corresponding to the RGB image in other manners, for example, receive the indoor RGB image and the depth image corresponding to the RGB image captured by an intelligent communication terminal having a binocular camera module, where the binocular camera module includes a color camera and a depth camera.

Step S220: and detecting whether children and dangerous goods exist in the RGB image.

Step S230: and if the children and the dangerous goods exist in the RGB image, calculating the approaching distance of the children and the dangerous goods according to the depth image.

Step S240: and judging whether the approaching distance is smaller than the safe distance.

Step S250: and if the approaching distance is less than the safe distance, sending a control signal for alarming.

The steps S120 to S150 can be referred to in sequence for the implementation of the steps S220 to S250, and are not described herein again.

In a fourth aspect, the present embodiment further provides a computer-readable storage medium, on which a computer program is stored, which, when executed, implements the method for detecting the approach of a child to a dangerous good according to the third aspect.

By above-mentioned embodiment, this application embodiment is through the depth map that shoots indoor RGB image and correspond, utilize RGB image to carry out children and detect and hazardous articles detect, when detecting out hazardous articles, judge based on the distance between children and the hazardous articles of depth image pair, report to the police when this distance is less than safe distance, when having solved unmanned nurse children, be difficult to discover that children are close to the problem of hazardous articles, through timely warning, be favorable to ensureing children's safety. Furthermore, the camera module group provided with the color camera and the depth camera is used for shooting the RGB images and the depth images corresponding to the RGB images, the distance between the child and the dangerous goods is calculated by using the depth images corresponding to the RGB images, and calculation errors caused by inconsistent visual angles of the RGB images and the depth images are reduced; due to the fact that the placing angles of the objects are diversified, when the distance between the children and the dangerous goods is calculated, a point with the minimum Euclidean distance from the children is selected from four vertexes of the dangerous goods prediction frame, the distance between the point and the children is calculated, and calculation errors caused by the diversification of the placing angles of the dangerous goods are reduced; the distance between the dangerous goods and the children in the multi-frame images in the image sequence is calculated in a multi-frame fusion mode, and instability of detecting the distance between the children and the dangerous goods by using single-frame images is reduced. In summary, this application embodiment can effectively judge whether have the hazardous articles within children's the safe distance, and judge that the accuracy is high.

Since the above embodiments are all described by referring to and combining with other embodiments, the same portions are provided between different embodiments, and the same and similar portions between the various embodiments in this specification may be referred to each other. And will not be described in detail herein.

It is noted that, in this specification, relational terms such as "first" and "second," and the like, are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a circuit structure, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such circuit structure, article, or apparatus. Without further limitation, the presence of an element identified by the phrase "comprising an … …" does not exclude the presence of other like elements in a circuit structure, article or device comprising the element.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

The above-described embodiments of the present application do not limit the scope of the present application.

Claims (10)

1. A device for detecting the proximity of a child to a hazardous material, comprising: a camera module and a controller, wherein,

the camera module comprises a color camera and a depth camera, and is used for shooting RGB images and depth images corresponding to the RGB images;

the controller is in communicative connection with the camera module, the controller configured to: detecting whether children and dangerous goods exist in the RGB image; if children and dangerous goods exist in the RGB image, calculating the approaching distance of the children and the dangerous goods according to the depth image; judging whether the approach distance is smaller than a safety distance; and if the approaching distance is less than the safe distance, sending a control signal for alarming.

2. The apparatus of claim 1, wherein the controller is configured to calculate the proximity distance between the child and the hazardous material according to the depth image, comprising:

calculating a first depth value of the hand of the child and a second depth value of the center of the dangerous goods according to the depth image;

calculating the Euclidean distance between a first coordinate of the hand of the child and a second coordinate of the dangerous goods prediction box based on the first depth value and the second depth value, wherein the first coordinate of the hand of the child and the second coordinate of the dangerous goods prediction box are obtained according to the RGB image;

and obtaining the approaching distance according to the Euclidean distance.

3. The apparatus for detecting the approach of a child to a dangerous material according to claim 2, wherein the second coordinate is a coordinate of a vertex of the dangerous material prediction box, and the euclidean distance is a minimum value of distances between the first coordinate and four second coordinates.

4. The apparatus for detecting the proximity of a child to a hazardous material according to claim 2, wherein said controller is configured to derive said proximity distance from said euclidean distance, comprising:

judging whether the Euclidean distance is smaller than a first distance, wherein the first distance is larger than a safety distance;

if the Euclidean distance is smaller than the first distance, calculating the approach distance according to the Euclidean distance;

and if the Euclidean distance is greater than or equal to the first distance, calculating the approach distance according to the next frame of RGB image and the depth image corresponding to the RGB image.

5. The apparatus for detecting the proximity of a child to a hazardous material according to claim 2, wherein said controller is configured to derive said proximity distance from said euclidean distance, comprising:

comparing Euclidean distances corresponding to a plurality of dangerous goods in the depth image, and determining the dangerous goods corresponding to the smallest Euclidean distance as the dangerous goods to be detected of the depth image;

and obtaining the approaching distance according to the Euclidean distance of the dangerous goods to be detected.

6. The apparatus for detecting the approaching of a child to a dangerous material according to claim 5, wherein the controller is configured to obtain the approaching distance according to the Euclidean distance of the dangerous material to be detected, and comprises:

comparing the occurrence times of various dangerous goods to be detected in the depth image with the preset frame number to obtain the dangerous goods to be detected with the maximum occurrence times;

and taking the average value of Euclidean distances corresponding to the dangerous goods to be detected with the largest occurrence frequency as the approach distance.

7. The apparatus for detecting the proximity of a child to a hazardous material according to claim 1, wherein the controller is configured to detect the presence of a child in the RGB image, comprising: and judging whether children exist in the RGB image or not by detecting the key points of the human bones of the RGB image.

8. The apparatus for detecting the proximity of a child to a hazardous material according to claim 1, wherein the controller is configured to detect whether a hazardous material is present in the RGB image, comprising: and judging whether dangerous goods exist in the RGB image or not through a pre-trained dangerous goods detection neural network.

9. A method of detecting the proximity of a child to a hazardous material, comprising:

shooting an RGB image and a depth image corresponding to the RGB image;

detecting whether children and dangerous goods exist in the RGB image;

if children and dangerous goods exist in the RGB image, calculating the approaching distance of the children and the dangerous goods according to the depth image;

judging whether the approach distance is smaller than a safety distance;

and if the approaching distance is less than the safe distance, sending a control signal for alarming.

10. A method of detecting the proximity of a child to a hazardous material, comprising:

receiving an RGB image and a depth image corresponding to the RGB image;

detecting whether children and dangerous goods exist in the RGB image;

if children and dangerous goods exist in the RGB image, calculating the approaching distance of the children and the dangerous goods according to the depth image;

judging whether the approach distance is smaller than a safety distance;

and if the approaching distance is less than the safe distance, sending a control signal for alarming.

Images