CN111428600A

CN111428600A - Smoking detection method, system and device and thermal infrared image processor

Info

Publication number: CN111428600A
Application number: CN202010188793.XA
Authority: CN
Inventors: 张焱; 张华宾; 林铭
Original assignee: Beijing Dushi Technology Co ltd
Current assignee: Beijing Dushi Technology Co ltd
Priority date: 2020-03-17
Filing date: 2020-03-17
Publication date: 2020-07-17

Abstract

The application discloses a smoking detection method, a system and a device and a thermal infrared image processor. Wherein, the method comprises the following steps: acquiring a first image containing a target object acquired by a thermal infrared image acquisition device; detecting a high-temperature pixel region in the first image, wherein the high-temperature pixel region comprises a pixel set of which the pixel value corresponds to the cigarette end temperature; determining a first head image area containing the head of the target object and/or a first hand image area containing the hand of the target object in the first image; and determining whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area.

Description

Smoking detection method, system and device and thermal infrared image processor

Technical Field

The present application relates to the field of smoking detection technologies, and in particular, to a smoking detection method, system, device, and thermal infrared image processor.

Background

Smoking is not only harmful to smokers, but also affects the health of others around. In addition, open fire of smoking easily leads to the fire, has increased the fire hidden danger. At present, for some superstores or large-scale public places, smoke control management is needed, most of the smoke control management needs to be carried out through visible light image acquisition equipment to acquire images of appointed scenes, then smoke heads in the images are identified according to the forms and colors of fire points through an image identification technology, then whether smoking behaviors exist in target objects or not is judged according to identification results, and corresponding early warning processing is carried out. However, the color and the form of the fire point in the image are easily interfered by background information to generate false alarm, for example, some articles are shielded, and a background which is similar to the color of the fire point exists in an acquired scene, and the form of the fire point is small, so that the fire point is difficult to identify in a visible light image, and the accuracy rate of smoking detection is low; the existing thermal infrared high-temperature detection is difficult to distinguish the difference between a high-temperature point and smoking, so that the false alarm rate is high.

Aiming at the technical problems that in the prior art, the false alarm rate is high due to the fact that the cigarette end detection is carried out by identifying the fire point in the visible light image through the image identification technology, the fire point is small in form and not easy to identify, the detection accuracy rate is not high, the existing thermal infrared high-temperature detection is difficult to distinguish the difference between the high-temperature point and the smoking, and an effective solution is not provided at present.

Disclosure of Invention

The present disclosure provides a smoking detection method, system, device and thermal infrared image processor, which at least solve the technical problems in the prior art that the false alarm rate is high due to the influence of interference information easily when the cigarette end is detected by identifying the fire point in a visible light image through an image identification technology, the detection accuracy is not high due to the small form of the fire point which is difficult to identify, and the difference between the high temperature point and smoking is difficult to distinguish in the existing thermal infrared high temperature detection.

According to an aspect of an embodiment of the present disclosure, there is provided a smoking detection method including: acquiring a first image containing a target object acquired by a thermal infrared image acquisition device; detecting a high-temperature pixel region in the first image, wherein the high-temperature pixel region comprises a pixel set of which the pixel value corresponds to the cigarette end temperature; determining a first head image area containing the head of the target object and/or a first hand image area containing the hand of the target object in the first image; and determining whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area.

According to another aspect of the embodiments of the present disclosure, there is provided a thermal infrared image processor, including a high temperature pixel region detection module configured to acquire a first image including a target object acquired by a thermal infrared image acquisition device, and detect a high temperature pixel region in the first image, where the high temperature pixel region includes a set of pixels whose pixel values correspond to a smoke head temperature; an artificial intelligence processing module configured to determine, in the first image, a first head image region containing a head of the target object and/or a first hand image region containing a hand of the target object; and the smoking detection module is connected with the high-temperature pixel area detection module and the artificial intelligence processing module and is configured for judging whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area.

There is also provided, in accordance with another aspect of embodiments of the present disclosure, a smoking detection system, including: a thermal infrared image acquisition device; and the thermal infrared image processor of any one of the above, wherein the thermal infrared image processor is in communication connection with the thermal infrared image acquisition device, and is configured to perform smoking detection on the first image acquired by the thermal infrared image acquisition device.

According to another aspect of the embodiments of the present disclosure, there is also provided a smoking detection device, including: the thermal infrared image acquisition module is used for acquiring a first image which is acquired by thermal infrared image acquisition equipment and contains a target object; the high-temperature pixel region detection module is used for detecting a high-temperature pixel region in the first image, wherein the high-temperature pixel region comprises a pixel set of which the pixel value corresponds to the cigarette end temperature; an image area determination module for determining a first head image area containing the head of the target object and/or a first hand image area containing the hand of the target object in the first image; and the smoking detection module is used for judging whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area.

According to another aspect of the embodiments of the present disclosure, there is also provided a smoking detection device, including: a processor; and a memory coupled to the processor for providing instructions to the processor for processing the following processing steps: acquiring a first image containing a target object acquired by a thermal infrared image acquisition device; detecting a high-temperature pixel area in the first image, wherein the high-temperature pixel area comprises pixels of which the pixel values correspond to the cigarette end temperature; determining a first head image area containing the head of the target object and/or a first hand image area containing the hand of the target object in the first image; and determining whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area.

According to another aspect of the disclosed embodiments, there is also provided a storage medium. The storage medium comprises a stored program, wherein the above described method is performed by a processor when the program is run.

In the embodiment of the invention, a first image which is acquired by a thermal infrared image acquisition device and contains a target object is acquired, and then a high-temperature pixel area is detected in the first image based on the characteristics that the cigarette end is high in temperature, stable and small in occupied pixel value. Wherein the high temperature pixel region comprises a set of pixels having pixel values corresponding to the cigarette end temperature. By the mode, the influence of interference information is not easy to influence, and the cigarette end can be accurately detected. Further, a first head image area including the head of the target object and/or a first hand image area including the hand of the target object are determined in the first image, and finally whether the target object smokes or not is determined according to a positional relationship between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area. Through the mode, the influence of interference information is not easily caused, the cigarette end can be accurately detected, the characteristics that the cigarette end is high in temperature, stable and small in occupied high-temperature pixel area are utilized, the fire point form is effectively prevented from being small and difficult to identify through acquiring the thermal infrared image and carrying out smoking detection, and the smoking detection accuracy is greatly improved. And whether the target object smokes is determined according to the position relation between the head and/or the hand of the target object and the cigarette end, so that the high-temperature point and the smoking are well distinguished, and false detection is eliminated. And then solved the fire point that exists among the prior art in discerning the visible light image through image recognition technology and carry out the cigarette end detection, easily receive the influence of interference information and lead to the false alarm rate higher to the fire point form is less difficult for discerning and lead to the not high rate of accuracy of detection, and the existing thermal infrared high temperature detects, is difficult to distinguish the technical problem of the difference between high temperature point and smoking.

The above and other objects, advantages and features of the present application will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.

Drawings

Some specific embodiments of the present application will be described in detail hereinafter by way of illustration and not limitation with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:

fig. 1 is a flow chart of a smoking detection method according to embodiment 1 of the present disclosure;

fig. 2 is a schematic view of a first image with an added identification graphic according to embodiment 1 of the present disclosure;

fig. 3 is yet another schematic view of a first image with an added identification graphic according to embodiment 1 of the present disclosure;

figure 4 is a schematic diagram of a smoking detection system according to a third aspect of embodiment 1 of the present disclosure;

figure 5 is a schematic view of a smoking detection device according to embodiment 2 of the present disclosure; and

figure 6 is a schematic view of a smoking detection device according to embodiment 3 of the present disclosure.

Detailed Description

It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

In order to make the technical solutions of the present disclosure better understood by those skilled in the art, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only some embodiments of the present disclosure, not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances for describing the embodiments of the disclosure herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.

Example 1

Fig. 1 is a schematic diagram of a smoking detection system according to embodiment 1 of the present application. Referring to fig. 1, a first aspect of embodiment 1 of the present application provides a smoking detection method, including:

s102: acquiring a first image containing a target object acquired by a thermal infrared image acquisition device;

s104: detecting a high-temperature pixel region in the first image, wherein the high-temperature pixel region comprises a pixel set of which the pixel value corresponds to the cigarette end temperature;

s106: determining a first head image area containing the head of the target object and/or a first hand image area containing the hand of the target object in the first image; and

s108: and judging whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area.

As mentioned in the background, smoking is not only harmful to the smoker, but also affects the health of others around. In addition, open fire of smoking easily leads to the fire, has increased the fire hidden danger. At present, for some superstores or large-scale public places, smoke control management is needed, most of the smoke control management needs to be carried out through visible light image acquisition equipment to acquire images of appointed scenes, then smoke heads in the images are identified according to the forms and colors of fire points through an image identification technology, then whether smoking behaviors exist in target objects or not is judged according to identification results, and corresponding early warning processing is carried out. However, the color and the form of the fire point in the image are easily interfered by background information to generate false alarm, for example, some articles are shielded, and a background which is similar to the color of the fire point exists in an acquired scene, and the form of the fire point is small, so that the fire point is difficult to identify in a visible light image, and the accuracy rate of smoking detection is low; the existing thermal infrared high-temperature detection is difficult to distinguish the difference between a high-temperature point and smoking, so that the false alarm rate is high.

Specifically, in view of the above-mentioned problems, referring to fig. 1, the smoking detection method provided by the first aspect of the present embodiment first acquires a first image containing a target object acquired by a thermal infrared image acquisition device. Then, based on the characteristics that the cigarette end temperature is high and stable and the occupied high-temperature pixel value area is small, the high-temperature pixel area is detected in the first image. Wherein the high temperature pixel region comprises a set of pixels having pixel values corresponding to the cigarette end temperature. By the mode, the influence of interference information is not easy to influence, and the cigarette end can be accurately detected.

Further, in the case where the butt is detected, it does not mean that the target object has smoking behavior. Therefore, it is also necessary to determine the correlation (for example, the overlap or contact between the designated part and the cigarette butt) between the detected cigarette butt and the designated part (hand and/or head) of the target object, so as to be able to determine whether the target object has smoking behavior. Specifically, first, a first head image region including the head of the target object and/or a first hand image region including the hand of the target object is specified in the first image. And then judging whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area. Through the mode, the influence of interference information is not easily caused, the cigarette end can be accurately detected, the characteristics that the cigarette end is high in temperature, stable and small in occupied pixel value are utilized, the fire point form is effectively prevented from being small and difficult to identify through acquiring the thermal infrared image and carrying out smoking detection, and the accuracy of smoking detection is greatly improved. And whether the target object smokes is determined according to the position relation between the head and/or the hand of the target object and the cigarette end, so that the high-temperature point and the smoking are well distinguished, and false detection is eliminated. And then solved the fire point that exists among the prior art in discerning the visible light image through image recognition technology and carry out the cigarette end detection, easily receive the influence of interference information and lead to the false alarm rate higher to the fire point form is less difficult for discerning and lead to the not high rate of accuracy of detection, and the existing thermal infrared high temperature detects, is difficult to distinguish the technical problem of the difference between high temperature point and smoking.

Alternatively, the operation of determining whether the target object smokes based on a positional relationship between the high-temperature pixel region in the first image and the first head-image region and/or the first hand-image region includes: determining whether a distance between a high-temperature pixel region in the first image and the first head image region and/or the first hand image region is less than a predetermined threshold; and determining that the target object smokes if it is determined that the distance between the high-temperature pixel region in the first image and the first head image region and/or the first hand image region is less than a predetermined threshold value.

Specifically, in the case where a butt is detected, it does not mean that the target object has smoking behavior. Normally, when a target object smokes, there is an overlapping or contacting relationship between the cigarette butt and the hands and/or head of the target object. In this embodiment, the distance between the hand and/or the head and the cigarette end when a person smokes can be calculated and counted by collecting a large number of smoking images, and then a reasonable threshold value is preset according to the statistical result. Therefore, in determining whether or not the target object smokes, it is determined whether or not a distance between the high-temperature pixel region in the first image and the first head image region and/or the first hand image region is smaller than a predetermined threshold value. And then determining that the target object smokes in a case where it is determined that the distance between the high-temperature pixel region in the first image and the first head image region and/or the first hand image region is less than a predetermined threshold value. Otherwise, the result is no. In this way, whether the target object smokes can be accurately determined.

Optionally, the operation of determining, in the first image, a first head image region containing the head of the target object and a first hand image region containing the hand of the target object includes: generating a second image corresponding to the first image, wherein the second image is suitable for a preset image detection model to detect; detecting a second head image region including the head of the target object and a second hand image region including the hand of the target object in the second image by an image detection model; and determining the first head image area and the first hand image area in the first image according to the position information of the second head image area and the second hand image area in the second image.

Specifically, since the current image detection model generally supports recognition of images with resolutions within a limited range (for example, the resolutions are 512 × 512, 640 × 360, 640 × 480 or others), in order to ensure that the artificial intelligence processing module 230 can effectively detect the target object in the first image, it is necessary to generate a second image corresponding to the first image and suitable for the preset image detection model to detect. Then, a second head image region including the head of the target object and a second hand image region including the hand of the target object are detected in the second image by the image detection model. When the second head image region and the second hand image region are detected, the first head image region and the first hand image region need to be identified in the first image based on the position information of the second head image region and the second hand image region in the second image and the position mapping relationship between the first image and the second image. Therefore, in this way, not only the target object in the first image can be effectively detected, but also the first head image region and the first hand image region can be accurately specified in the first image.

Optionally, the operation of detecting, by the image detection model, a second head image region including the head of the target object and a second hand image region including the hand of the target object in the second image includes: detecting a target object image area containing a target object in the second image through an image detection model; and determining a second head image region and a second hand image region in the target object image region.

Specifically, in order to extract an image region including the target object from the second image, a target object image region including the target object is first detected in the second image by the image detection model, and then a second head image region and a second hand image region are determined in the target object image region.

Optionally, the operation of generating a second image corresponding to the first image comprises at least one of: a resolution conversion operation of converting a resolution of the image into a resolution matching the image detection model; and an image enhancement operation for enhancing detail information in the image.

In particular, since current image detection models generally support recognition of images with a resolution within a limited range, a resolution conversion operation needs to be performed on a first image, that is, the resolution of the first image is converted into a resolution matching the image detection model, so as to generate a second image suitable for detection by the image detection model.

Or, due to the imaging characteristics of the thermal infrared sensor, low resolution and the like, the thermal infrared image is often noisy, and thus the edge information of the object is interfered. Aiming at the problem of high noise, the detailed information in the first image is enhanced by carrying out image enhancement operation on the first image, so that a second image suitable for detecting by an image detection model is generated. Wherein image enhancement operations are performed such as, but not limited to: and performing edge enhancement by using a preset edge sharpening algorithm to enhance the detail information of the object. Common edge sharpening algorithms include, for example, laplacian filter algorithm and sobel filter algorithm.

Alternatively, the first image is first subjected to a resolution conversion operation, that is, the resolution of the first image is converted into a resolution matching the image detection model, and then the image obtained by the resolution conversion operation is subjected to an image enhancement operation, so that detail information in the image obtained by the resolution conversion operation is enhanced, thereby generating a second image suitable for the detection of the image detection model.

Optionally, the operation of determining the first head image area and the first hand image area in the first image according to the position information of the second head image area and the second hand image area in the second image includes: determining the position information of the first head image area and the first hand image area in the first image according to the position information of the second head image area and the second hand image area in the second image and the position mapping relation between the first image and the second image; and determining the first head image area and the first hand image area in the first image according to the position information of the first head image area and the first hand image area in the first image.

Specifically, first, the position information in the first image of the first head image region and the first hand image region is determined by converting the position information in the second image into corresponding position information in the first image according to the position information in the second image of the second head image region and the second hand image region and the position mapping relationship between the first image and the second image, for example, by using a preset coordinate conversion algorithm. The determined position information of the first head image area and the first hand image area in the first image may include x, y, w, h, i.e. x, y coordinates and width and height information of the first head image area and the first hand image area in the first image, for example. Then, the first head image area and the first hand image area are determined in the first image based on the position information of the first head image area and the first hand image area in the first image. In this way, the accuracy of the determined first head image area and the first hand image area is guaranteed.

Optionally, the method further comprises: and sending out early warning information when the target object is judged to smoke. By the method, under the condition that the target object smokes, related workers can be warned in time.

Optionally, the method further comprises: adding an identification graph for indicating the high-temperature pixel area at the position of the high-temperature pixel area; and adding identification graphics respectively indicating the head and the hand of the target object at the positions of the first head image area and the first hand image area.

In practice, a monitoring worker monitors a target object, usually by watching a monitoring video. Therefore, if marks for identifying the target object and the high-temperature pixel region (for example, marking graphics such as a head and a hand of the target object with a color rectangular frame, and marking graphics for marking the high-temperature pixel region) can be added to the video, it is more beneficial for the monitoring staff to observe the monitoring video.

Optionally, the operation of including a plurality of target objects in the first image and detecting a high-temperature pixel region in the first image includes: detecting a plurality of high temperature pixel regions in the first image; an operation of determining a first head image region containing the head of the target object and/or a first hand image region containing the hand of the target object in the first image, comprising: determining a plurality of first head image areas respectively containing heads of a plurality of target objects and/or a plurality of first hand image areas respectively containing hands of a plurality of target objects in the first image; and an operation of determining whether the target object smokes based on a positional relationship between the high-temperature pixel region in the first image and the first head image region and/or the first hand image region, including: whether the plurality of target objects smoke or not is determined according to the position relation between the plurality of high-temperature pixel areas and the plurality of first head image areas and/or the plurality of first hand image areas in the first image. By the method, under the condition that the acquired first image contains a plurality of target objects, smoking detection can be simultaneously performed on the plurality of target objects, and the situations of missing detection, error detection and the like are effectively avoided.

Specifically, fig. 2 exemplarily shows one schematic view of the first image with the identification figure added thereto, and fig. 3 exemplarily shows still another schematic view of the first image with the identification figure added thereto. Referring to fig. 1, 2 and 3, in the present embodiment, the image fusion module 250 adds corresponding identification patterns at the positions of the first head image region and the first hand image region, and also adds corresponding identification patterns at the high-temperature pixel region. As shown in fig. 2 and 3, for example, a color rectangular frame is added to the first head image area and the first hand image area to mark the positions of the head and the hand of the detected target object in the first image. Similarly, the high temperature pixel region may be marked with a rectangular frame of a different color. Therefore, by the mode, the monitoring video with high definition and marks can be provided for monitoring workers, and the monitoring workers can monitor the monitoring video conveniently.

In the smoking detection method shown in fig. 1, referring to fig. 4, a second aspect of embodiment 1 of the present application proposes a thermal infrared image processor 200, which includes a high temperature pixel region detection module 220 configured to acquire a first image including a target object acquired by the thermal infrared image acquisition device 100, and detect a high temperature pixel region in the first image, where the high temperature pixel region includes pixels whose pixel values correspond to a smoke head temperature; an artificial intelligence processing module 230 configured to determine, in the first image, a first head image region containing a head of the target object and/or a first hand image region containing a hand of the target object; and a smoking detection module 240, connected to the high-temperature pixel region detection module 220 and the artificial intelligence processing module 230, configured to determine whether the target object smokes according to a positional relationship between the high-temperature pixel region in the first image and the first head image region and/or the first hand image region.

As mentioned in the background, smoking is not only harmful to the smoker, but also affects the health of others around. In addition, open fire of smoking easily leads to the fire, has increased the fire hidden danger. At present, for some superstores or large-scale public places, smoke control management is needed, most of the smoke control management needs to be carried out through visible light image acquisition equipment to acquire images of appointed scenes, then smoke heads in the images are identified according to the forms and colors of fire points through an image identification technology, then whether smoking behaviors exist in target objects or not is judged according to identification results, and corresponding early warning processing is carried out. However, the color and the form of the fire point in the image are easily interfered by background information to generate false alarm, for example, some articles are blocked, and a background which is similar to the color of the fire point exists in the collected scene, and the form of the fire point is small, so that the fire point is difficult to identify in the visible light image, and the accuracy rate of smoking detection is low.

Specifically, in view of the above-mentioned problems, referring to fig. 4, the thermal infrared image processor 200 according to the second aspect of the present embodiment first acquires a first image including a target object acquired by the thermal infrared image acquisition device 100 using the artificial intelligence processing module 230. Then, the high-temperature pixel region detection module 220 detects the high-temperature pixel region in the first image based on the characteristics that the cigarette end temperature is high, stable and less pixel regions are occupied. Wherein the high temperature pixel region comprises a set of pixels having pixel values corresponding to the cigarette end temperature. By the mode, the influence of interference information is not easy to influence, and the cigarette end can be accurately detected.

Further, in the case where the butt is detected, it does not mean that the target object has smoking behavior. Therefore, it is also necessary to determine the correlation (for example, the overlap or contact between the designated part and the cigarette butt) between the detected cigarette butt and the designated part (hand and/or head) of the target object, so as to be able to determine whether the target object has smoking behavior. Specifically, a first head image region including the head of the target object and/or a first hand image region including the hand of the target object is determined in the first image by the artificial intelligence processing module 230. Finally, the smoking detection module 240 determines whether the target object smokes according to the position relationship between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area. Through the mode, the influence of interference information is not easily caused, the cigarette end can be accurately detected, the characteristics that the cigarette end is high in temperature, stable and small in occupied pixel area are utilized, the fire point form is effectively prevented from being small and difficult to identify through acquiring the thermal infrared image and carrying out smoking detection, and the accuracy of smoking detection is greatly improved. And whether the target object smokes is determined according to the position relation between the head and/or the hand of the target object and the cigarette end, so that the high-temperature point and the smoking are well distinguished, and false detection is eliminated. And then solved the fire point that exists among the prior art in discerning the visible light image through image recognition technology and carry out the cigarette end detection, easily receive the influence of interference information and lead to the false alarm rate higher to the fire point form is less difficult for discerning and lead to the not high rate of accuracy of detection, and the existing thermal infrared high temperature detects, is difficult to distinguish the technical problem of the difference between high temperature point and smoking.

Optionally, the thermal infrared image processor 200 further comprises: the preprocessing module 210 is configured to generate a second image corresponding to the first image, where the second image is suitable for a preset image detection model to perform detection; the artificial intelligence processing module 230 includes a human body part detection unit 231 and a human body part mapping unit 232, wherein the human body part detection unit 231 is connected to the preprocessing module 210 and configured to detect a second head image region including the head of the target object and a second hand image region including the hand of the target object in the second image through an image detection model; the human body part mapping unit 232 is configured to specify the first head image region and the first hand image region in the first image based on the position information of the second head image region and the second hand image region in the second image.

Specifically, referring to fig. 1, the thermal infrared image processor 200 further includes a preprocessing module 210 for generating a second image corresponding to the first image, wherein the second image is suitable for a preset image detection model to perform detection. Since the current image detection model generally supports recognition of images with resolutions within a limited range (for example, the resolutions are 512 × 512, 640 × 360, 640 × 480 or others), in order to ensure that the artificial intelligence processing module 230 can effectively detect the target object in the first image, in this embodiment, the pre-processing module 210 needs to pre-process the acquired first image, so as to generate a second image suitable for the artificial intelligence processing module 230 to detect.

Further, the artificial intelligence processing module 230 includes a human body part detection unit 231 and a human body part mapping unit 232. The human body part detection unit 231 is connected to the preprocessing module 210, and is used for detecting the model, and detecting a second head image region including the head of the target object and a second hand image region including the hand of the target object in the second image. When the second head image region and the second hand image region are detected, the human body part mapping section 232 needs to specify the first head image region and the first hand image region in the first image based on the position information of the second head image region and the second hand image region in the second image. Therefore, in this way, not only the target object in the first image can be effectively detected, but also the first head image region and the first hand image region can be accurately specified in the first image.

Optionally, the operation of determining the first head image area and the first hand image area in the first image according to the position information of the second head image area and the second hand image area in the second image includes: detecting a target object image area containing a target object in the second image through an image detection model; and determining a second head image region and a second hand image region in the target object image region.

Specifically, in order to extract an image region including the target object from the second image, the human body part detection unit 231 first detects a target object image region including the target object in the second image by the image detection model, and then specifies the second head image region and the second hand image region in the target object image region.

Optionally, the pre-processing module 210 comprises at least one of: a resolution conversion unit 211 configured to convert the resolution of the image into a resolution matching the image detection model; and an image enhancement unit 212 configured to enhance detail information in the image.

Specifically, referring to fig. 1, the pre-processing module 210 includes at least one of a resolution conversion unit 211 and an image enhancement unit 212. In a case that the resolution of the first image acquired by the thermal infrared image acquisition device 100 is lower than the resolution of the image that can be detected by the artificial intelligence processing module 230, the resolution conversion unit 211 may be an upsampling unit, configured to perform an upsampling operation on the first image, for example, the upsampling may be performed by using a polyphase filter or a linear filter, so as to complete the improvement from the low resolution to the high resolution. Therefore, the model does not need to be retrained based on the collected thermal infrared image, the resolution of the image is converted into the resolution matched with the image detection model, and then the low-resolution thermal infrared image is effectively detected by utilizing the existing artificial intelligence detection function.

Further, in the case where the resolution of the first image acquired by the thermal infrared image acquisition apparatus 100 is higher than the resolution of the image that can be detected by the artificial intelligence processing module 230, the resolution conversion unit 211 may be a down-sampling unit for performing a down-sampling operation on the first image, thereby converting the resolution of the first image into a resolution matching the image detection model.

Further, due to the imaging characteristics of the thermal infrared sensor, low resolution and the like, the thermal infrared image is often noisy, and thus the edge information of the object is interfered. For the problem of high noise, the embodiment performs denoising by using a preset denoising filtering algorithm through the image enhancement unit 212 to suppress noise in the image without damaging the edge of the object. Common denoising and filtering algorithms include, for example, a bilateral filtering algorithm and a guided filtering algorithm.

Preferably, since the thermal infrared image is imaged according to the surface temperature of the object, and the temperature difference between the object and the background in the actual scene is not very large, the edge details of the object are not obvious in the thermal infrared image. To address this problem, the present embodiment may also perform edge enhancement by using a preset edge sharpening algorithm through the image enhancement unit 212, so as to enhance the detail information of the object. Common edge sharpening algorithms include, for example, laplacian filter algorithm and sobel filter algorithm.

In addition, it should be specifically noted that the image enhancement unit 212 is not limited to include a denoising filter algorithm and an edge sharpening algorithm, and may also include other algorithms capable of enhancing image quality.

Preferably, the preprocessing module 210 of the present embodiment may also first convert the resolution of the first image into a resolution matching the image detection model by the resolution conversion unit 211. Then, the image enhancement unit 212 performs an image enhancement operation on the image output from the resolution conversion unit 211 to suppress noise in the image and enhance detail information in the image, thereby generating a second image suitable for detection by the artificial intelligence processing module 230.

Specifically, the human body part mapping unit 232 first determines the position information of the first head image region and the first hand image region in the first image by converting the position information in the second image into corresponding position information in the first image according to the position information of the second head image region and the second hand image region in the second image and the position mapping relationship between the first image and the second image, for example, by using a preset coordinate conversion algorithm. The determined position information of the first head image area and the first hand image area in the first image may include x, y, w, h, i.e. x, y coordinates and width and height information of the first head image area and the first hand image area in the first image, for example. Then, the first head image area and the first hand image area are determined in the first image based on the position information of the first head image area and the first hand image area in the first image. In this way, the accuracy of the determined first head image area and the first hand image area is guaranteed.

Optionally, the thermal infrared image processor 200 further includes an image fusion module 250, connected to the high temperature pixel region detection module 220 and the artificial intelligence processing module 230, configured to add an identification pattern for indicating the high temperature pixel region at the position of the high temperature pixel region, and add identification patterns for indicating the head and the hand of the target object at the positions of the first head image region and the first hand image region, respectively.

Specifically, referring to fig. 2 and 3, for example, color rectangular frames are added to the first head image area and the first hand image area for marking the positions of the head and the hand of the detected target object in the first image, and then the high-temperature pixel area may also be marked using rectangular frames of different colors. Therefore, by the mode, the monitoring video with high definition and marks can be provided for monitoring workers, and the monitoring workers can monitor the monitoring video conveniently.

A third aspect of embodiment 1 of the present application provides a smoking detection system, including: a thermal infrared image acquisition device 100; and the thermal infrared image processor 200 of any one of the above, wherein the thermal infrared image processor 200 is in communication connection with the thermal infrared image capturing device 100, and is configured to perform smoking detection on the first image captured by the thermal infrared image capturing device 100.

Specifically, referring to fig. 4, a third aspect of embodiment 1 of the present application provides a smoking detection system including a thermal infrared image capture device 100 and a thermal infrared image processor 200 of any one of the above. Thus, a thermal infrared image (corresponding to the first image in fig. 4) may be collected by the thermal infrared image collecting device 100 (e.g., a thermal infrared camera), and then the collected thermal infrared image is transmitted to the thermal infrared image processor 200 by the thermal infrared image collecting device 100. The thermal infrared image processor 200 performs smoking detection on the thermal infrared image after receiving the thermal infrared image acquired by the thermal infrared image acquisition device 100.

Optionally, the smoking detection system further comprises: and a display module 310, communicatively connected to the image fusion module 250 of the thermal infrared image processor 200, for displaying the first image with the added identification pattern.

Specifically, referring to the above, the image fusion module 250 is configured to add the identification pattern at the positions of the high-temperature pixel region, the first head image region, and the first hand image region. Therefore, referring to fig. 4, the smoking detection system further comprises a display module 310 connected to the image fusion module 250 for displaying the first image with the added identification pattern. Thereby, a first image of the mark with the position information and smoking information of the target object can be displayed to the relevant staff.

Optionally, the smoking detection system further comprises a network interface 320 communicatively connected to the image fusion module 250 of the thermal infrared image processor 200 for transmitting the first image with the added identification pattern over a network.

Specifically, referring to fig. 1, the smoking detection system further includes a network interface 320, and the related data of the target object position and the first image added with the identification pattern may be sent to a remote server through the network interface 320 for further data analysis.

Optionally, the smoking detection system further comprises an early warning module 330, communicatively connected to the smoking detection module 240 of the thermal infrared image processor 200, for sending out early warning information in case that the smoking detection module 240 determines that the target object smokes.

Specifically, referring to fig. 1, the smoking detection system further includes an early warning module 330 communicatively coupled to the smoking detection module 240. In the event that the smoking detection module 240 determines that the target object smokes, the early warning module 330 issues early warning information. Therefore, in this way, under the condition that the target object smokes, the relevant staff can be warned in time.

Further, according to a fourth aspect of the present embodiment, there is provided a storage medium. The storage medium comprises a stored program, wherein the method of any of the above is performed by a processor when the program is run.

Example 2

Fig. 5 shows a smoking detection device 500 according to the present embodiment, the device 500 corresponding to the method according to the first aspect of embodiment 1. Referring to fig. 5, the apparatus 500 includes: a thermal infrared image acquisition module 510, configured to acquire a first image including a target object acquired by a thermal infrared image acquisition device; a high temperature pixel region detection module 520, configured to detect a high temperature pixel region in the first image, where the high temperature pixel region includes a pixel region whose pixel value corresponds to the cigarette end temperature; an image area determining module 530 for determining a first head image area containing the head of the target object and/or a first hand image area containing the hand of the target object in the first image; and a smoking detection module 540, configured to determine whether the target object smokes according to a positional relationship between the high-temperature pixel region in the first image and the first head image region and/or the first hand image region.

Optionally, the smoking detection module 540 comprises: a first determination submodule for determining whether a distance between a high-temperature pixel region in the first image and the first head image region and/or the first hand image region is smaller than a predetermined threshold value; and a second judging submodule for judging that the target object smokes under the condition that the distance between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area is judged to be less than a preset threshold value.

Optionally, the image region determining module 530 includes: the image generation submodule is used for generating a second image corresponding to the first image, wherein the second image is suitable for a preset image detection model to detect; a detection sub-module for detecting, by an image detection model, a second head image region containing a head of the target object and a second hand image region containing a hand of the target object in the second image; and an image area determination submodule for determining the first head image area and the first hand image area in the first image based on the position information of the second head image area and the second hand image area in the second image.

Optionally, the detection submodule includes: a detection unit configured to detect a target object image region including a target object in the second image by an image detection model; and an image area determination unit configured to determine a second head image area and a second hand image area in the target object image area.

Optionally, an image generation sub-module comprising at least one of: a resolution converting unit for converting a resolution of the image into a resolution matching the image detection model; and an image enhancement unit for enhancing detail information in the image.

Optionally, the image region determination sub-module comprises: a position information determining unit, which is used for determining the position information of the first head image area and the first hand image area in the first image according to the position information of the second head image area and the second hand image area in the second image and the position mapping relation between the first image and the second image; and an image area determination unit configured to determine the first head image area and the first hand image area in the first image based on position information of the first head image area and the first hand image area in the first image.

Optionally, the smoking detection device 500 further comprises: and the early warning module is used for sending out early warning information under the condition that the target object smokes.

Optionally, the smoking detection device 500 further comprises: and the identification graph adding module is used for adding identification graphs for indicating the high-temperature pixel areas at the positions of the high-temperature pixel areas and adding identification graphs for indicating the heads and the hands of the target objects at the positions of the first head image area and the first hand image area respectively.

Accordingly, the smoking detection apparatus 500 provided in this embodiment first acquires, by the thermal infrared image acquisition module 510, a first image including a target object acquired by the thermal infrared image acquisition device, and then detects, by the high temperature pixel area detection module 520, a high temperature pixel area in the first image based on the characteristics that the cigarette end temperature is high, stable, and the occupied pixel area is small. Wherein the high temperature pixel region comprises a set of pixels having pixel values corresponding to the cigarette end temperature. By the mode, the influence of interference information is not easy to influence, and the cigarette end can be accurately detected. Further, a first head image area including the head of the target object and/or a first hand image area including the hand of the target object are determined in the first image by the image area determination module 530, and finally, whether the target object smokes is determined by the smoking detection module 540 according to the position relationship between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area. Through the mode, the influence of interference information is not easily caused, the cigarette end can be accurately detected, the characteristics that the cigarette end is high in temperature, stable and small in occupied pixel value are utilized, the fire point form is effectively prevented from being small and difficult to identify through acquiring the thermal infrared image and carrying out smoking detection, and the accuracy of smoking detection is greatly improved. And whether the target object smokes is determined according to the position relation between the head and/or the hand of the target object and the cigarette end, so that the high-temperature point and the smoking are well distinguished, and false detection is eliminated. And then solved the fire point that exists among the prior art in discerning the visible light image through image recognition technology and carry out the cigarette end detection, easily receive the influence of interference information and lead to the false alarm rate higher to the fire point form is less difficult for discerning and lead to the not high rate of accuracy of detection, and the existing thermal infrared high temperature detects, is difficult to distinguish the technical problem of the difference between high temperature point and smoking.

Example 3

Fig. 6 shows a smoking detection device 600 according to the present embodiment, the device 600 corresponding to the method according to the first aspect of embodiment 1. Referring to fig. 6, the apparatus 600 includes: a processor 610; and a memory 620 coupled to the processor 610 for providing instructions to the processor 610 to process the following processing steps: acquiring a first image containing a target object acquired by a thermal infrared image acquisition device; detecting a high-temperature pixel region in the first image, wherein the high-temperature pixel region comprises a pixel set of which the pixel value corresponds to the cigarette end temperature; determining a first head image area containing the head of the target object and/or a first hand image area containing the hand of the target object in the first image; and determining whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area.

Optionally, the operation of determining the first head image area and the first hand image area in the first image includes: generating a second image corresponding to the first image, wherein the second image is suitable for a preset image detection model to detect; detecting a second head image region including the head of the target object and a second hand image region including the hand of the target object in the second image by an image detection model; and determining the first head image area and the first hand image area in the first image according to the position information of the second head image area and the second hand image area in the second image.

Optionally, the memory 620 is further configured to provide the processor with instructions to process the following processing steps: and sending out early warning information when the target object is judged to smoke.

Optionally, the memory 620 is further configured to provide the processor with instructions to process the following processing steps: and adding identification graphics for indicating the high-temperature pixel region at the position of the high-temperature pixel region, and adding identification graphics for indicating the head and the hand of the target object at the positions of the first head image region and the first hand image region respectively.

Thus, according to the present embodiment, the smoking detection apparatus 600 provided in the present embodiment first acquires a first image including a target object acquired by a thermal infrared image acquisition device, and then detects a high-temperature pixel region in the first image based on the characteristics that the cigarette end temperature is high, stable, and the occupied pixel value is small. Wherein the high temperature pixel region comprises pixels with pixel values corresponding to the cigarette end temperature. By the mode, the influence of interference information is not easy to influence, and the cigarette end can be accurately detected. Further, a first head image area including the head of the target object and/or a first hand image area including the hand of the target object are determined in the first image, and finally whether the target object smokes or not is determined according to a positional relationship between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area. Through the mode, the influence of interference information is not easily caused, the cigarette end can be accurately detected, the characteristics that the cigarette end is high in temperature, stable and small in occupied pixel value are utilized, the fire point form is effectively prevented from being small and difficult to identify through acquiring the thermal infrared image and carrying out smoking detection, and the accuracy of smoking detection is greatly improved. And whether the target object smokes is determined according to the position relation between the head and/or the hand of the target object and the cigarette end, so that the high-temperature point and the smoking are well distinguished, and false detection is eliminated. And then solved the fire point that exists among the prior art in discerning the visible light image through image recognition technology and carry out the cigarette end detection, easily receive the influence of interference information and lead to the false alarm rate higher to the fire point form is less difficult for discerning and lead to the not high rate of accuracy of detection, and the existing thermal infrared high temperature detects, is difficult to distinguish the technical problem of the difference between high temperature point and smoking.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

The relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise. Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

Spatially relative terms, such as "above … …," "above … …," "above … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial relationship to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" can include both an orientation of "above … …" and "below … …". (the device may also be oriented 90 degrees or at other orientations in different ways), and the spatially relative descriptors used herein interpreted accordingly.

In the description of the present disclosure, it is to be understood that the orientation or positional relationship indicated by the directional terms such as "front, rear, upper, lower, left, right", "lateral, vertical, horizontal" and "top, bottom", etc., are generally based on the orientation or positional relationship shown in the drawings, and are presented only for the convenience of describing and simplifying the disclosure, and in the absence of a contrary indication, these directional terms are not intended to indicate and imply that the device or element being referred to must have a particular orientation or be constructed and operated in a particular orientation, and therefore, should not be taken as limiting the scope of the disclosure; the terms "inner and outer" refer to the inner and outer relative to the profile of the respective component itself.

The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of smoking detection, comprising:

acquiring a first image containing a target object acquired by a thermal infrared image acquisition device;

detecting a high-temperature pixel region in the first image, wherein the high-temperature pixel region comprises a pixel set of which the pixel value corresponds to the cigarette end temperature;

determining a first head image area containing the head of the target object and/or a first hand image area containing the hand of the target object in the first image; and

and judging whether the target object smokes according to the position relation between the high-temperature pixel area in the first image and the first head image area and/or the first hand image area.

2. The method according to claim 1, wherein the operation of determining whether the target object smokes based on a positional relationship between a high-temperature pixel region in the first image and the first head-image region and/or the first hand-image region includes:

determining whether a distance between a high-temperature pixel region in the first image and the first head image region and/or the first hand image region is less than a predetermined threshold; and

determining that the target object smokes if it is determined that a distance between a high-temperature pixel region in the first image and the first head image region and/or the first hand image region is less than a predetermined threshold value.

3. The method of claim 1, wherein the operation of determining in the first image a first head-image region containing the head of the target object and a first hand-image region containing the hands of the target object comprises:

generating a second image corresponding to the first image, wherein the second image is suitable for a preset image detection model to detect;

detecting, by the image detection model, a second head image region including a head of the target object and a second hand image region including a hand of the target object in the second image; and

and determining the first head image area and the first hand image area in the first image according to the position information of the second head image area and the second hand image area in the second image and the position mapping relation between the first image and the second image.

4. The method according to claim 3, wherein the operation of detecting, by the image detection model, a second head image region including the head of the target object and a second hand image region including the hand of the target object in the second image includes:

detecting a target object image area containing the target object in the second image through the image detection model; and

determining the second head image region and the second hand image region in the target object image region.

5. The method of claim 3, wherein the operation of generating a second image corresponding to the first image comprises at least one of:

a resolution conversion operation of converting a resolution of an image into a resolution matching the image detection model; and

and the image enhancement operation is used for enhancing the detail information in the image.

6. The method according to claim 3, wherein the operation of determining the first head image region and the first hand image region in the first image based on the position information of the second head image region and the second hand image region in the second image comprises:

determining the position information of the first head image area and the first hand image area in the first image according to the position information of the second head image area and the second hand image area in the second image and the position mapping relation between the first image and the second image; and

the first head image area and the first hand image area are specified in the first image based on position information of the first head image area and the first hand image area in the first image.

7. The method of claim 1, further comprising: and sending out early warning information under the condition that the target object smokes.

8. The method of claim 1, further comprising: adding an identification graph for indicating the high-temperature pixel region at the position of the high-temperature pixel region; and

adding identification graphics respectively indicating the head and the hand of the target object at the positions of the first head image area and the first hand image area.

9. The method of claim 1, wherein the first image includes a plurality of target objects, and wherein the first image includes a plurality of target objects

An operation of detecting a high temperature pixel region in the first image, comprising: detecting a plurality of high temperature pixel regions in the first image;

an operation of determining in the first image a first head image region containing the head of the target object and/or a first hand image region containing the hand of the target object, comprising: determining, in the first image, a plurality of first head image areas respectively containing heads of the plurality of target objects and/or a plurality of first hand image areas respectively containing hands of the plurality of target objects; and

an operation of determining whether the target object smokes based on a positional relationship between a high-temperature pixel region in the first image and the first head image region and/or the first hand image region, including: and determining whether the target objects smoke or not according to the position relations between the high-temperature pixel areas and the first head image areas and/or the first hand image areas in the first image.

10. A storage medium comprising a stored program, wherein the method of any one of claims 1 to 9 is performed by a processor when the program is run.

11. A thermal infrared image processor (200), comprising:

a high temperature pixel region detection module (220) configured to acquire a first image containing a target object acquired by a thermal infrared image acquisition device (100), and detect a high temperature pixel region in the first image, wherein the high temperature pixel region comprises a set of pixels having pixel values corresponding to a smoke head temperature;

an artificial intelligence processing module (230) configured for determining in the first image a first head image region containing the head of the target object and/or a first hand image region containing the hand of the target object; and

a smoking detection module (240), connected to the high-temperature pixel region detection module (220) and the artificial intelligence processing module (230), configured to determine whether the target object smokes according to a positional relationship between the high-temperature pixel region in the first image and the first head image region and/or the first hand image region.

12. A smoking detection system, comprising: a thermal infrared image acquisition device (100); and a thermal infrared image processor (200) as set forth in claim 11 wherein

The thermal infrared image processor (200) is in communication connection with the thermal infrared image acquisition equipment (100) and is used for carrying out smoking detection on a first image acquired by the thermal infrared image acquisition equipment (100).

13. A smoking detection device, comprising:

a thermal infrared image acquisition module (510) for acquiring a first image containing a target object acquired by a thermal infrared image acquisition device;

a high temperature pixel region detection module (520) for detecting a high temperature pixel region in the first image, wherein the high temperature pixel region comprises a set of pixels whose pixel values correspond to a cigarette butt temperature;

an image region determination module (530) for determining in the first image a first head image region containing the head of the target object and/or a first hand image region containing the hand of the target object; and

a smoking detection module (540) for determining whether the target object smokes according to a positional relationship between a high-temperature pixel region in the first image and the first head image region and/or the first hand image region.

14. A smoking detection device, comprising:

a processor (610); and

a memory (620) coupled to the processor (610) for providing instructions to the processor (610) for processing the following processing steps:

detecting a high-temperature pixel region in the first image, wherein the high-temperature pixel region comprises pixels of which the pixel values correspond to the cigarette end temperature;