CN111144312B - Image processing method, device, equipment and medium - Google Patents
Image processing method, device, equipment and medium Download PDFInfo
- Publication number
- CN111144312B CN111144312B CN201911374654.XA CN201911374654A CN111144312B CN 111144312 B CN111144312 B CN 111144312B CN 201911374654 A CN201911374654 A CN 201911374654A CN 111144312 B CN111144312 B CN 111144312B
- Authority
- CN
- China
- Prior art keywords
- image
- foreground image
- energy difference
- processing
- motion foreground
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 25
- 238000012545 processing Methods 0.000 claims abstract description 82
- 239000000779 smoke Substances 0.000 claims abstract description 65
- 238000000034 method Methods 0.000 claims abstract description 36
- 230000008569 process Effects 0.000 claims description 13
- 238000004422 calculation algorithm Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 9
- 238000012935 Averaging Methods 0.000 claims description 7
- 239000000835 fiber Substances 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 239000000758 substrate Substances 0.000 claims 1
- 230000003287 optical effect Effects 0.000 description 15
- 238000001514 detection method Methods 0.000 description 11
- 239000010902 straw Substances 0.000 description 10
- 238000012544 monitoring process Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012806 monitoring device Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000011410 subtraction method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/44—Event detection
Abstract
The embodiment of the invention discloses an image processing method, an image processing device, image processing equipment and an image processing medium. The method comprises the following steps: acquiring a motion foreground image in a target picture; traversing the moving foreground image by utilizing a sliding window to obtain a plurality of images to be identified, wherein the size of the sliding window is determined based on the size of the moving foreground image; and carrying out smoke recognition on the plurality of images to be recognized. According to the image processing method, the device, the equipment and the medium provided by the embodiment of the invention, the smoke recognition precision can be improved.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to an image processing method, apparatus, device, and medium.
Background
With the progressive development of image processing technology, smoke recognition can be performed on images. At present, a smoke recognition scheme needs to train a model based on a large number of smoke data sets to realize smoke detection. However, because the smoke detail features are less, feature vectors are difficult to extract, and the whole image is directly identified, so that the smoke identification accuracy is low. There is therefore a need for a solution that can improve the accuracy of smoke recognition.
Disclosure of Invention
The embodiment of the invention provides an image processing method, an image processing device, image processing equipment and a medium, which can improve the smoke recognition precision.
In a first aspect, there is provided an image processing method including: acquiring a motion foreground image in a target picture; traversing the moving foreground image by utilizing a sliding window to obtain a plurality of images to be identified, wherein the size of the sliding window is determined based on the size of the moving foreground image; and carrying out smoke recognition on the plurality of images to be recognized.
According to the image processing method provided by the embodiment of the invention, after the motion foreground image is extracted from the target image, the motion foreground image can be traversed by utilizing the sliding window to obtain a plurality of images to be identified, and smoke detection is carried out on the images to be identified. The size of the sliding window is determined based on the size of the moving foreground image, and the size of the image to be identified is the same as the size of the sliding window, so that the size of the image to be identified can be adaptively adjusted according to the size of the moving foreground image, and the identification precision is improved.
In an alternative embodiment, the ratio of the size of the sliding window to the size of the moving foreground image is equal to a preset size ratio.
According to the embodiment, the size of the sliding window can be dynamically adjusted according to the size of the moving foreground image, and recognition accuracy degradation caused by overlarge sliding window and overlarge sliding window is prevented.
In an alternative embodiment, the method further comprises: acquiring a video stream from a video acquisition device, wherein the video stream comprises multiple frames of pictures; and sequentially taking the multi-frame pictures as target pictures, and carrying out smoke recognition on the images to be recognized corresponding to the target pictures.
Through the embodiment, the smoke identification of the video stream can be realized, and the accuracy and the speed of the smoke identification of the video stream are ensured.
In an alternative embodiment, acquiring the motion foreground image in the target picture includes: acquiring a video stream from a video acquisition device, wherein the video stream comprises multiple frames of pictures; carrying out a background model on the multi-frame pictures to obtain respective corresponding background images of the multi-frame pictures; extracting background images of the target pictures from the background images corresponding to the multi-frame pictures respectively; and extracting a motion foreground image from the target picture by utilizing the background image of the target picture.
The background image of the target picture can be generated by combining the background characteristics of the multi-frame pictures in a background modeling mode of the multi-frame pictures in the video stream, so that the accuracy of the background image is higher. Therefore, the background image of the target picture is obtained by using the background modeling mode, and the anti-interference capability of the image processing scheme for obtaining the moving target area in the target picture by using the background image is high. The method has higher smoke recognition accuracy for complex outdoor scenes, large-view-field video scenes and the like.
In an alternative embodiment, acquiring a motion foreground image in a target picture includes: respectively carrying out gray processing on the motion foreground image extracted from the target picture and the background image of the target picture to obtain a motion foreground image after gray processing and a background image after gray processing; performing energy difference processing on the motion foreground image after gray level processing by using an energy difference algorithm and the background image after gray level processing to obtain the motion foreground image after energy difference processing, so as to traverse the motion foreground image after energy difference processing by using a sliding window; each pixel point of the motion foreground image after the energy difference value processing corresponds to the energy difference value of each pixel point.
By the method, before the traversing operation is executed, the motion foreground can be processed by using an energy difference algorithm and the background image of the target picture, so that the recognition accuracy is improved.
In an alternative embodiment, the energy difference value of the pixel point is used to reflect the change of the gray gradient value of the pixel point between the motion foreground image and the background image.
In an alternative embodiment, the energy difference D of each pixel is calculated e Energy difference algorithm f of (2) e The calculation formula of the (a) comprises:
wherein f e (,) represents an energy difference function, B e For each pixel point at backgroundGray gradient value, C, on an image e For each pixel point, the gray gradient value, P, on the motion foreground image B For each pixel point, the gray value, P, on the background image C For the gray value of each pixel point on the motion foreground image,gray gradient values along the first direction on the background image for each pixel,/->Gray gradient values along a first direction on the motion foreground image for each pixel point, +.>Gray gradient values along the second direction on the background image for each pixel,/->Gray gradient values along the second direction on the motion foreground image for each pixel point.
In an alternative embodiment, smoke recognition is performed on a plurality of images to be recognized, including: for each frame image of the plurality of images to be identified, performing the following operations: averaging the energy difference values of all pixel points of each frame of image to obtain the energy difference value of each frame of image; if the energy difference value of each frame of image is larger than a preset energy difference value threshold value, determining that smoke exists in the area where each frame of image is located.
Through the average processing in the embodiment, whether the frame image has smoke can be comprehensively judged according to the energy difference values of all the pixel points in each frame image, so that the identification accuracy is ensured.
In an alternative embodiment, the calculation formula of the averaging process includes:
where avg represents the energy difference value of each frame image, l represents the length of each frame image, and h represents the width of each frame image.
In an alternative embodiment, the target picture is monitored for suburban areas where the fiber optic cable is deployed or farmland where the fiber optic cable is deployed.
The target image obtained by monitoring the suburban area with the optical cable or the farmland with the optical cable is processed, so that straw burning behavior can be found in time, and the safety of the optical cable is ensured.
In a second aspect, there is provided an image processing apparatus comprising: the foreground image acquisition module is used for acquiring a motion foreground image in the target picture; the window traversing module is used for traversing the moving foreground image by utilizing the sliding window to obtain a plurality of images to be identified, wherein the size of the sliding window is determined based on the size of the moving foreground image; and the smoke recognition module is used for carrying out smoke recognition on the plurality of images to be recognized.
According to the image processing device provided by the embodiment of the invention, after the motion foreground image is extracted from the target image, the motion foreground image can be traversed by utilizing the sliding window to obtain a plurality of images to be identified, and smoke detection is carried out on the images to be identified. The size of the sliding window is determined based on the size of the moving foreground image, and the size of the image to be identified is the same as the size of the sliding window, so that the size of the image to be identified can be adaptively adjusted according to the size of the moving foreground image, and the identification precision is improved.
In a third aspect, there is provided an image processing apparatus comprising: a memory for storing a program;
a processor for executing a program stored in a memory to perform the image processing method of the first aspect or any optional implementation of the first aspect.
In a fourth aspect, there is provided a computer storage medium having stored thereon computer program instructions which, when executed by a processor, implement the image processing method provided in the first aspect or any of the alternative embodiments of the first aspect.
According to the image processing method, the device, the equipment and the medium, after the motion foreground image is extracted from the target image, the motion foreground image can be traversed by utilizing the sliding window to obtain a plurality of images to be identified, and smoke detection is carried out on the images to be identified. The size of the sliding window is determined based on the size of the moving foreground image, and the size of the image to be identified is the same as the size of the sliding window, so that the size of the image to be identified can be adaptively adjusted according to the size of the moving foreground image, and the identification precision is improved.
Drawings
In order to more clearly illustrate the technical solution of the embodiments of the present invention, the drawings that are needed to be used in the embodiments of the present invention will be briefly described, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of an image processing method provided in an embodiment of the present invention;
fig. 2 is a schematic structural view of an image processing apparatus according to an embodiment of the present invention;
fig. 3 is a block diagram of an exemplary hardware architecture of an image processing apparatus according to an embodiment of the present invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely configured to illustrate the invention and are not configured to limit the invention. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the invention by showing examples of the invention.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
The embodiment of the invention provides an image processing scheme which is used for various scenes for identifying smoke. For example, the method can be used in a specific scene of carrying out straw smoke recognition on monitoring images of village and town areas to ensure the safety of the optical cable. Since straw burning is one of the common damage causes of optical cables deployed in suburban areas and farms, for example, straw burning may cause damage to the optical cables due to exposure, etc. Straw smoke recognition is carried out on monitoring images of rural areas, so that straw burning behaviors can be found in time, countermeasures are formulated in time, and the safety of the optical cable is guaranteed. Accordingly, in a specific scene where the monitoring image of the village and town area is subjected to straw smoke recognition to ensure the safety of the optical cable, the smoke recognition can be performed on a picture which can acquire an image of a suburban area where the optical cable is deployed or an image of a farmland where the optical cable is deployed.
According to the image processing scheme provided by the embodiment of the invention, after the motion foreground image is extracted from the target image, the motion foreground image can be traversed by utilizing the sliding window to obtain a plurality of images to be identified, and smoke detection is carried out on the images to be identified. The size of the sliding window is determined based on the size of the moving foreground image, and the size of the image to be identified is the same as the size of the sliding window, so that the size of the image to be identified can be adaptively adjusted according to the size of the moving foreground image, and the identification precision is improved.
For a better understanding of the present invention, an image processing method, apparatus, device, and medium according to embodiments of the present invention will be described in detail below with reference to the accompanying drawings, and it should be noted that these embodiments are not intended to limit the scope of the present disclosure.
Fig. 1 is a schematic flow chart of an image processing method according to an embodiment of the present invention. As shown in fig. 1, the image processing method 100 in the present embodiment may include the following S110 to S130.
S110, acquiring a motion foreground image in the target picture.
First, for the target picture, the target picture may be a single frame picture in the video stream acquired by the video acquisition device, or may be one of multiple pictures continuously acquired by the image acquisition device. For example, in a specific scenario where straw smoke recognition is performed on a monitored image of a rural area to ensure the safety of an optical cable, the target picture may be obtained by monitoring, by a monitoring device, a suburban area where the optical cable is deployed or a farmland where the optical cable is deployed. Alternatively, the monitoring device may be a high-altitude wide-angle lens installed in a town area. For example, the monitoring device may be installed in suburban areas or agricultural fields.
Second, for a motion foreground image, the motion foreground image is relative to the background image. Specifically, the image of the subject can be divided into a motion foreground image and a background image according to the motion state of the subject. In the embodiment of the invention, the picture which is kept unchanged for a long time in the image acquisition process is called a background image, and the moving picture is called a moving foreground image. For example, in monitoring villages and towns, images of buildings, the earth, farmlands, and the like in the photographed pictures may be regarded as background images in the photographed pictures, and passing vehicles, pedestrians, and the like may be regarded as moving foreground images in the photographed pictures. In the embodiment of the invention, the smoke is considered to have a shielding effect on the background, and the smoke is not static but continuously flutters, so that the smoke in the shot picture can be identified as a moving foreground image in the shot picture.
In one embodiment, a background image of the target picture may be acquired first, and then a motion foreground image in the target picture may be acquired according to the background image. Accordingly, S110 specifically includes the following four steps.
The first step is to acquire a video stream from a video acquisition device. Wherein the video stream comprises a plurality of frames of pictures. The video stream may be, for example, one or more pieces of video captured by a video capture device. The video capture device may be a monitor with camera functionality, for example. The video capturing device may be other devices, for example, a digital camera having an imaging function, an aerial camera, or the like, which is not limited thereto.
And a second step, carrying out a background model on the multi-frame pictures contained in the video stream to obtain respective corresponding background images of the multi-frame pictures. I.e. by background modeling, each frame of image corresponds to a background image. The background modeling can be a single-Gaussian model or a mixed Gaussian model, namely, the background modeling can be carried out on multi-frame pictures in the video stream by utilizing a single-Gaussian function or a mixed Gaussian function. Accordingly, a background image obtained using a single gaussian function or a mixed gaussian function may be referred to as a gaussian background image.
And thirdly, extracting the background image of the target picture from the background images corresponding to the multi-frame pictures. For example, if the video stream includes 5 frames of pictures, each of the 5 frames of pictures corresponds to one background image, and if the 3 rd frame of pictures is a target picture, the background image corresponding to the third frame of pictures may be used as the background image of the target picture.
And a fourth step of extracting a motion foreground image from the target picture by utilizing the background image of the target picture. As an example, after the background image of the target picture is acquired, the motion foreground image may be extracted from the target picture by a background subtraction method.
Specifically, the fourth step may include the following three sub-steps, and the detailed description of the three sub-steps is as follows.
In the first substep, after the background image of the target picture is acquired, a motion foreground mask of the target picture may be constructed first. The motion foreground mask may be, for example, a binarized picture of the same size as the target picture. The gray value of the pixel point in the binarized picture is 0 or 1. The region where the gray value of the pixel point is 0 corresponds to the region where the background image is located on the target image. The region with the gray value of 1 of the pixel point corresponds to the region where the motion foreground image is located on the target image.
And a second substep, carrying out operation on a moving foreground mask of the target picture and the target picture to obtain a first picture to be processed. The first picture to be processed eliminates the image features of the background area, and only retains the image features of the motion foreground image. For example, the size of the first to-be-processed picture may be the same as the target image, and the pixel value of the region where the motion foreground image is located in the first to-be-processed picture is the color feature value of the pixel, for example, if the target image is a color picture, the color feature value of each pixel in the region where the motion foreground image is located may be an RGB value. For another example, if the target picture is a gray-scale picture, the color feature value of each pixel point in the region where the motion foreground image is located may be a gray-scale value between 0 and 255. The gray value of the pixel point in the area where the background image is located in the first picture to be processed is 0.
And a third sub-step of acquiring a motion foreground image from the first picture to be processed. In one embodiment, since the region of the motion foreground image may be an irregular region, the region of the motion foreground image may be processed into a regular region for convenience of the subsequent traversing step. For example, an image of a minimum rectangular area in the target picture, which can include an area where the complete motion foreground exists, may be referred to as a motion foreground image of the target picture. For example, a minimum rectangular area of length L and width H may be provided. In the embodiment of the present invention, the pixel row direction may be referred to as a length direction, and the pixel column direction may be referred to as a width direction.
The background image of the target picture can be generated by combining the background characteristics of the multi-frame pictures in a background modeling mode of the multi-frame pictures in the video stream, so that the accuracy of the background image is higher. Therefore, the background image of the target picture is obtained by using the background modeling mode, and the anti-interference capability of the image processing scheme for obtaining the moving target area in the target picture by using the background image is high. The method has higher smoke recognition accuracy for complex outdoor scenes, large-view-field video scenes and the like.
In addition, compared with the technical scheme for realizing smoke detection based on a large number of smoke data set training models, a large number of picture data samples do not need to be prepared, so that the detection cost is reduced, and the workload of preparation work is greatly reduced. In addition, the smoke recognition rate of the image processing method provided by the embodiment of the invention can reach the millisecond level, and the recognition speed is improved. Especially, aiming at a straw smoke detection scene, the method can quickly identify and alarm, and the damage of the straw to the optical cable is avoided to the greatest extent.
In some embodiments, to improve the calculation accuracy, the motion foreground may be processed by using an energy difference algorithm and a background image of the target picture before performing the traversal operation. Specifically, S110 specifically includes:
and a first step of respectively carrying out gray processing on the motion foreground image extracted from the target picture and the background image of the target picture to obtain the motion foreground image after gray processing and the background image after gray processing. For example, in order to further improve the recognition accuracy, the background image of the target picture may be a gaussian background image, and specific content of the gaussian background image may be referred to the related description of the foregoing embodiments of the present invention, which is not described herein.
The values of the pixel points on the motion foreground image after gray level processing and the background image after gray level processing are gray level values of the pixel points.
If the target image acquired from the image capturing device is a grayscale image, the motion foreground image and the background image do not need to be subjected to grayscale processing.
And a second step of processing the motion foreground image after gray level processing by using an energy difference algorithm and the background image after gray level processing to obtain the motion foreground image after energy difference processing, so as to traverse the motion foreground image after energy difference processing by using a sliding window.
Each pixel point of the motion foreground image after the energy difference value processing corresponds to the energy difference value of each pixel point. Specifically, the energy difference value of each pixel point is used for reflecting the change condition of the gray gradient value of the pixel point between the motion foreground image and the background image.
As a specific example, the energy difference D of each pixel point is calculated e Energy difference algorithm f of (2) e The calculation formula of the (a) is shown as a formula (1):
wherein f e (,) represents an energy difference function, B e For each pixel point, the gray gradient value C on the background image e For each pixel point, the gray gradient value, P, on the motion foreground image B For each pixel point, the gray value, P, on the background image C For the gray value of each pixel point on the motion foreground image,gray gradient values along the first direction on the background image for each pixel,/->Gray gradient values along a first direction on the motion foreground image for each pixel point, +.>Gray gradient values along the second direction on the background image for each pixel,/->Gray gradient values along the second direction on the motion foreground image for each pixel point.
The first direction and the second direction are different, for example, the first direction may be a row direction of the pixel arrangement, and the second direction may be a column direction of the pixel arrangement.
It should be noted that, through the formula (1), the image characteristics of the motion foreground image on the frequency domain can be reflected, so that the subsequent calculation and processing are convenient.
And S120, traversing the motion foreground image by utilizing the sliding window to obtain a plurality of images to be identified. Specifically, during traversal, when the sliding window is at the position a, the local motion foreground image in the sliding window at the position a is extracted as an image to be identified, then the sliding window slides from the position a to the position B with a preset sliding step length, and the local motion foreground image in the sliding window at the position B is extracted as a next image to be identified. Illustratively, the sliding window a may traverse the motion foreground image line by line starting from a corner of the motion foreground image, and ensure that all pixels of the motion foreground image are traversed at least up to 1 time when the traversing is completed. The sliding step length can be set according to specific working situations and working requirements, and is not limited. However, the sliding step in the pixel row direction is not longer than the length of the sliding window in the row direction, and the sliding step in the pixel column direction is not longer than the length of the sliding window in the column direction.
Wherein the size of the sliding window is determined based on the size of the motion foreground image. As an example, the ratio of the size of the sliding window to the size of the moving foreground image is equal to a preset size ratio. I.e. the sliding window is scaled equally according to the motion foreground image. The predetermined dimensional ratio is less than 1. For example, if the preset size ratio is 1/a, the motion foreground image has a length L and a width H. The length of the sliding window l=l/a and the width of the sliding window h=h/a. Wherein a >1. The appropriate size ratio may be selected according to the recognition accuracy requirement, for example, the higher the recognition accuracy requirement, the larger the value of a.
S130, carrying out smoke recognition on the images to be recognized. Specifically, whether smoke exists in each image to be identified can be identified.
According to the image processing method provided by the embodiment of the invention, after the motion foreground image is extracted from the target image, the motion foreground image can be traversed by utilizing the sliding window to obtain a plurality of images to be identified, and smoke detection is carried out on the images to be identified. The size of the sliding window is determined based on the size of the moving foreground image, and the size of the image to be identified is the same as the size of the sliding window, so that the size of the image to be identified can be adaptively adjusted according to the size of the moving foreground image, and the identification precision is improved.
In addition, the scheme of directly recognizing the whole image can cause that interference noise is judged as a smoke area, and recognition accuracy is affected. By dynamically adjusting the size of the sliding window according to the size of the moving foreground image, the influence of interference noise on the identification result can be eliminated. In practical tests, if the size of the sliding window is directly set before detection, if the acquired motion foreground image is too small, the sliding window is too large relative to the motion foreground image, which may cause missing recognition. If the acquired motion foreground image is too large, the sliding window is too small relative to the motion foreground image, and false identification may occur, so that the target to be detected cannot be identified. Since the size of the moving foreground image acquired in the video acquisition process is randomly changed, it is difficult to select a sliding window size value suitable for all moving foreground images. The image processing method provided by the invention solves the problem that the sliding window is difficult to select the proper value, and reduces the probability of false recognition and missing recognition.
In some embodiments, if the sliding window is a traversal of the motion foreground image after the energy difference processing, the following first and second steps may be performed for each frame image of the plurality of images to be identified in S130.
The first step is to average the energy difference value of all pixel points of each frame of image in the plurality of images to be identified to obtain the energy difference value of each frame of image.
Wherein the calculation formula of the averaging process includes the following formula (2):
where avg represents the energy difference value of each frame image, l represents the length of each frame image, and h represents the width of each frame image.
And a second step of determining that smoke exists in the area where each frame of image is located if the energy difference value of each frame of image is larger than a preset energy difference value threshold Th. The preset energy difference threshold Th may be set in a specific working scenario and a specific working requirement, which will not be described herein.
Through the two steps, whether the frame image has smoke or not can be comprehensively judged according to the energy difference values of all pixel points in each frame image through average processing, and the identification accuracy is ensured.
In addition, there is a case where the energy difference value of each frame image is equal to or less than the preset energy difference value threshold Th. Accordingly, S130 further includes a third step.
And thirdly, if the energy difference value of each frame of image is smaller than or equal to a preset energy difference value threshold Th, determining that the area where the frame of image is positioned is free of smoke.
In addition, after judging whether the region where each frame image is located has smoke, the frame image may be marked. Specifically, if the area of the frame image is not in smoke, the area is marked as a smoke-free area, and if the area of the frame image is in smoke, the area is marked as a smoke area.
In one embodiment, if smoke recognition is required for the entire video stream of the image acquisition device, the image processing method 100 includes: a video stream is acquired from a video acquisition device. Wherein the video stream comprises a plurality of frames of pictures. And taking the multi-frame pictures of the video stream as target pictures in sequence, and performing smoke recognition on the images to be recognized corresponding to the target pictures by executing S110 to S130.
An apparatus according to an embodiment of the present invention will be described in detail below with reference to the accompanying drawings.
Based on the same inventive concept, an embodiment of the present invention provides an image processing apparatus. Fig. 2 is a schematic diagram showing the structure of an image processing apparatus according to another embodiment of the present invention. As shown in fig. 2, the image processing apparatus 200 includes a foreground image acquisition module 210, a window traversal module 220, and a smoke recognition module 230.
The foreground image acquisition module 210 is configured to acquire a motion foreground image in a target picture.
The window traversing module 220 is configured to traverse the motion foreground image by using the sliding window, so as to obtain a plurality of images to be identified. Wherein the size of the sliding window is determined based on the size of the motion foreground image.
The smoke recognition module 230 is configured to perform smoke recognition on a plurality of images to be recognized.
In some examples, the ratio of the size of the sliding window to the size of the motion foreground image is equal to a preset size ratio.
In some examples, the image processing apparatus 200 further comprises a video stream acquisition module.
The video stream acquisition module is used for acquiring a video stream from the video acquisition device, wherein the video stream comprises multiple frames of pictures.
Correspondingly, the smoke recognition module 230 is further configured to sequentially use the multiple frames of pictures as target pictures, and perform smoke recognition on images to be recognized corresponding to the target pictures.
In some examples, the foreground image acquisition module 210 is specifically configured to: acquiring a video stream from a video acquisition device, wherein the video stream comprises multiple frames of pictures; carrying out a background model on the multi-frame pictures to obtain respective corresponding background images of the multi-frame pictures; extracting background images of the target pictures from the background images corresponding to the multi-frame pictures respectively; and extracting a motion foreground image from the target picture by utilizing the background image of the target picture.
In some examples, the foreground image acquisition module 210 is specifically configured to: respectively carrying out gray processing on the motion foreground image extracted from the target picture and the background image of the target picture to obtain a motion foreground image after gray processing and a background image after gray processing; performing energy difference processing on the motion foreground image after gray level processing by using an energy difference algorithm and the background image after gray level processing to obtain the motion foreground image after energy difference processing, so as to traverse the motion foreground image after energy difference processing by using a sliding window; each pixel point of the motion foreground image after the energy difference value processing corresponds to the energy difference value of each pixel point.
In some examples, the energy difference value of the pixel point is used to reflect the change of the gray gradient value of the pixel point between the motion foreground image and the background image.
In some examples, the energy difference D for each pixel is calculated e Energy difference algorithm f of (2) e The calculation formula of the (d) may be formula (1). For details of formula (1), reference is made to the description related to formula (1) in the above embodiments of the present invention, and the description is omitted here.
In some examples, smoke identification module 230 is specifically configured to: for each frame image of the plurality of images to be identified, performing the following operations: averaging the energy difference values of all pixel points of each frame of image to obtain the energy difference value of each frame of image; if the energy difference value of each frame of image is larger than a preset energy difference value threshold value, determining that smoke exists in the area where each frame of image is located.
In some examples, the calculation formula of the averaging process is formula (2). For details of formula (2), reference is made to the description related to formula (2) in the above embodiments of the present invention, and the description is omitted here.
In some examples, the target picture is monitored for a suburban area where the fiber optic cable is deployed or a farmland where the fiber optic cable is deployed.
According to the image processing device provided by the embodiment of the invention, after the motion foreground image is extracted from the target image, the motion foreground image can be traversed by utilizing the sliding window to obtain a plurality of images to be identified, and smoke detection is carried out on the images to be identified. The size of the sliding window is determined based on the size of the moving foreground image, and the size of the image to be identified is the same as the size of the sliding window, so that the size of the image to be identified can be adaptively adjusted according to the size of the moving foreground image, and the identification precision is improved.
Other details of the image processing apparatus according to the embodiment of the present invention are similar to those of the image processing method according to the embodiment of the present invention described above in connection with fig. 1, and are not described here again.
Fig. 3 is a block diagram of an exemplary hardware architecture of an image processing apparatus in an embodiment of the present invention.
As shown in fig. 3, the image processing apparatus 300 includes an input apparatus 301, an input interface 302, a central processor 303, a memory 304, an output interface 305, and an output apparatus 306. The input interface 302, the central processing unit 303, the memory 304, and the output interface 305 are connected to each other through a bus 310, and the input device 301 and the output device 306 are connected to the bus 310 through the input interface 302 and the output interface 305, respectively, and further connected to other components of the image processing device 300.
Specifically, the input device 301 receives input information from the outside, and transmits the input information to the central processor 303 through the input interface 302; the central processor 303 processes the input information based on computer executable instructions stored in the memory 304 to generate output information, temporarily or permanently stores the output information in the memory 304, and then transmits the output information to the output device 306 through the output interface 305; the output device 306 outputs the output information to the outside of the image processing device 300 for use by a user.
That is, the image processing apparatus shown in fig. 3 may also be implemented to include: a memory storing computer-executable instructions; and a processor that when executing the computer-executable instructions can implement the image processing method described in connection with fig. 1.
The embodiment of the invention also provides a computer storage medium, and computer program instructions are stored on the computer storage medium, and when the computer program instructions are executed by a processor, the image processing method described by the embodiment of the invention in connection with fig. 1 is realized.
It should be understood that the invention is not limited to the particular arrangements and instrumentality described above and shown in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after appreciating the spirit of the present invention.
The functional blocks shown in the above block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
In the foregoing, only the specific embodiments of the present invention are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein.
Claims (12)
1. An image processing method, the method comprising:
acquiring a motion foreground image in a target picture;
traversing the motion foreground image by utilizing a sliding window to obtain a plurality of images to be identified, wherein the size of the sliding window is determined based on the size of the motion foreground image;
performing smoke recognition on the plurality of images to be recognized;
the obtaining the motion foreground image in the target picture comprises the following steps:
respectively carrying out gray processing on the moving foreground image extracted from the target picture and the background image of the target picture to obtain a moving foreground image after gray processing and a background image after gray processing;
performing energy difference processing on the motion foreground image after gray level processing by using an energy difference algorithm and the background image after gray level processing to obtain the motion foreground image after energy difference processing, so as to traverse the motion foreground image after energy difference processing by using the sliding window;
each pixel point of the motion foreground image after the energy difference value processing corresponds to the energy difference value of each pixel point.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the ratio of the size of the sliding window to the size of the moving foreground image is equal to a preset size ratio.
3. The method according to claim 1, wherein the method further comprises:
acquiring a video stream from a video acquisition device, wherein the video stream comprises a plurality of frames of pictures;
and taking the multi-frame pictures as target pictures in sequence, and carrying out smoke recognition on images to be recognized corresponding to the target pictures.
4. The method of claim 1, wherein the acquiring a motion foreground image in a target picture comprises:
acquiring a video stream from a video acquisition device, wherein the video stream comprises a plurality of frames of pictures;
performing a background model on the multi-frame pictures to obtain respective corresponding background images of the multi-frame pictures;
extracting background images of the target picture from the background images corresponding to the multi-frame pictures respectively;
and extracting the motion foreground image from the target picture by utilizing the background image of the target picture.
5. The method of claim 1, wherein the energy difference value of the pixel is used to reflect a change in gray scale gradient value of the pixel between the motion foreground image and the background image.
6. The method according to claim 5, wherein the energy difference D of each pixel is calculated e Is the energy difference algorithm f e The calculation formula of the (a) comprises:
wherein f e (,) represents an energy difference function, B e C for the gray gradient value of each pixel point on the background image e For the gray gradient value, P of each pixel point on the motion foreground image B For the gray value of each pixel point on the background image, P C For the gray value of each pixel point on the motion foreground image,gray gradient values along a first direction on the background image for each pixel point,/or->Gray gradient values along a first direction on the motion foreground image for each pixel point, +.>Gray gradient values along a second direction on the background image for each pixel point,/or->And (3) gray gradient values along a second direction on the motion foreground image for each pixel point.
7. The method of claim 1, wherein smoke recognition of the plurality of images to be recognized comprises:
for each frame image in the plurality of images to be identified, performing the following operations:
averaging the energy difference values of all pixel points of each frame of image to obtain the energy difference value of each frame of image;
and if the energy difference value of each frame of image is larger than a preset energy difference value threshold value, determining that smoke exists in the area where each frame of image is located.
8. The method of claim 7, wherein the calculation formula of the averaging process comprises:
where avg represents the energy difference of each frame of image, l represents the length of each frame of image, and h represents the width of each frame of image.
9. The method of claim 1, wherein the target picture is monitored for a suburban area where the fiber optic cable is deployed or a farmland where the fiber optic cable is deployed.
10. An image processing apparatus, characterized in that the apparatus comprises:
the foreground image acquisition module is used for acquiring a motion foreground image in the target picture;
the window traversing module is used for traversing the motion foreground image by utilizing a sliding window to obtain a plurality of images to be identified, wherein the size of the sliding window is determined based on the size of the motion foreground image;
the smoke recognition module is used for carrying out smoke recognition on the plurality of images to be recognized;
the obtaining the motion foreground image in the target picture comprises the following steps:
respectively carrying out gray processing on the moving foreground image extracted from the target picture and the background image of the target picture to obtain a moving foreground image after gray processing and a background image after gray processing;
performing energy difference processing on the motion foreground image after gray level processing by using an energy difference algorithm and the background image after gray level processing to obtain the motion foreground image after energy difference processing, so as to traverse the motion foreground image after energy difference processing by using the sliding window;
each pixel point of the motion foreground image after the energy difference value processing corresponds to the energy difference value of each pixel point.
11. An image processing apparatus, characterized in that the apparatus comprises:
a memory for storing a program;
a processor for executing the program stored in the memory to perform the image processing method of any one of claims 1 to 9.
12. A computer storage medium having stored thereon computer program instructions which, when executed by a processor, implement the image processing method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911374654.XA CN111144312B (en) | 2019-12-27 | 2019-12-27 | Image processing method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911374654.XA CN111144312B (en) | 2019-12-27 | 2019-12-27 | Image processing method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111144312A CN111144312A (en) | 2020-05-12 |
| CN111144312B true CN111144312B (en) | 2024-03-22 |
Family
ID=70520798
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911374654.XA Active CN111144312B (en) | 2019-12-27 | 2019-12-27 | Image processing method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111144312B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650600A (en) * | 2016-10-17 | 2017-05-10 | 东南大学 | Forest smoke and fire detection method based on video image analysis |
| CN107301375A (en) * | 2017-05-26 | 2017-10-27 | 天津大学 | A kind of video image smog detection method based on dense optical flow |
| CN107633212A (en) * | 2017-08-30 | 2018-01-26 | 清华大学苏州汽车研究院（吴江） | A kind of firework detecting method and device based on video image |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017084094A1 (en) * | 2015-11-20 | 2017-05-26 | 富士通株式会社 | Apparatus, method, and image processing device for smoke detection |
-
2019
- 2019-12-27 CN CN201911374654.XA patent/CN111144312B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650600A (en) * | 2016-10-17 | 2017-05-10 | 东南大学 | Forest smoke and fire detection method based on video image analysis |
| CN107301375A (en) * | 2017-05-26 | 2017-10-27 | 天津大学 | A kind of video image smog detection method based on dense optical flow |
| CN107633212A (en) * | 2017-08-30 | 2018-01-26 | 清华大学苏州汽车研究院（吴江） | A kind of firework detecting method and device based on video image |
Non-Patent Citations (4)
| Title |
|---|
| Simone Calderara 等.Vision based smoke detection system using image energy and color information.《Machine Vision and Applications》.2010,第22卷(第4期),第705-719页. * |
| 丁怀对 等.基于运动块追踪的视频烟雾探测方法.《安全与环境学报》.2016,第16卷(第4期),第96-100页. * |
| 林成忠.基于多特征的视频烟雾检测算法.《中国优秀硕士学位论文全文数据库 信息科技辑》.2019,(第4期),全文. * |
| 王琳 等.基于多特征融合的早期火灾烟雾检测.《大连海事大学学报》.2014,第40卷(第1期),第97-100页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111144312A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113643189A (en) | Image denoising method, device and storage medium | |
| US10600158B2 (en) | Method of video stabilization using background subtraction | |
| EP3798975B1 (en) | Method and apparatus for detecting subject, electronic device, and computer readable storage medium | |
| US20210227139A1 (en) | Video stabilization method and apparatus and non-transitory computer-readable medium | |
| CN111723644A (en) | Method and system for detecting occlusion of surveillance video | |
| CN114586337B (en) | Video anti-shake optimization processing method and device and electronic equipment | |
| CN112085768B (en) | Optical flow information prediction method, optical flow information prediction device, electronic equipment and storage medium | |
| CN109754034A (en) | A kind of terminal device localization method and device based on two dimensional code | |
| CN110599516A (en) | Moving target detection method and device, storage medium and terminal equipment | |
| CN104883511A (en) | Image processing method and electronic equipment | |
| CN115760912A (en) | Moving object tracking method, device, equipment and computer readable storage medium | |
| US20110085026A1 (en) | Detection method and detection system of moving object | |
| CN112435278B (en) | Visual SLAM method and device based on dynamic target detection | |
| CN113158773B (en) | Training method and training device for living body detection model | |
| CN111160340B (en) | Moving object detection method and device, storage medium and terminal equipment | |
| CN111144312B (en) | Image processing method, device, equipment and medium | |
| CN110399823B (en) | Subject tracking method and apparatus, electronic device, and computer-readable storage medium | |
| CN101753854A (en) | Image communication method and electronic device using same | |
| CN115334250B (en) | Image processing method and device and electronic equipment | |
| CN114255177B (en) | Exposure control method, device, equipment and storage medium in imaging | |
| CN113033355B (en) | Abnormal target identification method and device based on intensive power transmission channel | |
| US11495006B2 (en) | Object detection method for static scene and associated electronic device | |
| CN114268737A (en) | Automatic trigger method for shooting, certificate identification method, equipment and storage medium | |
| CN111476063B (en) | Target tracking method, device, storage medium and electronic equipment | |
| CN114037741A (en) | Adaptive target detection method and device based on event camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |