WO2020073505A1 - Image processing method, apparatus and device based on image recognition, and storage medium - Google Patents

Abstract

Disclosed are an image processing method, apparatus and device based on image recognition, and a storage medium. The method comprises: a photographing apparatus photographing a target area to obtain target video data of the target area; screening out a target frame image from the target video data according to a pre-set screening rule, and acquiring a reference image of the target area; comparing feature information of the target frame image with feature information of the reference image to obtain the degree of matching between the target frame image and the reference image; and when the degree of matching between the target frame image and the reference image is less than a pre-set threshold value, determining that a target object breaks into the target area, and automatically recognizing the object that breaks into a monitoring area, thereby improving the efficiency of image recognition.

Description

Image processing method, device, equipment and storage medium based on image recognition

This application requires the priority of the Chinese patent application submitted to the Chinese Patent Office on October 11, 2018, with the application number 201811186563.9 and the application name "Image Recognition-Based Image Processing Methods, Devices, Equipment, and Storage Media", all of its contents Incorporated by reference in this application.

Technical field

The present application relates to the field of medical technology, and in particular, to an image processing method, device, device, and storage medium based on image recognition.

Background technique

Image recognition is a method of processing, analyzing and understanding images to identify the objects in the image. This method is widely used in security video surveillance, image retrieval, automatic driving or quality inspection and other fields, giving users life and work Bring great convenience. In the field of security video surveillance, the staff mainly watch the video data taken for the monitored area to determine whether an abnormal object (such as a stranger) has intruded into the monitored area by comparing every two frames of the video data. It has been found in practice that this image recognition method requires a lot of time and labor resources, resulting in low efficiency of image recognition.

Summary of the invention

Embodiments of the present application provide an image processing method, device, device, and storage medium based on image recognition, which automatically recognizes objects that break into a monitoring area, and improves the efficiency of image recognition.

In a first aspect, an embodiment of the present application provides an image processing method based on image recognition. The method includes:

Shoot the target area through the shooting device to obtain the target video data of the target area;

Filter out the target frame image from the target video data according to the preset filtering rules, and obtain the reference image of the target area, the reference image refers to the image captured when there is no target object intrusion in the target area ;

Comparing feature information of the target frame image with feature information of the reference image to obtain a matching degree between the target frame image and the reference image;

When the matching degree of the target frame image and the reference image is less than a preset threshold, it is determined that the target object intrudes in the target area.

In a second aspect, an embodiment of the present application provides a base image processing device, which includes:

The shooting module is used for shooting the target area by the shooting device to obtain the target video data of the target area.

The filtering module is used to filter out target frame images from the target video data according to a preset filtering rule, and obtain a reference image of the target area, where the reference image means that there is no target object intrusion in the target area Images taken at the time.

The comparison module is used to compare the feature information of the target frame image with the feature information of the reference image to obtain the matching degree between the target frame image and the reference image.

The determining module is configured to determine that the target object intrudes in the target area when the matching degree between the target frame image and the reference image is less than a preset threshold.

In a third aspect, an embodiment of the present application provides a monitoring device, the device includes: a processor adapted to implement one or more instructions; and a computer-readable storage medium, the computer-readable storage medium storing one or More than one instruction, the one or more instructions are suitable to be loaded by the processor and execute the following steps:

Shoot the target area through the shooting device to obtain the target video data of the target area;

Filter out the target frame image from the target video data according to the preset filtering rules, and obtain the reference image of the target area, the reference image refers to the image captured when there is no target object intrusion in the target area ;

Comparing feature information of the target frame image with feature information of the reference image to obtain a matching degree between the target frame image and the reference image;

When the matching degree of the target frame image and the reference image is less than a preset threshold, it is determined that the target object intrudes in the target area.

According to a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing one or more instructions, the one or more instructions being suitable for being loaded by a processor and performing the following steps :

Shoot the target area through the shooting device to obtain the target video data of the target area;

Filter out the target frame image from the target video data according to the preset filtering rules, and obtain the reference image of the target area, the reference image refers to the image captured when there is no target object intrusion in the target area ;

Comparing feature information of the target frame image with feature information of the reference image to obtain a matching degree between the target frame image and the reference image;

When the matching degree of the target frame image and the reference image is less than a preset threshold, it is determined that the target object intrudes in the target area.

In the embodiments of the present application, objects that break into the monitoring area can be automatically identified based on the image, thereby improving the efficiency of image recognition.

BRIEF DESCRIPTION

In order to more clearly explain the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some of the applications For the embodiment, for those of ordinary skill in the art, without paying any creative labor, other drawings may be obtained based on these drawings.

1 is a schematic flowchart of an image processing method based on image recognition provided by an embodiment of the present application;

2 is a schematic flowchart of an image processing method based on image recognition provided by another embodiment of the present application;

3 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

4 is a schematic structural diagram of a monitoring device provided by another embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of this application.

The application example can be executed by a monitoring device, which can include a front-end part, a transmission part, and a back-end part. The front-end part mainly consists of a camera, a sensor, a lens, a gimbal, a protective cover, a bracket, a decoder, etc. Mainly used for shooting video, audio or images, etc .; transmission part: using cables and wires to transmit video, audio or control signals by overhead, underground or laying along the wall, etc .; rear part: mainly by picture splitter, monitor , Control devices, video storage devices, etc., mainly used for processing the number of videos or images.

The example of this application can be applied to security video surveillance scenes to analyze the video data of the scenes to determine whether there are target objects intruding in these scenes. Security video surveillance scenes include video surveillance scenes in residential areas and video surveillance in military areas Scenes or video surveillance scenes of shopping mall warehouses, etc., target objects include strangers or animals, etc. Specifically, in order to monitor the monitoring area, the shooting device of the monitoring device may shoot the monitoring area to obtain video data of the monitoring area. According to the preset filtering rules, the target frame image is selected from the video data of the monitoring area, and the reference image of the monitoring area is obtained. The reference image is an image in which no target object intrudes in the monitoring area. Compare with the characteristic information of the reference image to obtain the matching degree between the target frame image and the reference image. When the matching degree between the target frame image and the reference image is less than the preset threshold, it indicates the difference between the target frame image and the reference image It is relatively large, and it is determined that there is a target object intrusion in the monitoring area. By identifying the target frame image in the video data of the target area, it can automatically identify whether there is a target object intrusion in the target area, save labor resources, improve the efficiency of image recognition, and meet the user's automated and intelligent demand for video surveillance , Can effectively ensure the safety of users; in addition, by filtering the video data in the target area, only part of the frame images in the video data need to be identified, and all images in the video data do not need to be identified, further improving The efficiency of image recognition.

Please refer to FIG. 1, which is a schematic flowchart of an image processing method based on image recognition provided by an embodiment of the present application. The method of the embodiment of the present application may be executed by the above-mentioned monitoring device. In this embodiment, the image processing method based on image recognition includes the following steps.

S101. Shoot a target area by a shooting device to obtain target video data of the target area.

In the embodiment of the present application, in a target area where a target object such as a stranger or an animal is prohibited from entering, for the user's personal safety and property security, the target area may be photographed by a shooting device to obtain target video data of the target area . The shooting device may refer to a panoramic camera device or a hemisphere camera device, etc., and the target area may refer to an entrance area of a residential area, a garage, an area where a warehouse of a shopping mall is located, or a military area.

In one embodiment, when a sensor in the monitoring device detects that someone, an animal, etc. has entered the monitoring area, the shooting device of the monitoring device is triggered to shoot the monitoring area to obtain video data of the monitoring area, for example, by transmitting through the sensor Infrared spectrum, and receive the reflected infrared spectrum, calculate the time interval between the emitted infrared spectrum and the reflected infrared spectrum, when the time interval is lower than the preset time threshold, determine that there is a human or animal intrusion in the target area, trigger the monitoring equipment The shooting device shoots the monitoring area to obtain video data of the monitoring area.

In another embodiment, in order to reduce the pressure of the processing device for processing image data, the monitoring device may monitor the target area of a certain period of time. Specifically, set a shooting period for the shooting device when the time is within the shooting period of the shooting device When it is inside, the shooting device of the monitoring device is triggered to shoot the monitoring area to obtain the video data of the monitoring area. The shooting time period can be set according to the time law of the history of the target area breaking into the target object. The shooting time period can refer to The history of the target area breaking into the target object frequency is greater than the preset frequency period, such as the shooting time period refers to 6:00 to 12:00 in the evening.

S102. Filter target frame images from target video data according to a preset filtering rule, and obtain a reference image of a target area. The reference image refers to an image that is captured when a target object does not exist in the target area.

In the embodiment of the present application, in order to improve the efficiency of image recognition, the monitoring device may filter out the target frame image from the target video data according to a preset filtering rule, and obtain a reference image. The reference image may be obtained from the target video, or It may refer to the acquisition based on the historical video data of the target area, and the reference image refers to an image captured when there is no target object intrusion in the target area.

S103. Compare the feature information of the target frame image with the feature information of the reference image to obtain a matching degree between the target frame image and the reference image.

In the embodiment of the present application, the monitoring device may take the target frame image and the reference image as a whole to obtain the feature information of the target frame image and the reference image. The feature information may refer to a histogram of orientation gradient (Histogram of Oriented Gradient, HOG), at least one of Scale-invariant feature transform (SIFT) or color histogram Color Histogram, which compares the feature information of the target frame image with the feature information of the reference image to obtain the target frame The degree of matching between the image and the reference image; or divide the target frame image and the reference image into multiple sub-pictures, obtain the characteristic information of each sub-picture separately, and determine the relationship between the target frame image and the reference image according to the characteristic information of each sub-picture suitability. Among them, the greater the matching degree, the greater the similarity between the target frame image and the reference image, that is, the smaller the difference between the target frame image and the reference image; conversely, the smaller the matching degree, the target frame image and the reference image The smaller the similarity between the reference images, the greater the difference between the target frame image and the reference image.

In one embodiment, in order to improve the efficiency of acquiring the matching degree between the target frame image and the reference image, the partial feature information of the target frame image and the partial feature information of the reference image may be compared to obtain the target frame image and the reference The matching degree of the image. Specifically, the characteristic information of the target frame image is sampled according to the preset sampling frequency, and the characteristic information of the reference image is sampled according to the preset sampling frequency, and the sampling point of the target frame image is The feature information is compared with the feature information of the corresponding sampling point of the reference image to obtain the matching degree of the target frame image and the reference image.

In another embodiment, in order to improve the accuracy of acquiring the matching degree between the target frame image and the reference image, all the feature information of the target frame image and the corresponding feature information of the reference image may be compared to obtain the target frame The degree of matching between the image and the reference image.

It should be noted that, in order to improve the accuracy and flexibility of obtaining the matching degree between the target frame image and the reference image, the monitoring device can dynamically select a comparison strategy of feature information according to the stability of the target area. The comparison strategy includes all comparisons and partial comparisons. Yes, specifically, when it is detected that the stability of the target area is greater than or equal to the preset stable value, it indicates that the target area itself does not change much. For example, the background (objects inherent in the target area such as lighting and weather) changes slowly, and the target object breaks The probability of entering is small, you can select the part of the feature information to this strategy, that is, compare the part of the target frame image with the reference image, and use the matching degree between the target frame image and the reference image; when detecting When the stability of the target area is less than the preset stable value, it indicates that the target area itself changes greatly, such as the background changes quickly, the probability of the target object intruding is large, you can choose all of the feature information for this strategy, that is, the target frame image Compare all the feature information of the reference image with all the feature information of the reference image to the target frame image The degree of match between the reference image. The comparison strategy of the above characteristic information may also be manually selected by the user according to personal needs.

S104. When the matching degree between the target frame image and the reference image is less than a preset threshold, it is determined that there is a target object intrusion in the target area.

In the embodiment of the present application, when the matching degree between the target frame image and the reference image is greater than or equal to a preset threshold, it indicates that the difference between the target frame image and the reference image is small, and it is determined that there is no target object intrusion in the target area When the matching degree between the target frame image and the reference image is less than the preset threshold, it indicates that the difference between the target frame image and the reference image is large, and it is determined that there is a target object intrusion in the target area. In order to accurately identify the target object, the preset threshold may refer to the difference between the background of the target area and the target object, for example, when the characteristics of the target object (such as color) and the background content of the target area (such as the color of the background ) When it is very close, set the preset threshold to a smaller value. When the difference between the characteristics of the target object (such as color) and the background content of the target area (such as the color of the background) is large, the preset threshold Set the threshold to a larger value. .

In one example, in order to ensure the personal safety and property safety of the user, when the matching degree between the target frame image and the reference image is less than a preset threshold, the monitoring device may output a prompt message, the prompt message is used to prompt the target object to invade The prompt information may be information output in the form of voice, or flashing warning lights, vibration monitoring equipment, and the like.

In one embodiment, in order to notify the manager in time to deal with the event that the target area intrudes into the target object in a timely manner, when the matching degree between the target frame image and the reference image is less than the preset threshold, the contact information of the administrator is obtained, and the contact information will be The target frame image is sent to the device bound to the administrator's contact information. The contact information includes the administrator's instant messaging account, such as a phone number, WeChat account, or QQ account.

In the embodiments of the present application, objects that break into the monitoring area can be automatically identified based on the image, thereby improving the efficiency of image recognition.

Please refer to FIG. 2, which is a schematic flowchart of another image processing method based on image recognition provided by an embodiment of the present application. The method of the embodiment of the present application may be performed by the monitoring device mentioned above. In this embodiment, the image processing method based on image recognition includes the following steps.

S201. Shoot a target area by a shooting device to obtain target video data of the target area.

In one embodiment, the temperature information of the target area is obtained by a sensor, and when the temperature information of the target area indicates that the temperature value of the target area is greater than the preset temperature value, the target area is photographed by a shooting device to obtain the target area The step of target video data; or, the step of receiving a shooting instruction for the target area and performing the step of shooting the target area by the shooting device to obtain the target video data of the target area.

In order to reduce the pressure of the monitoring device to process image data, the monitoring device can trigger the shooting device to shoot video by the parameters in the target area. Specifically, the temperature information of the target area is obtained through the sensor. When the temperature information of the target area indicates that the temperature value of the target area is greater than the preset value At the temperature value, there is an intrusion of a temperature object in the target area. The object may refer to a person or an animal. In order to avoid the intrusion of a stranger or an animal, the shooting device of the monitoring device is triggered to shoot the monitoring area to be monitored Video data for the area.

Alternatively, the user can trigger the shooting device to shoot, specifically, receive the shooting instruction sent by the user, and trigger the shooting device of the monitoring device to shoot the monitoring area to obtain video data of the monitoring area, and the user can use touch (such as buttons, Slide or click) or voice to send shooting instructions to the shooting device.

S202. Filter target frame images from target video data according to preset filtering rules, and obtain a reference image of the target area. The reference image refers to an image that is captured when there is no target object intruding in the target area.

In one example, the preset screening rules include screening rules according to scene change parameters, and step S202 includes the following steps S11 to S12.

S11. Acquire a scene change parameter of the target area according to historical video data of the target area, where the scene change parameter is used to indicate the stability of the target area.

S12. Acquire a reference image of the target area according to the scene change parameters of the target area, and filter out the target frame image from the target video according to the scene change parameters of the target area.

In steps S11 to S12, the monitoring device can obtain the reference image and the target frame image according to the scene change parameters of the target area. Specifically, the monitoring device can obtain the historical data of the target area within a preset time period and obtain the target according to the historical data The scene change parameters of the area, and the reference image is obtained according to the scene change parameters of the target area, and the target frame image is selected from the target video data according to the scene change parameters.

In the same time period, the scene change parameters of the target area have a large similarity. Therefore, the monitoring device can obtain the current time, the historical video data of the target area corresponding to the current time, and according to the current time Historical video data determines the scene change parameters of the target area. For example, when the current time is 6:00 in the evening, the monitoring device may obtain historical video data of the target area in the evening from 6:00 to 12:00, based on the The historical video data acquires the scene change parameters of the target area.

In one embodiment, step S12 includes: when the scene change parameter indicates that the stability of the target area is greater than or equal to a preset stable value, acquiring a multi-frame image of the target object in the historical video data of the target area; the multi-frame image The pixel information of is subjected to averaging to obtain the reference image of the target area, an image is selected from the target video at a first preset time interval, and one frame of image at a time, the selected image is used as the target frame image.

When the scene change parameter indicates that the stability of the target area is greater than or equal to the preset stable value, it indicates that the target area itself does not change much, such as the background changes slowly, the probability of the target object intruding is small, and the moving speed of the target object intruding in the target area If the reference speed is less than the preset speed, the reference image can be obtained according to the historical video data, and the monitoring device can obtain the multi-frame image of the target object in the historical video data of the target area, and average the pixel information of the multi-frame image to Obtain the reference image of the target area, and select the image from the target video according to the first preset time interval, and one frame of image at a time, use the selected image as the target frame image, the first preset time interval can be based on the target area Set the time rule of breaking into the target object, for example, the probability of breaking into the target object at night is large, you can set the first preset time interval to a smaller value during working hours (such as 9: 00-17: 00 in the morning) ) The probability of breaking into the target object is small, and the first preset time interval may be set to a larger value.

In another embodiment, step S12 includes: when the scene change parameter indicates that the stability of the target area is less than a preset stable value, selecting images from the target video at a second preset time interval, and each time Select two frames of images at a time; use the first frame of the two frames as the reference image of the target area, and use the second frame of the two frames as the target frame image, the first The shooting time of the frame image is earlier than the shooting time of the second frame image.

When the scene change parameter indicates that the stability of the target area is less than the preset stable value, it indicates that the target area itself changes greatly, for example, the background changes quickly, the probability of the target object intruding is large, and the movement speed of the target object in the target area is faster than the preset Set the speed. At this time, the historical video data cannot reflect the characteristics of the target area itself, so the reference image can be obtained from the currently captured target video. The monitoring device can obtain the multi-frame image of the target object in the historical video data of the target area. Select images from the target video at two preset time intervals, and select two frames of images at a time; use the first frame of the two frames as the reference image of the target area, and the second frame of the two frames as the reference image For the target frame image, the shooting time of the first frame image is earlier than the shooting time of the second frame image.

In one embodiment, the above scene change parameters include at least one of the background change rate of the target area, the probability of the target object intruding, and the moving speed of the target object in the target area. The above scene change parameters indicate that the stability of the target area is greater than Or equal to the preset stable value may refer to: the background change rate of the target area is less than or equal to the preset change rate, and / or the probability of the target object intruding is less than or equal to the preset probability value, and / or the target object is in the target area The moving speed is less than or equal to the preset speed value; the above scene change parameter indicates that the stability of the target area is less than the preset stable value may refer to: the background change rate of the target area is greater than the preset change rate, and / or the probability of the target object intruding Is greater than the preset probability value, and / or the moving speed of the target object in the target area is greater than the preset speed value.

S203. Compare the feature information of the target frame image with the feature information of the reference image to obtain a matching degree between the target frame image and the reference image.

In one example, the target frame image and the reference image are divided into multiple sub-images respectively according to a preset division rule, and the feature information of each sub-image in the target frame image and the feature information of each sub-image in the reference image are obtained, The feature information of each sub-image in the target frame image is compared with the feature information of the corresponding sub-image in the reference image to obtain the matching degree between each sub-image in the target frame image and the corresponding sub-image in the reference image , Weighted and sum the determined matching degree to obtain the matching degree between the target frame image and the reference image.

In order to improve the accuracy of acquiring the matching degree between the target frame image and the reference image, the monitoring device may divide the target frame image and the reference image into multiple sub-images respectively according to a preset division rule, and the preset division rule includes horizontal division Rules and / or vertical division rules and / or diagonal division rules, and obtain the feature information of each sub-image in the target frame image, and the feature information of each sub-image in the reference image, and convert each sub-image in the target frame image The feature information of is compared with the feature information of the corresponding sub-image in the reference image to obtain the matching degree between each sub-image in the target frame image and the corresponding sub-image in the reference image. The map sets weights, and the weights of the sub-pictures are used to reflect the influence degree of the sub-pictures on the matching degree between the target frame image and the reference image, and the determined matching degrees are weighted and summed according to the weights of each sub-picture to obtain the target The degree of matching between the frame image and the reference image.

The above is that each sub-picture in the target frame image includes: setting the weight of the sub-picture according to the probability of the target object appearing in the area indicated in each sub-picture. For example, if the area indicated in the sub-picture is the wall area of the cell, then the The probability of the target object appearing in the area is small, and the weight of the sub-picture can be set to a small value. The area indicated in the sub-picture is the entrance area of the cell. The weight of the graph is set to a larger value.

Or set the weight of each sub-picture of the target frame image through a logistic regression classifier. The logistic regression classifier sets the weight of each sub-picture through the change characteristics (ie stability) of the area where the sub-picture is located. Specifically, it has a fixed change characteristic ( (The stability of the area indicated by the sub-picture is greater than or equal to the preset stability) The sub-picture sets a smaller weight, which is a sub-image that does not have a fixed change feature (that is, the stability of the area indicated by the sub-picture is less than the preset stability) The graph sets a larger weight. Example: The area referred to by the sub-picture is the area where the traffic lights at the intersection are located. The traffic signal has a fixed change feature, and the sub-picture can be set to a smaller weight; In the area where the zebra crossing is located, the flow of people usually does not have fixed characteristics, and the sub-graph can be set with a larger weight.

In one embodiment, in order to improve the accuracy of acquiring the matching degree between the target frame image and the reference image, and to improve the efficiency of acquiring the matching degree between the target frame image and the reference image, the monitoring device may follow a preset division rule Divide the target frame image and the reference image into multiple sub-images respectively, the preset division rules include horizontal division rule and / or vertical division rule and / or diagonal division rule, and obtain the characteristics of each sub-image in the target frame image Information, and feature information of each sub-image in the reference image, compare the feature information of each sub-image in the target frame image with the feature information of the corresponding sub-image in the reference image to obtain each sub-image in the target frame image The matching degree between the corresponding sub-images in the image and the reference image, the number of sub-pictures whose matching degree is less than the preset value is counted, and the matching degree between the target frame image and the reference image is determined according to the number of sub-pictures. For example, when the number of sub-pictures is less than the preset number threshold, it is determined that the matching degree between the target frame image and the reference image is less than the preset threshold; when the number of sub-pictures is greater than or equal to the preset number threshold, the target frame image and the reference image are determined The matching degree between them is greater than or equal to the preset threshold.

S204. When the matching degree between the target frame image and the reference image is less than a preset threshold, it is determined that there is a target object intrusion in the target area.

S205. Acquire a matching training image from the target frame image from the database.

S206. Obtain the object information in the training image from the database, the database includes multiple training images, and the object information in each training image.

S207: Use the object information in the training image as the object information of the target object of the target frame image, and output the object information of the target object.

In steps S204-S207, when it is determined that a target object intrudes in the target area, the monitoring device may acquire the object information of the target object (that is, the label of the target object), so that the user can take corresponding measures in time to reduce the target object to the user The harm brought. Specifically, obtain a matching training image with the target frame image from the database, such as obtaining the same training image as the target frame image, or a training image whose similarity to the target frame image is greater than a preset similarity value, and from the database Obtain the object information in the training image, use the object information in the training image as the object information of the target object of the target frame image, and output the object information of the target object. When the target object is a person, the object information of the target object includes the identity information of the target object, and / or the log information of the target object, etc. The identity information of the target object includes the name, place of origin, age, etc .; when the target When the object is an animal, the object information of the target object includes the name or type.

In the embodiments of the present application, objects that break into the monitoring area can be automatically identified based on the image, thereby improving the efficiency of image recognition.

Please refer to FIG. 3, which is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application. The apparatus of the embodiment of the present application may be provided in the above-mentioned monitoring device. In this embodiment, the device includes:

The shooting module 301 is used for shooting a target area by a shooting device to obtain target video data of the target area.

The filtering module 302 is configured to filter out target frame images from the target video data according to a preset filtering rule and obtain a reference image of the target area, where the reference image means that there is no target object in the target area Image taken at the time of import.

The comparison module 303 is used to compare the feature information of the target frame image with the feature information of the reference image to obtain the matching degree between the target frame image and the reference image.

The determining module 304 is configured to determine that the target object intrudes in the target area when the matching degree between the target frame image and the reference image is less than a preset threshold.

Optionally, the shooting module 301 is specifically configured to acquire the temperature information of the target area through a sensor; when the temperature information of the target area indicates that the temperature value of the target area is greater than a preset temperature value, the pass shooting is performed The device shoots the target area to obtain the target video data of the target area; or, receives a shooting instruction for the target area and executes the shooting of the target area by the shooting device to obtain the target of the target area Steps for video data.

Optionally, the preset screening rules include screening rules according to scene change parameters; the screening module 302 is specifically configured to obtain scene change parameters of the target area based on historical video data of the target area, and the scene changes The parameter is used to indicate the stability of the target area; obtain a reference image of the target area according to the scene change parameter of the target area, and filter out the target video from the target video according to the scene change parameter of the target area Target frame image.

Optionally, the screening module 302 is specifically configured to obtain that the target object does not exist in the historical video data of the target area when the scene change parameter indicates that the stability of the target area is greater than or equal to a preset stable value Multi-frame images; averaging the pixel information of the multi-frame images to obtain the reference image of the target area; selecting images from the target video at a first preset time interval, one frame at a time , Use the selected image as the target frame image.

Optionally, the screening module 302 is specifically configured to select an image from the target video at a second preset time interval when the scene change parameter indicates that the stability of the target area is less than a preset stable value, and each Select two frames of images at a time; use the first frame of the two frames as the reference image of the target area, and use the second frame of the two frames as the target frame image, the first The shooting time of the frame image is earlier than the shooting time of the second frame image.

Optionally, the comparison module 303 is specifically configured to divide the target frame image and the reference image into multiple sub-images respectively according to a preset division rule; obtain feature information of each sub-image in the target frame image , And the feature information of each sub-image in the reference image; compare the feature information of each sub-image in the target frame image with the feature information of the corresponding sub-image in the reference image to obtain the The degree of matching between each sub-image in the target frame image and the corresponding sub-image in the reference image; weighting and summing the determined degree of matching to obtain the match between the target frame image and the reference image degree.

Optionally, the obtaining module 305 is configured to obtain a training image matching the target frame image from a database, and obtain object information in the training image from the database, and the database includes multiple training images , And the object information in each training image.

Optionally, the output module 306 is configured to use the object information in the training image as the object information of the target object in the target frame image; and output the object information of the target object.

In the embodiments of the present application, objects that break into the monitoring area can be automatically identified based on the image, thereby improving the efficiency of image recognition.

Please refer to FIG. 4, which is a schematic structural diagram of a monitoring device provided by an embodiment of the present application. As shown in the figure, the monitoring device in this embodiment may include: one or more processors 401; one or more input devices 402 , One or more output devices 403 and memory 404. The processor 401, the input device 402, the output device 403, and the memory 404 are connected via a bus 405.

The processor 401 may be a central processing unit (Central Processing Unit, CPU), the processor may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC ), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The input device 402 may include a touchpad, a fingerprint sensor (for collecting user's fingerprint information and direction information of the fingerprint), a temperature sensor (for acquiring temperature information of the target area), and a shooting device (for acquiring video of the target area Data), a microphone, etc., the output device 403 may include a display (LCD, etc.), a speaker, etc., and the output device 403 may output object information of a target object.

The memory 404 may include a read-only memory and a random access memory, and provide instructions and data to the processor 401. A part of the memory 404 may further include a non-volatile random access memory, the memory 404 is used to store a computer program, the computer program includes program instructions, and the processor 401 is used to execute the program instructions stored in the memory 404 to be used to execute a An image processing method based on image recognition is used to perform the following operations:

Shoot the target area through the shooting device to obtain the target video data of the target area;

Filter out the target frame image from the target video data according to the preset filtering rules, and obtain the reference image of the target area, the reference image refers to the image captured when there is no target object intrusion in the target area ;

Comparing feature information of the target frame image with feature information of the reference image to obtain a matching degree between the target frame image and the reference image;

When the matching degree of the target frame image and the reference image is less than a preset threshold, it is determined that the target object intrusion exists in the target area.

Optionally, the processor 401 is used to execute the program instructions stored in the memory 404 to perform the following operations:

Acquiring temperature information of the target area through a sensor; when the temperature information of the target area indicates that the temperature value of the target area is greater than a preset temperature value, executing the shooting of the target area by the shooting device to obtain the target Steps of target video data in the area; or,

Receiving a shooting instruction for the target area, and performing the step of shooting the target area by the shooting device to obtain target video data of the target area.

Optionally, the processor 401 is used to execute the program instructions stored in the memory 404 to perform the following operations:

Acquiring scene change parameters of the target area according to historical video data of the target area, the scene change parameters being used to indicate the stability of the target area;

Acquire the reference image of the target area according to the scene change parameters of the target area, and filter out the target frame image from the target video according to the scene change parameters of the target area.

Optionally, the processor 401 is used to execute the program instructions stored in the memory 404 to perform the following operations:

When the scene change parameter indicates that the stability of the target area is greater than or equal to a preset stable value, acquire multiple frames of the target object in the historical video data of the target area;

Averaging the pixel information of the multi-frame image to obtain the reference image of the target area;

An image is selected from the target video at a first preset time interval, and one frame of image at a time, the selected image is used as the target frame image.

Optionally, the processor 401 is used to execute the program instructions stored in the memory 404 to perform the following operations:

When the scene change parameter indicates that the stability of the target area is less than a preset stable value, select images from the target video at a second preset time interval, and select two frames of images at a time;

The first frame of the two frames is used as the reference image of the target area, and the second frame of the two frames is used as the target frame. The first frame of the image is taken early At the shooting time of the second frame image.

Optionally, the processor 401 is used to execute the program instructions stored in the memory 404 to perform the following operations:

Divide the target frame image and the reference image into multiple sub-images according to a preset division rule;

Acquiring feature information of each sub-image in the target frame image, and feature information of each sub-image in the reference image;

Comparing the feature information of each sub-image in the target frame image with the feature information of the corresponding sub-image in the reference image to obtain each sub-image in the target frame image and the reference image in the reference image The matching degree between corresponding sub-images;

Weighting and summing the determined matching degree to obtain the matching degree between the target frame image and the reference image.

Optionally, the processor 401 is used to execute the program instructions stored in the memory 404 to perform the following operations:

Obtain a matching training image from the target frame image from the database;

Obtaining object information in the training images from the database, the database includes multiple training images, and object information in each training image;

Use the object information in the training image as the object information of the target object in the target frame image;

Output the object information of the target object.

In the embodiments of the present application, objects that break into the monitoring area can be automatically identified based on the image, thereby improving the efficiency of image recognition.

The processor 401, the input device 402, and the output device 403 described in the embodiments of the present application may execute the implementation methods described in the first and second embodiments of the image processing method based on image recognition provided by the embodiments of the present application It can also implement the implementation of the monitoring device described in the embodiments of the present application, which will not be repeated here.

An embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The program instructions are executed by a processor to implement a diagram of the present application. 1 and the image processing method based on image recognition shown in the embodiment of FIG. 2.

The computer-readable storage medium may be an internal storage unit of the monitoring device described in any of the foregoing embodiments, such as a hard disk or a memory of the control device. The computer-readable storage medium may also be an external storage device of the control device, for example, a plug-in hard disk equipped on the control device, a smart memory card (Smart, Media, Card, SMC), and secure digital (SD, Digital) ) Card, flash card (Flash Card), etc. Further, the computer-readable storage medium may also include both an internal storage unit of the control device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the control device. The computer-readable storage medium may also be used to temporarily store data that has been or will be output.

Those of ordinary skill in the art may realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two, in order to clearly explain the hardware and software. Interchangeability, in the above description, the composition and steps of each example have been generally described according to function. Whether these functions are executed in hardware or software depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working processes of the control devices and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed control device and method may be implemented in other ways. For example, the device embodiments described above are schematic. For example, the division of the unit may be a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or may Integration into another system, or some features can be ignored, or not implemented. The above is only the specific implementation of this application, but the scope of protection of this application is not limited to this, any person skilled in the art can easily think of various equivalents within the technical scope disclosed in this application Modifications or replacements, these modifications or replacements should be covered within the scope of protection of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (20)

1. An image processing method based on image recognition, characterized in that it includes:

   Shoot the target area through the shooting device to obtain the target video data of the target area;

   Filter out the target frame image from the target video data according to the preset filtering rules, and obtain the reference image of the target area, the reference image refers to the image captured when there is no target object intrusion in the target area ;

   Comparing feature information of the target frame image with feature information of the reference image to obtain a matching degree between the target frame image and the reference image;

   When the matching degree of the target frame image and the reference image is less than a preset threshold, it is determined that the target object intrudes in the target area.
2. The method of claim 1, further comprising:

   Acquiring temperature information of the target area through a sensor; when the temperature information of the target area indicates that the temperature value of the target area is greater than a preset temperature value, executing the shooting of the target area by the shooting device to obtain the target Steps of target video data in the area; or,

   Receiving a shooting instruction for the target area, and performing the step of shooting the target area by the shooting device to obtain target video data of the target area.
3. The method according to claim 1 or 2, wherein the preset screening rules include screening rules according to scene change parameters;

   The filtering out the target frame image from the target video data according to the preset filtering rule and acquiring the reference image of the target area includes:

   Acquiring scene change parameters of the target area according to historical video data of the target area, the scene change parameters being used to indicate the stability of the target area;

   Acquire the reference image of the target area according to the scene change parameters of the target area, and filter out the target frame image from the target video according to the scene change parameters of the target area.
4. The method according to claim 3, wherein the reference image of the target area is acquired according to the scene change parameter of the target area, and filtered from the target video according to the scene change parameter of the target area Out target frame image, including:

   When the scene change parameter indicates that the stability of the target area is greater than or equal to a preset stable value, acquire multiple frames of the target object in the historical video data of the target area;

   Averaging the pixel information of the multi-frame image to obtain the reference image of the target area;

   An image is selected from the target video at a first preset time interval, and one frame of image at a time, the selected image is used as the target frame image.
5. The method according to claim 3, wherein the reference image of the target area is acquired according to the scene change parameter of the target area, and filtered from the target video according to the scene change parameter of the target area Out target frame image, including:

   When the scene change parameter indicates that the stability of the target area is less than a preset stable value, select images from the target video at a second preset time interval, and select two frames of images at a time;

   The first frame of the two frames is used as the reference image of the target area, and the second frame of the two frames is used as the target frame. The first frame of the image is taken early At the shooting time of the second frame image.
6. The method according to claim 4 or 5, wherein the feature information of the target frame image is compared with the feature information of the reference image to obtain the target frame image and the reference image The degree of matching, including:

   Divide the target frame image and the reference image into multiple sub-images according to a preset division rule;

   Acquiring feature information of each sub-image in the target frame image, and feature information of each sub-image in the reference image;

   Comparing the feature information of each sub-image in the target frame image with the feature information of the corresponding sub-image in the reference image to obtain each sub-image in the target frame image and the reference image in the reference image The matching degree between corresponding sub-images;

   Weighting and summing the determined matching degree to obtain the matching degree between the target frame image and the reference image.
7. The method according to claim 1, wherein after determining that a target object intrudes in the target area, the method further comprises:

   Obtain a matching training image from the target frame image from the database;

   Obtaining object information in the training images from the database, the database includes multiple training images, and object information in each training image;

   Use the object information in the training image as the object information of the target object in the target frame image;

   Output the object information of the target object.
8. The method of claim 1, wherein the method further comprises:

   When it is determined that the target object intrusion exists in the target area, prompt information is output, and the prompt information is used to prompt the target object intrusion in the target area.
9. The method according to claim 1, wherein the shooting of the target area by the shooting device to obtain target video data of the target area includes:

   When the time is within a preset time period, the target area is captured by the shooting device to obtain the target video data of the target area. The preset time period means that the frequency of the target object appearing in the target area is greater than the Set the frequency period.
10. An image processing device, characterized in that it includes:

    The shooting module is used to shoot the target area through the shooting device to obtain the target video data of the target area;

    The filtering module is used to filter out target frame images from the target video data according to a preset filtering rule, and obtain a reference image of the target area, where the reference image means that there is no target object intrusion in the target area Images taken at the time;

    A comparison module, configured to compare the feature information of the target frame image with the feature information of the reference image to obtain a matching degree between the target frame image and the reference image;

    The determining module is configured to determine that the target object intrudes in the target area when the matching degree between the target frame image and the reference image is less than a preset threshold.
11. The device of claim 10, further comprising:

    The shooting module is used for acquiring temperature information of the target area through a sensor; when the temperature information of the target area indicates that the temperature value of the target area is greater than a preset temperature value, the target area is executed by the shooting device Performing shooting to obtain the target video data of the target area; or, receiving a shooting instruction for the target area and performing the step of shooting the target area by the shooting device to obtain the target video data of the target area .
12. The apparatus according to claim 10 or 11, wherein the preset screening rules include screening rules according to scene change parameters; and the screening module is configured to obtain the screening rules based on historical video data of the target area The scene change parameter of the target area, the scene change parameter is used to indicate the stability of the target area; the reference image of the target area is obtained according to the scene change parameter of the target area, and according to the scene change of the target area The parameters select the target frame image from the target video.
13. The apparatus according to claim 12, wherein the screening module is configured to obtain a historical video of the target area when the scene change parameter indicates that the stability of the target area is greater than or equal to a preset stable value There is no multi-frame image of the target object in the data; the pixel information of the multi-frame image is averaged to obtain the reference image of the target area; from the target video at the first preset time interval Select an image, and one frame at a time, use the selected image as the target frame image.
14. The apparatus according to claim 12, wherein the screening module is configured to, when the scene change parameter indicates that the stability of the target area is less than a preset stable value, to select Select images in the target video, and select two frames of images at a time; use the first frame of the two frames as the reference image of the target area, and the second frame of the two frames as the selected image In the target frame image, the shooting time of the first frame image is earlier than the shooting time of the second frame image.
15. The apparatus according to claim 13 or 14, wherein the comparison module is configured to divide the target frame image and the reference image into multiple sub-images respectively according to a preset division rule; Feature information of each sub-image in the target frame image, and feature information of each sub-image in the reference image; the feature information of each sub-image in the target frame image and the corresponding sub-image in the reference image Compare the feature information of to obtain the matching degree between each sub-image in the target frame image and the corresponding sub-image in the reference image; weight and sum the determined matching degrees to obtain the target The degree of matching between the frame image and the reference image.
16. The device of claim 10, further comprising:

    An acquisition module for acquiring a training image matching the target frame image from a database; acquiring object information in the training image from the database, the database includes multiple training images, and each training Object information in the image;

    The output module is configured to use the object information in the training image as the object information of the target object in the target frame image; and output the object information of the target object.
17. The device according to claim 10, characterized in that

    The output module is configured to output prompt information when it is determined that the target object intrudes in the target area, and the prompt information is used to prompt the target object intrusion in the target area.
18. The device according to claim 10, characterized in that

    The shooting module is configured to shoot the target area through the shooting device when the time is within a preset time period to obtain target video data of the target area. The preset time period refers to the location where the target area appears The time period when the frequency of the target object is greater than the preset frequency.
19. A monitoring device, characterized in that it includes:

    A processor, suitable for implementing one or more instructions; and,

    A computer-readable storage medium, the computer-readable storage medium stores one or more instructions, the one or more instructions are adapted to be loaded and executed by the processor according to any one of claims 1-9 Image processing method based on image recognition.
20. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more instructions, the one or more instructions are suitable for being loaded by a processor and executed as claimed in any one of claims 1-9 Item-based image processing method based on image recognition.

Images