CN114445769A - Fishing behavior detection method, device and system - Google Patents

Abstract

The application provides a fishing behavior detection method, a fishing behavior detection device and a fishing behavior detection system, which are used for solving the problems that the method for detecting a fishing rod and a person in the existing fishing behavior detection method is low in accuracy and robustness and high in false alarm rate of a mode for determining a fishing behavior, and the method comprises the following steps: acquiring an image to be detected from a video acquired aiming at a fishing forbidden region; performing target detection on an image to be detected based on a target detection model to obtain a detected image, and determining whether a first fishing rod target set and a first human body target set exist in the detected image; if the first set of fishing rod objects and the first set of human objects exist, it is determined whether a set of fishing behavior objects exists in the detected image.

Description

Method, device and system for detecting fishing behavior

technical field

The present application relates to the technical field of image processing, and in particular, to a method, device and system for detecting fishing behavior.

Background technique

Fishing is a popular leisure method, but some areas are prohibited from fishing due to safety considerations or management needs, such as scenic lakes, privately contracted reservoirs, etc. These prohibited fishing areas generally set warnings to drive away fishermen. sign or arrange for management to patrol.

At present, due to the ineffective driving effect of warning signs, it wastes manpower and material resources to arrange management personnel to patrol. Therefore, real-time monitoring of prohibited fishing areas is often performed. The monitoring pictures of fishing rods and people are sent to managers, thereby enhancing the driving effect and saving manpower and material resources. However, using traditional image processing methods to detect fishing rods and people has low accuracy and weak robustness, and only detects fishing rods and people to determine the existence of fishing behavior, and the false positive rate is high.

It can be seen that the existing fishing behavior detection methods have the problems of low accuracy and weak robustness in detecting fishing rods and people, and high false positive rate in the way of determining fishing behavior.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a fishing behavior detection method, device, and system, which are used to solve the problem that the existing fishing behavior detection methods have low accuracy and weak robustness for detecting fishing rods and people, and determine the fishing behavior. The way the false positive rate is higher.

In the first aspect, in order to solve the above technical problems, the embodiments of the present application provide a method for detecting phishing behavior, including:

Obtain the image to be detected;

Perform target detection on the image to be detected based on the target detection model, obtain a detected image, and determine whether there are a first fishing rod target set and a first human target set in the detected image; wherein, in the detected image The position parameters of the detection frame include the coordinates, width, height and inclination angle of the center point, and the inclination angle is the angle between the detection frame and the horizontal axis of the detected image;

If there is a first fishing rod target set and a first human body target set, determine whether there is a fishing behavior target set in the detected image; wherein, the fishing behavior target set includes any fish in the first fishing rod target set The rod target and the first human target set associated with any human target.

In this embodiment of the present application, an image to be detected may be acquired, target detection may be performed on the image to be detected based on a target detection model, a detected image may be obtained, and it may be determined whether a first fishing rod target set and a first human target set exist in the detected image, Among them, the position parameters of the detection frame in the detected image include the coordinates, width, height and inclination angle of the center point, and the inclination angle is the angle between the detection frame and the horizontal axis of the detected image. If there is a first fishing rod target set and For the first human target set, it is determined whether there is a fishing behavior target set in the detected image, wherein the fishing behavior target set includes any fishing rod target in the first fishing rod target set and any human target in the associated first human target set. The position of the detection frame of the fishing rod target and the human target in the detected image is indicated by the position parameters such as the coordinates, width, height and inclination angle of the center point, so as to improve the accuracy and robustness of the detection results of the fishing rod target and the human target. and determine whether the fishing rod target in the detected image is related to the human target, so as to reduce the false alarm rate of fishing behavior in prohibited fishing areas.

In an optional embodiment, the target detection is performed on the to-be-detected image based on the target detection model, and before the detected image is obtained, the method further includes:

Obtain at least one sample image; wherein, the position parameters of the real frame of the fishing rod target and the human target in the sample image include the coordinates of the center point, the width, the height and the inclination angle, and the inclination angle is the real frame and the inclination angle. The angle between the horizontal axis of the sample image;

A preset detection model is trained based on the at least one sample image to obtain the target detection model; wherein, the target detection model is used to detect a fishing rod target and a human target in the image.

In an optional implementation manner, determining whether there is a phishing behavior target set in the detected image includes:

Deleting the first fishing rod target set with a confidence level less than the first preset threshold value, and the first human body target set with a confidence level less than the second preset threshold value, to obtain a second rod target set and the second human target set;

determining the intersection ratio of the detection frame of any fishing rod target in the second fishing rod target set and the detection frame of any human body target in the second human target set;

If the intersection ratio is greater than a third preset threshold, correlate the fishing rod target and the human body target corresponding to the intersection ratio, and determine that the fishing behavior target set exists in the detected image;

If the intersection ratio is not greater than the third preset threshold, determine the distance between the fishing rod target and the human target corresponding to the intersection ratio; wherein, the distance is each of the detection frames of the fishing rod target. The minimum Euclidean distance between the vertex and each edge of the detection frame of the human target;

If the distance is not greater than the fourth preset threshold, the fishing rod target and the human body target corresponding to the intersection ratio are associated, and it is determined that the fishing behavior target set exists in the detected image.

In an optional embodiment, determining the intersection ratio of the detection frame of any fishing rod target in the second fishing rod target set and the detection frame of any human target in the second human target set includes:

Obtain the position parameter of the detection frame of any fishing rod target in the second fishing rod target set and the position parameter of the detection frame of any human body target in the second human target set;

Based on the position parameter of the detection frame of any fishing rod target and the position parameter of the detection frame of any human target, determine the intersection of the detection frame of any fishing rod target and the detection frame of any human target coordinate;

Based on the coordinates of the intersection point, the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target is determined.

In an optional embodiment, based on the coordinates of the intersection point, determining the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target, including:

Based on the coordinates of the intersection point, a preset formula is used to determine the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target;

The preset formula is specifically:

为所述任一鱼竿目标的检测框的面积，

为所述任一人体目标的检测框的面积，(m_i,n_i)为第i个所述交点的坐标，n为所述交点的个数。Wherein, IOU is the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target,

is the area of the detection frame of any fishing rod target,

is the area of the detection frame of any human target, (m _i , n _i ) is the coordinate of the i-th intersection point, and n is the number of the intersection points.

In an optional embodiment, it also includes:

If the first human body target set does not exist or the fishing behavior target set does not exist, sending the detected image to the manager;

If the phishing behavior target set exists, the detected image will be sent to the management personnel, and a warning voice will be played in the area where phishing is prohibited.

In a second aspect, the embodiments of the present application further provide a device for detecting fishing behavior, including:

an acquisition module for acquiring the image to be detected;

A detection module, configured to perform target detection on the to-be-detected image based on a target detection model, obtain a detected image, and determine whether there is a first fishing rod target set and a first human target set in the detected image; The position parameter of the detection frame in the detected image includes the coordinates, width, height and tilt angle of the center point, and the tilt angle is the angle between the detection frame and the horizontal axis of the detected image;

A determination module, configured to determine whether there is a fishing behavior target set in the detected image if there is a first fishing rod target set and a first human body target set; wherein, the fishing behavior target set includes the first fishing rod Any fishing rod target in the target set and any human target in the associated first human target set.

In an optional embodiment, the device further includes a training module for:

Obtain at least one sample image; wherein, the position parameters of the real frame of the fishing rod target and the human target in the sample image include the coordinates of the center point, the width, the height and the inclination angle, and the inclination angle is the real frame and the inclination angle. The angle between the horizontal axis of the sample image;

A preset detection model is trained based on the at least one sample image to obtain the target detection model; wherein, the target detection model is used to detect a fishing rod target and a human target in the image.

In an optional implementation manner, the determining module is specifically used for:

Deleting the first fishing rod target set with a confidence level less than the first preset threshold value, and the first human body target set with a confidence level less than the second preset threshold value, to obtain a second rod target set and the second human target set;

determining the intersection ratio of the detection frame of any fishing rod target in the second fishing rod target set and the detection frame of any human body target in the second human target set;

If the intersection ratio is greater than a third preset threshold, correlate the fishing rod target and the human body target corresponding to the intersection ratio, and determine that the fishing behavior target set exists in the detected image;

If the intersection ratio is not greater than the third preset threshold, determine the distance between the fishing rod target and the human target corresponding to the intersection ratio; wherein, the distance is each of the detection frames of the fishing rod target. The minimum Euclidean distance between the vertex and each edge of the detection frame of the human target;

If the distance is not greater than the fourth preset threshold, the fishing rod target and the human body target corresponding to the intersection ratio are associated, and it is determined that the fishing behavior target set exists in the detected image.

In an optional implementation manner, the determining module is specifically used for:

Obtain the position parameter of the detection frame of any fishing rod target in the second fishing rod target set and the position parameter of the detection frame of any human body target in the second human target set;

Based on the position parameter of the detection frame of any fishing rod target and the position parameter of the detection frame of any human target, determine the intersection of the detection frame of any fishing rod target and the detection frame of any human target coordinate;

Based on the coordinates of the intersection point, the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target is determined.

In an optional implementation manner, the determining module is specifically used for:

Based on the coordinates of the intersection point, a preset formula is used to determine the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target;

The preset formula is specifically:

为所述任一鱼竿目标的检测框的面积，

为所述任一人体目标的检测框的面积，(m_i,n_i)为第i个所述交点的坐标，n为所述交点的个数。Wherein, IOU is the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target,

is the area of the detection frame of any fishing rod target,

is the area of the detection frame of any human target, (m _i , n _i ) is the coordinate of the i-th intersection point, and n is the number of the intersection points.

In an optional implementation manner, the apparatus further includes a processing module for:

If the first human body target set does not exist or the fishing behavior target set does not exist, sending the detected image to the manager;

If the phishing behavior target set exists, the detected image will be sent to the management personnel, and a warning voice will be played in the area where phishing is prohibited.

In a third aspect, an embodiment of the present application further provides a system for detecting phishing behavior, including:

memory for storing program instructions;

The processor is configured to call the program instructions stored in the memory, and execute the steps included in any one of the implementation manners of the first aspect according to the obtained program instructions.

In a fourth aspect, an embodiment of the present application provides a storage medium, where the storage medium stores computer-executable instructions, where the computer-executable instructions are used to cause a computer to execute the steps included in any one of the implementation manners in the first aspect.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are only for the present application. some examples.

1 is a schematic structural diagram of a phishing behavior detection system according to an embodiment of the present application;

2a is a schematic flowchart of a method for detecting phishing behavior provided by an embodiment of the present application;

2b is a schematic diagram of a detected image provided by an embodiment of the present application;

2c is a schematic diagram of the intersection of a detection frame of a fishing rod target and a detection frame of a human target provided by an embodiment of the present application;

3 is a schematic structural diagram of a device for detecting fishing behavior according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of another phishing behavior detection system provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application. The embodiments in the present application and the features in the embodiments may be arbitrarily combined with each other if there is no conflict. Also, although a logical order is shown in the flowcharts, in some cases steps shown or described can be performed in an order different from that herein.

The terms "first" and "second" in the description and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protections. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

In the embodiments of the present application, "at least one" may mean at least two, for example, two, three or more, which are not limited in the embodiments of the present application.

In addition, the term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, There are three cases of B alone. In addition, the character "/" in this text, unless otherwise specified, generally indicates that the related objects before and after are an "or" relationship.

At present, due to the ineffective driving effect of warning signs, it wastes manpower and material resources to arrange management personnel to patrol. Therefore, real-time monitoring of prohibited fishing areas is often performed. The monitoring pictures of fishing rods and people are sent to managers, thereby enhancing the driving effect and saving manpower and material resources. However, using traditional image processing methods to detect fishing rods and people has low accuracy and weak robustness, and only detects fishing rods and people to determine the existence of fishing behavior, and the false positive rate is high. It can be seen that the existing fishing behavior detection methods have the problems of low accuracy and weak robustness in detecting fishing rods and people, and high false positive rate in the way of determining fishing behavior.

In view of this, an embodiment of the present application provides a method for detecting fishing behavior, which can acquire an image to be detected, perform target detection on the image to be detected based on a target detection model, obtain a detected image, and determine whether there is a first fish in the detected image. The rod target set and the first human target set, wherein the position parameters of the detection frame in the detected image include the coordinates, width, height and inclination angle of the center point, and the inclination angle is the angle between the detection frame and the horizontal axis of the detected image , if there are the first fishing rod target set and the first human body target set, determine whether there is a fishing behavior target set in the detected image, wherein the fishing behavior target set includes any fishing rod target in the first fishing rod target set and the associated The first human target set includes any human target. The position of the detection frame of the fishing rod target and the human target in the detected image is indicated by the position parameters such as the coordinates, width, height and inclination angle of the center point, so as to improve the accuracy and robustness of the detection results of the fishing rod target and the human target. and determine whether the fishing rod target in the detected image is related to the human target, so as to reduce the false alarm rate of fishing behavior in prohibited fishing areas.

In order to better understand the above technical solutions, the technical solutions of the present application will be described in detail below through the accompanying drawings and specific embodiments of the description. Rather than limiting the technical solutions of the present application, the embodiments of the present application and the technical features in the embodiments may be combined with each other without conflict.

As shown in FIG. 1, it is a schematic structural diagram of a fishing behavior detection system provided by an embodiment of the present application. Of course, the method provided by the embodiment of the present application can be applied to a variety of fishing behavior detection systems. It should be understood that the system shown in FIG. 1 The phishing behavior detection system is a simple description of the phishing behavior detection system applicable to the method provided by the embodiment of the present application, rather than a limitation on the phishing behavior detection system applicable to the method provided by the embodiment of the present application.

The phishing behavior detection system shown in FIG. 1 includes a memory 101 , a processor 102 , and a bus interface 103 . The memory 101 and the processor 102 are connected through a bus interface 103 . Memory 101 is used to store program instructions. The processor 102 is configured to call the program instructions stored in the memory 101, and execute all steps included in the method for detecting phishing behavior according to the obtained program instructions.

It should be noted that, in the embodiment of the present application, the fishing behavior detection system may be installed in a smart camera, and the smart camera is mainly used to monitor areas where fishing is prohibited, such as lakes and privately contracted reservoirs in scenic spots.

As shown in FIG. 2a , a schematic flowchart of a method for detecting a phishing behavior provided in an embodiment of the present application, the method may be executed by the phishing behavior detection system shown in FIG. 1 . The specific flow of the method is described as follows.

Step 201: Acquire an image to be detected.

In this embodiment of the present application, the phishing behavior detection system may obtain the image to be detected from the video collected by the smart camera for the fishing-prohibited area. The image code stream is sampled based on the preset time interval to obtain the sampled image code stream. Then, since the storage space of the image in YUV format is smaller than that of the image in RGB format, the format conversion process is performed on the image code stream after sampling processing. The image code stream after format conversion processing is obtained, and the image to be detected is obtained from the image code stream after format conversion processing.

Step 202 : perform target detection on the image to be detected based on the target detection model, obtain a detected image, and determine whether there is a first fishing rod target set and a first human target set in the detected image.

In the embodiment of the present application, after acquiring the to-be-detected image from the video collected by the smart camera for the prohibited fishing area, the phishing behavior detection system may perform target detection on the to-be-detected image based on the target detection model, obtain the detected image, and determine that the detected image has been detected. Whether there is a first fishing rod target set and a first human target set in the image, where the position parameters of the detection frame in the detected image include the coordinates, width, height and inclination angle of the center point, and the inclination angle is the detection frame and the detected frame. The angle between the horizontal axis of the image;

Exemplarily, as shown in FIG. 2b, which is a schematic diagram of a detected image provided by an embodiment of the present application, the position parameters of the detection frames of the human target and the fishing rod target in FIG. 2b are both (C _x , C _y , w,h,θ), where (C _x ,C _y ) are the coordinates of the center point of the detection frame, w is the width of the detection frame, h is the height of the detection frame, and θ is the inclination angle of the detection frame (that is, the detection frame and the angle between the horizontal axis of the image), since the detection frame of the human target is generally parallel to the horizontal axis, the position parameter θ of the detection frame of the human target can be fixed to 0. In addition, the position parameters of the detection frame can be converted from (C _x ,C _y ,w,h,θ) to (X ₀ ,Y ₀ ;X ₁ ,Y ₁ ;X ₂ ,Y ₂ ;X ₃ ,Y ₃ ) , where (X ₀ , Y ₀ ), (X ₁ , Y ₁ ), (X ₂ , Y ₂ ), (X ₃ , Y ₃ ) are the sequence (counterclockwise or clockwise) of each detection frame The coordinates of the vertices, such as (X ₀ , Y ₀ ) are the coordinates of the upper left corner of the detection frame, (X ₁ , Y ₁ ) are the coordinates of the upper right corner of the detection frame, and (X ₂ , Y ₂ ) are the lower right corner of the detection frame. The coordinates of (X ₃ , Y ₃ ) are the coordinates of the lower left corner of the detection frame.

其中，t_θ为检测框的倾斜角度θ的改正值，

为真实框的倾斜角度θ^*的改正值，θ_a为先验框的倾斜角度。预设检测模型的回归损失函数，具体为：It should be noted that, in this embodiment of the present application, before the target detection is performed on the image to be detected based on the target detection model, and the detected image is obtained, at least one sample image may be obtained, wherein the real fishing rod target and the human target in the sample image are The position parameters of the frame include the coordinates, width, height and inclination angle of the center point, and the inclination angle is the angle between the real frame and the horizontal axis of the image. A target detection model for fishing rod targets and human targets. Among them, the preset detection model is a detection model based on a convolutional neural network, such as a regional convolutional neural network (Region Convolutional Neural Network, R-CNN), YOLO (You Only Look Once), a single-shot multi-box detector (Single Shot MultiBox Detector, SSD), etc. The regression parameters of the preset detection model include the coordinates (C _x , C _y ) of the center point of the detection frame, the width w of the detection frame, the height h of the detection frame and the inclination angle θ of the detection frame, and the regression of the inclination angle θ of the detection frame The formula is

Among them, t _θ is the correction value of the inclination angle θ of the detection frame,

is the corrected value of the inclination angle θ ^* of the real frame, and θ _a is the inclination angle of the prior frame. The regression loss function of the preset detection model, specifically:

Among them, L _r is the regression loss, f( ) is the logarithmic function or square root function, m ₁ is the coordinates of the center point of the detection frame, and m ₂ is the coordinates of the center point of the real frame.

Step 203: If there are the first fishing rod target set and the first human body target set, determine whether there is a fishing behavior target set in the detected image.

In the embodiment of the present application, after the target detection is performed on the image to be detected based on the target detection model, the detected image is obtained, and it is determined whether the first fishing rod target set and the first human target set exist in the detected image, if the fishing behavior detection The system determines that there are the first fishing rod target set and the first human target set in the detected image, and then determines whether there is a fishing behavior target set in the detected image, wherein the fishing behavior target set includes any fishing rod in the first fishing rod target set The target and the associated first human target set are any human target.

Specifically, the first fishing rod target set whose confidence is less than the first preset threshold is deleted, and the first human body target whose confidence is less than the second preset threshold is deleted to obtain the second fishing rod target set and the first Two human target sets, determine the intersection ratio between the detection frame of any fishing rod target in the second fishing rod target set and the detection frame of any human target in the second human target set, if the intersection ratio is greater than the third preset threshold, then Correlate the corresponding fishing rod target and human body target, and determine that there is a fishing behavior target set in the detected image. distance, where the distance is the minimum Euclidean distance between each vertex of the detection frame of the fishing rod target and each edge of the detection frame of the human target, if the distance is not greater than the fourth preset threshold, then the relative intersection is compared with the corresponding fishing rod Target and human target, determine the existence of fishing behavior target set in the detected image.

It should be noted that, in the implementation of this application, when the intersection ratio of the detection frame of any fishing rod target in the second fishing rod target set and the detection frame of any human target in the second human target set is determined, the second fish can be obtained. The position parameter of the detection frame of any fishing rod target in the rod target set and the position parameter of the detection frame of any human target in the second human target set, based on the position parameter of the detection frame of any fishing rod target and the detection of any human target The position parameter of the frame determines the coordinates of the intersection of the detection frame of any fishing rod target and the detection frame of any human target. Based on the coordinates of the intersection, a preset formula is used to determine the detection frame of any fishing rod target and any human target. The intersection ratio of the detection frame, where the preset formula is:

为任一鱼竿目标的检测框的面积，

为任一人体目标的检测框的面积，(m_i,n_i)为第i个交点的坐标，n为交点的个数。Among them, IOU is the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target,

is the area of the detection frame of any fishing rod target,

is the area of the detection frame of any human target, (m _i , n _i ) is the coordinate of the ith intersection point, and n is the number of intersection points.

Exemplarily, as shown in FIG. 2c , which is a schematic diagram of the intersection of the detection frame of any fishing rod target and the detection frame of any human target provided by the embodiment of the application, the detection frame of any fishing rod target is divided into The position parameter and the position parameter of the detection frame of any human target are converted from (C _x , _Cy , w, h, θ) to (X ₀ , Y ₀ ; X ₁ , Y ₁ ; X ₂ , Y ₂ ; X _{3 )} , Y ₃ ), the detection frame B ₁ of any fishing rod target (X ₀ , Y ₀ ; X ₁ , Y ₁ ; X ₂ , Y ₂ ; X ₃ , Y ₃ ), the detection frame B of any human target ₂ (X ₀ , Y ₀ ; X ₁ , Y ₁ ; X ₂ , Y ₂ ; X ₃ , Y ₃ ), based on two points to determine a straight line and two intersecting straight lines must have an intersection, to determine the detection of any fishing rod target The coordinates of the intersection of the position parameter of the frame and the detection frame of any human target are (m ₁ , n ₁ ), (m ₂ , n ₂ ), (m ₃ , n ₃ ), (m ₄ , n ₄ ), ( m ₅ , n ₅ ), (m ₆ , n ₆ ), (m ₇ , n ₇ ), (m ₈ , n ₈ ).

After performing target detection on the image to be detected based on the target detection model, obtaining a detected image, and determining whether the first fishing rod target set and the first human target set exist in the detected image, if the fishing behavior detection system determines that the detected image does not exist in the detected image If there is the first fishing rod target set or the first fishing rod target set and the first human body target set but no fishing behavior target set, only the detected image is sent to the management personnel, and the warning voice is not played in the fishing prohibited area.

After performing target detection on the image to be detected based on the target detection model, obtaining the detected image, and determining whether the first fishing rod target set and the first human target set exist in the detected image, if the fishing behavior detection system determines that the detected image exists If the first fishing rod target set and the first human body target set exist and there is a fishing behavior target set, the detected image will be sent to the management personnel, and a warning voice will be played in the fishing prohibited area.

In the above scheme, the fishing behavior detection system indicates the position of the detection frame of the fishing rod target and the human target in the detected image through the position parameters such as the coordinates, width, height and inclination angle of the center point, so as to improve the detection of the fishing rod target and the human target. The accuracy and robustness of the results, and when it is determined that there is an associated rod target and human target in the detected image, the detected image is sent to the manager, and a warning voice is played in the no-fishing area, and the detected image is determined. When there is only a fishing rod target, only the detected image will be sent to the management personnel, and the warning voice will not be played in the prohibited fishing area, so as to reduce the false alarm rate of fishing behavior in the prohibited fishing area.

Based on the same inventive concept, an embodiment of the present application further provides a fishing behavior detection device, which can be applied to the fishing behavior detection system shown in FIG. 1 above. The phishing behavior detection device can implement the functions corresponding to the foregoing phishing behavior detection methods. The phishing behavior detection device may be a hardware structure, a software module, or a hardware structure plus a software module. The phishing behavior detection device can be implemented by a chip system, and the chip system can be composed of chips, and can also include chips and other discrete devices. Please refer to FIG. 3 , which is a schematic structural diagram of a device for detecting phishing behavior according to an embodiment of the present application. The device for detecting phishing behavior includes an acquisition module 301 , a detection module 302 , a determination module 303 , a first processing module 304 and a second processing module 305.

an acquisition module 301, configured to acquire an image to be detected;

The detection module 302 is configured to perform target detection on the to-be-detected image based on a target detection model, obtain a detected image, and determine whether there is a first fishing rod target set and a first human target set in the detected image; wherein, The position parameters of the detection frame in the detected image include the coordinates, width, height and tilt angle of the center point, and the tilt angle is the angle between the detection frame and the horizontal axis of the detected image;

The determining module 303 is configured to determine whether there is a fishing behavior target set in the detected image if there is a first fishing rod target set and a first human target set; wherein, the fishing behavior target set includes the first fish Any rod target in the rod target set and any human target in the associated first human target set.

In an optional embodiment, the device further includes a training module for:

Obtain at least one sample image; wherein, the position parameters of the real frame of the fishing rod target and the human target in the sample image include the coordinates of the center point, the width, the height and the inclination angle, and the inclination angle is the real frame and the inclination angle. The angle between the horizontal axis of the sample image;

A preset detection model is trained based on the at least one sample image to obtain the target detection model; wherein, the target detection model is used to detect a fishing rod target and a human target in the image.

In an optional implementation manner, the determining module 303 is specifically configured to:

Deleting the first fishing rod target set with a confidence level less than the first preset threshold value, and the first human body target set with a confidence level less than the second preset threshold value, to obtain a second rod target set and the second human target set;

determining the intersection ratio of the detection frame of any fishing rod target in the second fishing rod target set and the detection frame of any human body target in the second human target set;

If the intersection ratio is greater than a third preset threshold, correlate the fishing rod target and the human body target corresponding to the intersection ratio, and determine that the fishing behavior target set exists in the detected image;

If the intersection ratio is not greater than the third preset threshold, determine the distance between the fishing rod target and the human target corresponding to the intersection ratio; wherein, the distance is each of the detection frames of the fishing rod target. The minimum Euclidean distance between the vertex and each edge of the detection frame of the human target;

If the distance is not greater than the fourth preset threshold, the fishing rod target and the human body target corresponding to the intersection ratio are associated, and it is determined that the fishing behavior target set exists in the detected image.

In an optional implementation manner, the determining module 303 is specifically configured to:

Obtain the position parameter of the detection frame of any fishing rod target in the second fishing rod target set and the position parameter of the detection frame of any human body target in the second human target set;

Based on the position parameter of the detection frame of any fishing rod target and the position parameter of the detection frame of any human target, determine the intersection of the detection frame of any fishing rod target and the detection frame of any human target coordinate;

Based on the coordinates of the intersection point, the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target is determined.

In an optional implementation manner, the determining module 303 is specifically configured to:

Based on the coordinates of the intersection point, a preset formula is used to determine the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target;

The preset formula is specifically:

为所述任一鱼竿目标的检测框的面积，

为所述任一人体目标的检测框的面积，(m_i,n_i)为第i个所述交点的坐标，n为所述交点的个数。Wherein, IOU is the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human target,

is the area of the detection frame of any fishing rod target,

is the area of the detection frame of any human target, (m _i , n _i ) is the coordinate of the i-th intersection point, and n is the number of the intersection points.

In an optional implementation manner, the apparatus further includes a processing module for:

If the first human body target set does not exist or the fishing behavior target set does not exist, sending the detected image to the manager;

If the phishing behavior target set exists, the detected image will be sent to the management personnel, and a warning voice will be played in the area where phishing is prohibited.

Based on the same inventive concept, an embodiment of the present application also provides a phishing behavior detection system. Please refer to FIG. 4 , which is a schematic structural diagram of a phishing behavior detection system provided by an embodiment of the present application. The phishing behavior detection system includes at least one processor 402, and the memory 401 connected to at least one processor, the specific connection medium between the processor 402 and the memory 401 is not limited in the embodiments of the present application. For example, the bus 400 is represented by a thick line in FIG. 4 , and the connection modes between other components are only for schematic illustration, and are not limited thereto. The bus 400 can be divided into an address bus, a data bus, a control bus, etc. For convenience of illustration, only one thick line is used in FIG. 4 , but it does not mean that there is only one bus or one type of bus.

In this embodiment of the present application, the memory 401 stores instructions that can be executed by at least one processor 402 , and the at least one processor 402 can execute the steps included in the foregoing fishing behavior detection method by calling the instructions stored in the memory 401 . Among them, the processor 402 is the control center of the phishing behavior detection system, which can use various interfaces and lines to connect various parts of the entire phishing behavior detection system, and execute the instructions stored in the memory 401, thereby realizing various types of phishing behavior detection systems. Function. Optionally, the processor 402 may include one or more processing units, and the processor 402 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem The modulation processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 402. In some embodiments, the processor 402 and the memory 401 may be implemented on the same chip, and in some embodiments, they may be implemented separately on separate chips.

As a non-volatile computer-readable storage medium, the memory 401 can be used to store non-volatile software programs, non-volatile computer-executable programs and modules. The memory 401 may include at least one type of storage medium, for example, may include a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a Programmable Read Only Memory (PROM), Read Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Memory, Disk, CD and so on. The memory 401 is, but is not limited to, any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory 401 in this embodiment of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and/or data.

In this embodiment of the present application, the processor 402 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic Devices and discrete hardware components can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the method for detecting phishing behavior disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

By designing and programming the processor 402, the code corresponding to the phishing behavior detection method described in the foregoing embodiment can be solidified into the chip, so that the chip can execute the steps of the foregoing phishing behavior detection method when it is running. The design and programming of the controller 402 is a well-known technology for those skilled in the art, and details are not described here.

Based on the same inventive concept, an embodiment of the present application further provides a storage medium, where computer instructions are stored in the storage medium, and when the computer instructions are run on a computer, the computer executes the steps of the aforementioned method for detecting phishing behavior.

In some possible implementations, various aspects of the phishing behavior detection method provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on the phishing behavior detection system, the program code uses In order to make the phishing behavior detection system execute the steps in the phishing behavior detection method according to various exemplary embodiments of the present application described above in this specification.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (10)

1. A fishing behavior detection method, comprising:

acquiring an image to be detected;

performing target detection on the image to be detected based on a target detection model to obtain a detected image, and determining whether a first fishing rod target set and a first human body target set exist in the detected image; the position parameters of the detection frame in the detected image comprise the coordinate, the width, the height and the inclination angle of a central point, and the inclination angle is an included angle between the detection frame and a horizontal axis of the detected image;

if a first fishing rod target set and a first human body target set exist, determining whether a fishing behavior target set exists in the detected image; wherein the set of fishing performance goals includes any fishing rod goal in the first set of fishing rod goals and any human goal in the associated first set of human goals.

2. The method of claim 1, wherein before performing the target detection on the image to be detected based on the target detection model to obtain the detected image, the method further comprises:

acquiring at least one sample image; the position parameters of the real frames of the fishing rod target and the human body target in the sample image comprise the coordinate, the width, the height and the inclination angle of a central point, and the inclination angle is an included angle between the real frame and a horizontal axis of the sample image;

training a preset detection model based on the at least one sample image to obtain the target detection model; the target detection model is used for detecting a fishing rod target and a human body target in the image.

3. The method of claim 1 or 2, wherein determining whether a set of phishing behavior objects is present in the detected image comprises:

deleting the fishing rod targets with the confidence coefficient smaller than a first preset threshold value in the first fishing rod target set and the human body targets with the confidence coefficient smaller than a second preset threshold value in the first human body target set to obtain a second fishing rod target set and a second human body target set;

determining the intersection and combination ratio of the detection frame of any fishing rod target in the second fishing rod target set and the detection frame of any human body target in the second human body target set;

if the intersection comparison is larger than a third preset threshold value, associating the corresponding fishing rod target and the human body target, and determining that the fishing behavior target set exists in the detected image;

if the intersection comparison is not larger than the third preset threshold, determining the distance between the corresponding fishing rod target and the human body target; wherein the distance is the minimum Euclidean distance between each vertex of the detection frame of the fishing rod target and each edge of the detection frame of the human body target;

and if the distance is not greater than a fourth preset threshold value, associating the intersection and comparing the corresponding fishing rod target and the human body target, and determining that the fishing behavior target set exists in the detected image.

4. The method of claim 3, wherein determining an intersection ratio of a detection frame of any one of the second set of fishing rod targets and a detection frame of any one of the second set of human targets comprises:

acquiring position parameters of a detection frame of any one fishing rod target in the second fishing rod target set and position parameters of a detection frame of any one human target in the second human target set;

determining coordinates of intersection points of the detection frame of any fishing rod target and the detection frame of any human body target based on the position parameters of the detection frame of any fishing rod target and the position parameters of the detection frame of any human body target;

and determining the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human body target based on the coordinates of the intersection points.

5. The method of claim 4, wherein determining an intersection ratio of the detection frame of the any fishing rod target and the detection frame of the any human target based on coordinates of the intersection point comprises:

determining the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human body target by adopting a preset formula based on the coordinates of the intersection points;

the preset formula specifically includes:

wherein the IOU is the intersection ratio of the detection frame of any fishing rod target and the detection frame of any human body target,

the area of the detection frame of any fishing rod target,

the area of the detection frame of any human target (m)_i,n_i) Is the coordinate of the ith intersection point, and n is the number of the intersection points.

6. The method of claim 1, further comprising:

if the first human body target set does not exist or the phishing behavior target set does not exist, the detected image is sent to a manager;

and if the fishing behavior target set exists, sending the detected image to the manager, and playing warning voice in a fishing-forbidden area.

7. A fishing behavior detection device, comprising:

the acquisition module is used for acquiring an image to be detected;

the detection module is used for carrying out target detection on the image to be detected based on a target detection model to obtain a detected image and determining whether a first fishing rod target set and a first human body target set exist in the detected image or not; the position parameters of the detection frame in the detected image comprise the coordinate, the width, the height and the inclination angle of a central point, and the inclination angle is an included angle between the detection frame and a horizontal axis of the detected image;

a determining module, configured to determine whether a fishing behavior target set exists in the detected image if a first fishing rod target set and a first human body target set exist; wherein the set of fishing performance goals includes any fishing rod goal in the first set of fishing rod goals and any human goal in the associated first set of human goals.

8. The apparatus of claim 7, wherein the apparatus further comprises a training module to:

acquiring at least one sample image; the position parameters of the real frames of the fishing rod target and the human body target in the sample image comprise the coordinate, the width, the height and the inclination angle of a central point, and the inclination angle is an included angle between the real frame and a horizontal axis of the sample image;

training a preset detection model based on the at least one sample image to obtain the target detection model; the target detection model is used for detecting a fishing rod target and a human body target in the image.

9. A fishing behavior detection system, comprising:

a memory for storing program instructions;

a processor for calling program instructions stored in said memory and for executing the steps comprised by the method of any one of claims 1 to 6 in accordance with the obtained program instructions.

10. A storage medium storing computer-executable instructions for causing a computer to perform the steps comprising the method of any one of claims 1-6.

Images