CN117746021A

CN117746021A - Target detection method and device based on infrared sequence image, electronic equipment and medium

Info

Publication number: CN117746021A
Application number: CN202311761159.0A
Authority: CN
Inventors: 刘雨菡; 周光尧; 胡玉新; 刘昭然
Original assignee: Aerospace Information Research Institute of CAS
Current assignee: Aerospace Information Research Institute of CAS
Priority date: 2023-12-20
Filing date: 2023-12-20
Publication date: 2024-03-22

Abstract

The disclosure provides a target detection method, device, equipment and medium based on infrared sequence images, which can be applied to the infrared target detection field and the infrared remote sensing image processing field. The method comprises the following steps: threshold segmentation is carried out on the obtained infrared sequence image to obtain an alternative point set, wherein the infrared sequence image is suitable for representing a moving target object; processing the candidate point set by using the distance screening condition to obtain a plurality of initial tracks; determining effective points from the initial track points according to the position relation between the initial track points in the initial track; determining the confidence coefficient of the initial track according to the number of the effective points in the initial track; and determining a target track representing the motion state of the object to be detected from the plurality of initial tracks according to the confidence level.

Description

Target detection method and device based on infrared sequence image, electronic equipment and medium

Technical Field

The present disclosure relates to the field of infrared target detection and infrared remote sensing image processing, and more particularly, to a target detection method, device, electronic apparatus, and medium based on infrared sequential images.

Background

For the problem of target detection in infrared remote sensing images, currently mainly adopted technologies include detection based on salient features, detection based on background consistency assumption and detection based on low-rank sparse decomposition. For the infrared remote sensing image, on one hand, a large number of false alarm sources with strong radiation characteristics as the targets are detected while the targets are successfully detected, so that the detected false alarm rate is higher; on the other hand, for remote sensing images with relatively large image sizes, techniques such as dicing and sliding window are required to have the expected effect, so that the demand of computing resources is high.

Disclosure of Invention

In view of the foregoing, the present disclosure provides an infrared sequence image-based object detection method, apparatus, device, medium, and program product.

According to a first aspect of the present disclosure, there is provided an infrared sequence image-based object detection method, including: threshold segmentation is carried out on the obtained infrared sequence image to obtain an alternative point set, wherein the infrared sequence image is suitable for representing a moving target object; processing the alternative point set by using a distance screening condition to obtain a plurality of initial tracks; determining effective points from the initial track points according to the position relation between the initial track points in the initial track; determining the confidence coefficient of the initial track according to the number of the effective points in the initial track; and determining a target track representing the motion state of the object to be detected from a plurality of initial tracks according to the confidence level.

According to an embodiment of the present disclosure, the target detection method based on the infrared sequence image further includes: carrying out image difference calculation on infrared sequence images of different frames in the multi-frame infrared sequence images to obtain frame difference infrared sequence images; the method for obtaining the candidate point set comprises the following steps of: and carrying out threshold segmentation on the frame difference infrared sequence image to obtain the alternative point set in the frame difference infrared sequence image.

According to an embodiment of the present disclosure, the processing the candidate point set using the distance screening condition to obtain a plurality of initial trajectories includes: if at least two candidate points in the candidate point set meet a threshold rule, determining the candidate points meeting the threshold rule as target candidate points; and determining the initial track according to the position relation between the target candidate points.

According to an embodiment of the present disclosure, in a case where the initial candidate point set satisfies a threshold rule, determining the target candidate point set includes: the threshold rule characterizes: among the candidate points of the multi-frame infrared sequence images, the distance between the reference candidate point in the reference frame infrared sequence image and the adjacent candidate point in the adjacent frame infrared sequence image is larger than or equal to a first distance threshold value and smaller than or equal to a second distance threshold value, wherein the first distance threshold value is smaller than the second distance threshold value; or the distance between the reference candidate point in the infrared sequence image of the reference frame and the adjacent candidate point in the infrared sequence image of the adjacent frame in the candidate points of the infrared sequence images of the plurality of frames is smaller than or equal to a third distance threshold.

According to an embodiment of the present disclosure, determining an effective point from the initial track points according to a positional relationship of the initial track points in the initial track includes: calculating a kth initial track length between a kth initial track point and a 0 th initial track point of the initial track aiming at a kth initial track point in the initial track, wherein the kth initial track length is represented based on the number of image pixel points passing from the 0 th initial track point to the kth track point in the initial track, wherein k is more than or equal to 1, and the 0 th initial track point is a starting track point of the initial track; and determining the kth initial trajectory point as the effective point when the kth initial trajectory length indicates that the number of image pixel points passing from the 0 th initial trajectory point to the kth trajectory point is greater than the number of initial trajectory points passing from the 0 th initial trajectory point to the kth trajectory point in the initial trajectory;

according to an embodiment of the present disclosure, before determining the confidence level, further comprising: and establishing a track confidence coefficient formula based on the number of the effective points of the initial track. The track confidence coefficient formula is shown as formula (1):

Wherein Cred is the confidence, C is a constant, and M is the number of effective points in the initial trajectory.

According to an embodiment of the present disclosure, determining, from among a plurality of the initial trajectories, a target trajectory characterizing a motion state of the object to be detected according to the confidence level, includes: determining a confidence coefficient curve according to the confidence coefficient corresponding to each of the plurality of initial tracks; determining a confidence coefficient threshold according to the curvature change relation of the confidence coefficient curve; and determining the target track from a plurality of initial tracks according to comparison results between the confidence threshold and a plurality of confidence levels.

Determining a confidence coefficient curve according to the confidence coefficient corresponding to each of the plurality of initial tracks; determining a confidence coefficient threshold according to the curvature change relation of the confidence coefficient curve; and determining the target track from the initial tracks according to the comparison result between the confidence threshold and the confidence levels.

A second aspect of the present disclosure provides an object detection method apparatus based on an infrared sequence image, including:

the alternative point set module is used for carrying out threshold segmentation on the obtained infrared sequence image to obtain an alternative point set, wherein the infrared sequence image is suitable for representing a moving target object;

The initial track obtaining module is used for processing the alternative point set by utilizing the distance screening condition to obtain a plurality of initial tracks;

the effective point determining module is used for determining effective points from the initial track points according to the position relation between the initial track points in the initial track;

the confidence coefficient determining module is used for determining the confidence coefficient of the initial track according to the number of the effective points in the initial track; and

and the target track determining module is used for determining a target track representing the motion state of the object to be detected from a plurality of initial tracks according to the confidence coefficient.

A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method.

A fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described method.

A fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above method.

According to the target detection method, the device, the electronic equipment and the medium based on the infrared sequence image, the candidate point set is extracted on the basis of the high radiation characteristic of the target object, and the target object is associated to form a track according to the position relation between the initial track points in the initial track by combining the time domain motion characteristic. Because of the low resolution and poor quality of the infrared image, a large number of alternative points are generated in the detection process, so that a plurality of tracks are formed. In order to reduce the false alarm, the candidate points of the target track can be screened according to the position relation between the initial track points in the initial track, and confidence calculation is carried out on the basis, so that irregular and false tracks are removed, the suppression of the false alarm is realized, and the rapid moving target detection in the low-quality infrared remote sensing image is completed.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be more apparent from the following description of embodiments of the disclosure with reference to the accompanying drawings, in which:

FIG. 1 schematically illustrates an application scenario diagram of an infrared sequence image-based object detection method, apparatus, device, medium and program product according to an embodiment of the present disclosure;

FIG. 2 schematically illustrates a flow chart of a method of infrared sequence image-based object detection in accordance with an embodiment of the present disclosure;

FIG. 3 schematically illustrates a schematic diagram of a trace formed by association according to an embodiment of the present disclosure;

FIG. 4 schematically illustrates a dual threshold distance constraint schematic in accordance with an embodiment of the present disclosure;

FIG. 5 schematically illustrates a graph of a change rule of track confidence with track valid point number according to an embodiment of the present disclosure;

FIG. 6 schematically illustrates a block diagram of a target detection method apparatus based on infrared sequential images according to an embodiment of the disclosure; and

fig. 7 schematically illustrates a block diagram of an electronic device adapted to implement an infrared sequence image based object detection method, according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

The infrared remote sensing imaging and detection technology mainly acquires a radiation signal of a target by means of an infrared band, converts the radiation signal into an image signal, and then utilizes a method in image processing to realize tasks such as detection and identification of the target, and the like, and plays an important role in the fields such as security protection, weather, geographical monitoring and the like at present. However, the resolution of infrared imaging is low compared to visible light detection, and the signal-to-noise ratio of images is low and the image quality is poor due to the problems of atmospheric transmission, clutter interference, sensor inherent noise, the existence of other false alarm sources with high radiation characteristics, and the like. On the other hand, the infrared remote sensing image does not have obvious characteristics such as color, texture, geometric form and the like, and certain difficulty is brought to infrared target detection. Therefore, the current infrared remote sensing imaging and detecting system is mainly used for finding fast moving targets with strong radiation characteristics and obvious movement characteristics.

Since the resolution of infrared imaging is generally low, the image quality is poor, the number of pixels after imaging the target is small, generally only a few to tens of pixels, and the morphological contour is relatively blurred. Meanwhile, false alarm sources with high radiation characteristics, such as cloud, water body, ground facilities and the like, exist in the ground object, and a plurality of false alarm interferences are brought to the detection of the target. Therefore, it is needed to construct a method for detecting a fast-flying and moving target for a low-quality infrared remote sensing image.

In view of this, the present disclosure provides a target detection method, apparatus, electronic device and medium based on infrared sequential images, where the method includes: threshold segmentation is carried out on the obtained infrared sequence image to obtain an alternative point set, wherein the infrared sequence image is suitable for representing a moving target object; processing the candidate point set by using the distance screening condition to obtain a plurality of initial tracks; determining effective points from the initial track points according to the position relation between the initial track points in the initial track; determining the confidence coefficient of the initial track according to the number of the effective points in the initial track; and determining a target track representing the motion state of the object to be detected from the plurality of initial tracks according to the confidence level.

It should be noted that, the method and the device for detecting an object based on an infrared sequence image provided by the present disclosure may be used in the field of infrared object detection, for example, object detection, and may also be used in any field other than the field of infrared object detection, for example, the field of infrared remote sensing image processing, where the application field of the method and the device for detecting an object based on an infrared sequence image provided by the present disclosure is not limited.

In the technical scheme of the invention, the related user information (including but not limited to user personal information, user image information, user equipment information, such as position information and the like) and data (including but not limited to data for analysis, stored data, displayed data and the like) are information and data authorized by a user or fully authorized by all parties, and the processing of the related data such as collection, storage, use, processing, transmission, provision, disclosure, application and the like are all conducted according to the related laws and regulations and standards of related countries and regions, necessary security measures are adopted, no prejudice to the public welfare is provided, and corresponding operation inlets are provided for the user to select authorization or rejection.

Fig. 1 schematically illustrates an application scenario diagram of an infrared sequence image-based target detection method according to an embodiment of the present disclosure.

As shown in fig. 1, an application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 101, 102, 103.

The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (by way of example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that, the method for detecting an object based on an infrared sequence image according to the embodiment of the disclosure may be generally performed by the server 105. Accordingly, the object detection method apparatus based on infrared sequential images provided in the embodiments of the present disclosure may be generally disposed in the server 105. The infrared sequential image-based object detection method provided by the embodiments of the present disclosure may also be performed by a server or server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the object detection method apparatus based on infrared sequential images provided in the embodiments of the present disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

The infrared sequential image-based object detection method of the disclosed embodiment will be described in detail below with reference to the scenario described in fig. 1 by fig. 2 to 5.

Fig. 2 schematically illustrates a flowchart of an infrared sequence image-based object detection method according to an embodiment of the present disclosure.

As shown in fig. 2, the infrared-series image-based target detection method of this embodiment includes operations S210 to S250.

In operation S210, the acquired infrared sequence image is subjected to threshold segmentation to obtain a set of candidate points, wherein the infrared sequence image is suitable for characterizing a moving target object.

In operation S220, the candidate point set is processed using the distance screening condition to obtain a plurality of initial trajectories.

In manipulation S230, an effective point is determined from the initial trajectory points according to the positional relationship between the initial trajectory points in the initial trajectory.

In operation S240, a confidence level of the initial trajectory is determined according to the number of valid points in the initial trajectory.

In operation S250, a target trajectory characterizing a motion state of the object to be detected is determined from a plurality of initial trajectories according to the confidence level.

According to the embodiment of the disclosure, the radiation signal of the moving target object can be acquired by means of infrared wave bands and converted into image signals, and finally infrared sequence images are formed.

According to embodiments of the present disclosure, the infrared sequence image is suitable for characterizing a moving target object, such as, but not limited to, an unmanned aerial vehicle, an airplane, a hot air balloon, etc., and embodiments of the present disclosure are not limited to a specific type of target object.

According to the embodiment of the disclosure, the acquired infrared sequence image is subjected to threshold segmentation, so that an alternative point set can be obtained. The set of candidate points characterizes a set of pixel points, and a plurality of candidate points can be included in the set of candidate points. The threshold segmentation may include, but is not limited to, a global threshold, an Otsu threshold, an iterative threshold, etc., and embodiments of the present disclosure do not limit the specific type of threshold segmentation.

According to the embodiment of the disclosure, on the basis of obtaining the candidate point set, the candidate points in the image can be formed into a plurality of corresponding initial tracks according to the distance screening conditions by combining the characteristic of high-speed movement of the target object.

According to embodiments of the present disclosure, the initial trajectory formed may have a turn-back, repeated occurrence, or association error, and the trajectory of the normal high-speed moving target object is smoothly and stably advanced along a relatively fixed method. Therefore, the effective point can be determined from the initial trajectory points according to the positional relationship between the initial trajectory points in the initial trajectory. The number of the effective points obtained from the initial track points is smaller than or equal to the number of the points contained in the initial track.

According to embodiments of the present disclosure, the confidence of the initial trajectory becomes greater as the number of valid points of the initial trajectory increases. All the initial tracks detected in the low-quality infrared remote sensing image can be screened through calculating the confidence coefficient of the initial tracks, and finally, the target track of the motion state of the object to be searched can be determined from a plurality of initial tracks according to the set confidence coefficient, so that the suppression of false alarms is realized, and the real target track is reserved.

According to the embodiment of the disclosure, the candidate point set can be extracted on the basis of the high radiation characteristic of the target object, and the target object is associated to form a track according to the position relation between the initial track points in the initial track by combining the time domain motion characteristic. Because of the low resolution and poor quality of the infrared image, a large number of alternative points are generated in the detection process, so that a plurality of tracks are formed. In order to reduce the false alarm, the candidate points of the target track can be screened according to the position relation between the initial track points in the initial track, and confidence calculation is carried out on the basis, so that irregular and false tracks are removed, the suppression of the false alarm is realized, and the rapid moving target detection in the low-quality infrared remote sensing image is completed.

According to an embodiment of the present disclosure, the infrared sequence image-based target detection method of the embodiment further includes: carrying out image difference calculation on infrared sequence images of different frames in the multi-frame infrared sequence images to obtain frame difference infrared sequence images; the method for obtaining the candidate point set comprises the following steps of: and carrying out threshold segmentation on the frame difference infrared sequence image to obtain an alternative point set in the frame difference infrared sequence image.

According to the embodiment of the disclosure, the frame difference infrared sequence image may represent that different frames of infrared sequence images are selected from multiple frames of infrared sequence images to perform image difference calculation, and the selected different frames may represent continuous frames or 1 frame or multiple frames at intervals, but not limited thereto, the embodiment of the disclosure does not limit the number of frames at intervals, and may be selected according to practical situations.

According to embodiments of the present disclosure, an image difference value may be used to detect a difference between two images. Subtracting the pixel values at the corresponding positions of the two images may result in the occurrence of a negative value, which may be taken as an absolute value or set as 0, where the specific operation of the negative value is not limited in the embodiments of the present disclosure.

According to the embodiment of the disclosure, the frame difference infrared sequence image is used and subjected to threshold segmentation, the alternative point set is screened out, and the information of motion and change in the infrared sequence image can be seen more clearly, so that the acquisition of a target track is facilitated.

According to an embodiment of the present disclosure, processing a set of candidate points using a distance screening condition to obtain a plurality of initial trajectories includes: in the case that at least two candidate points in the candidate point set meet the threshold rule, determining the candidate points meeting the threshold rule as target candidate points; and determining an initial track according to the position relation among the target candidate points.

According to an embodiment of the present disclosure, an initial candidate point is selected, and when there is a candidate point satisfying the threshold rule, the candidate point is determined as a target candidate point associated with the initial candidate point according to the threshold rule. And screening other alternative points according to the same threshold rule, and finally determining an initial track.

Fig. 3 schematically illustrates a schematic diagram of a trace formed by association according to an embodiment of the present disclosure.

As shown in fig. 3, due to the complex imaging background of the infrared remote sensing image, the extracted candidate points are more, so that the formed track may have back and forth turning, repeated occurrence and association errors, and the track of the normal high-speed moving target object should be smooth, regular and stably advanced along a relatively fixed direction.

According to an embodiment of the present disclosure, in a case where the initial candidate point set satisfies a threshold rule, determining the target candidate point set includes: threshold rule characterization: among the candidate points of the multi-frame infrared sequence images, the distance between the reference candidate point in the reference frame infrared sequence image and the adjacent candidate point in the adjacent frame infrared sequence image is larger than or equal to a first distance threshold value and smaller than or equal to a second distance threshold value, wherein the first distance threshold value is smaller than the second distance threshold value; or the distance between the reference candidate point in the reference frame infrared sequence image and the adjacent candidate point in the adjacent frame infrared sequence image is smaller than or equal to a third distance threshold value.

According to embodiments of the present disclosure, the infrared sequence image may include an original infrared sequence image, a frame difference infrared sequence image, and herein, embodiments of the present disclosure do not limit a specific type of the infrared sequence image.

According to embodiments of the present disclosure, a target candidate point set may be determined in the event that the initial candidate point set meets a threshold rule. The threshold rule may include combining the characteristic of high-speed motion of the target objects on the basis of the set of candidate points of the infrared sequence image, wherein the distance that the respective candidate point in each target object moves in different frames is greater than or equal to a first distance threshold and less than or equal to a second distance threshold relative to the distance of the reference frame image. A set of target candidate points may be determined according to such a double-threshold rule. Embodiments of the present disclosure do not limit specific values of the first distance threshold and the second distance threshold.

According to an embodiment of the present disclosure, the threshold rule may further include a third distance threshold or less between a reference candidate point in the reference frame infrared sequence image and an adjacent candidate point in the adjacent frame infrared sequence image, among the candidate points of each of the plurality of frame infrared sequence images. The target candidate point set may be determined according to such a single threshold rule. Embodiments of the present disclosure do not limit specific values of the third distance threshold.

Fig. 4 schematically illustrates a dual threshold distance constraint schematic in accordance with an embodiment of the present disclosure.

As shown in fig. 4, the dual threshold distance constraint schematic of the embodiment of the present disclosure includes a minimum distance 410, a maximum distance 420, an initial candidate point 41, a point to be associated 42, a point to be associated 43, and a point to be associated 44.

According to an embodiment of the present disclosure, the initial candidate point 41 may be determined according to a double threshold distance constraint condition, i.e., the distance between the associated target candidate point and the initial candidate point is not less than or equal to the minimum distance 410 and not greater than or equal to the maximum distance 420, as in fig. 4, according to the distance constraint condition.

According to an embodiment of the present disclosure, determining an effective point from initial trajectory points according to a positional relationship of the initial trajectory points in the initial trajectory includes: calculating a kth initial track length between a kth initial track point and a 0 th initial track point of the initial track aiming at the kth initial track point in the initial track, wherein the kth initial track length is characterized based on the number of image pixel points passing from the 0 th initial track point to the kth track point in the initial track, k is more than or equal to 1, and the 0 th initial track point is a starting track point of the initial track; and determining the kth initial track point as an effective point when the kth initial track length represents the number of image pixel points passing from the 0 th initial track point to the kth track point and is larger than the number of initial track points passing from the 0 th initial track point to the kth track point in the initial track.

According to the embodiment of the disclosure, due to the high-speed motion characteristic of the target object, the track length of the initial track and the number of initial track points should be in a direct proportion relation, the longer the track length, the more the number of the initial track points is included, and the number of the image pixel points through which the track length passes is greater than or equal to the number of the initial track points.

According to an embodiment of the present disclosure, for example, for the 10 th initial track point in the initial track, a track length between the 10 th initial track point and the start track point is calculated, and the track length may represent the number of image pixel points passing from the start track point to the 10 th initial track point. When the number of passing image pixel points is greater than the number of initial trajectory points passing from the start trajectory point to the 10 th initial trajectory point, the 10 th initial trajectory point is determined as an effective point. Similarly, the valid point may be determined from the initial trajectory point.

According to an embodiment of the present disclosure, before determining the confidence level, further comprising: and establishing a track confidence formula based on the number of the effective points of the initial track. The track confidence coefficient formula is shown as formula (1):

where Cred is the confidence, C is a constant, and M is the number of valid points in the initial trajectory.

According to embodiments of the present disclosure, a trajectory confidence formula may be established based on the number of valid points of the initial trajectory. For example, when C is 1.7, the confidence Cred is about 0.8 when the number M of valid points in the initial trajectory is 15.

According to an embodiment of the present disclosure, determining a target trajectory characterizing a motion state of an object to be detected from among a plurality of initial trajectories according to a confidence level includes: determining a confidence coefficient curve according to the confidence coefficient corresponding to each of the plurality of initial tracks; determining a confidence coefficient threshold according to the curvature change relation of the confidence coefficient curve; and determining a target track from the plurality of initial tracks according to the comparison result between the confidence threshold and the plurality of confidence levels.

According to the embodiment of the disclosure, the confidence coefficient curve can be determined according to the confidence coefficient corresponding to each of the plurality of initial tracks, the confidence coefficient threshold value is determined according to the curvature change relation of the confidence coefficient curve, and the more gradual and stable the curvature change is, the more accurate the corresponding target track is. The confidence coefficient threshold value can be selected according to the actual situation, and when the confidence coefficient is larger than the confidence coefficient threshold value, the track corresponding to the confidence coefficient can be determined to be the target track. The target trajectory may be determined from among a plurality of initial trajectories based on a plurality of comparison results.

According to the embodiment of the disclosure, all tracks detected in the low-quality infrared remote sensing image can be screened through the calculation of the track confidence coefficient, and finally the suppression of false alarms is realized according to the set confidence coefficient threshold value, and the real target track is reserved, so that the detection of the rapid flying target in the infrared remote sensing image is realized.

Fig. 5 schematically illustrates a schematic diagram of a change rule of track confidence with track valid point number according to an embodiment of the present disclosure.

As shown in fig. 5, the abscissa is the trace point number and the ordinate is the trace confidence. According to the track confidence coefficient formula, when the constant C is set to be 1.7, the change rule of the track confidence coefficient along with the number of the track effective points can be obtained. When the number of track points is larger, the track confidence coefficient is higher, a certain track point number is reached, the track confidence coefficient change is gentle, and when the track confidence coefficient is higher, the track confidence coefficient is higher.

Based on the target detection method based on the infrared sequence image, the disclosure also provides a target detection method device based on the infrared sequence image. The device will be described in detail below in connection with fig. 6.

Fig. 6 schematically illustrates a block diagram of a target detection method apparatus based on infrared sequential images according to an embodiment of the present disclosure.

As shown in fig. 6, the infrared-sequence image-based target detection method apparatus 600 of this embodiment includes an alternative point collection module 610, an initial trajectory obtaining module 620, an effective point determining module 630, a confidence determining module 640, and a target trajectory determining module 670.

The candidate point set module 610 is configured to perform threshold segmentation on an acquired infrared sequence image to obtain a candidate point set, where the infrared sequence image is suitable for characterizing a moving target object. In an embodiment, the alternative point set module 610 may be configured to perform the operation S210 described above, which is not described herein.

The initial trajectory obtaining module 620 is configured to process the candidate point set by using a distance screening condition to obtain a plurality of initial trajectories. In an embodiment, the initial trajectory obtaining module 620 may be used to perform the operation S220 described above, which is not described herein.

The valid point determining module 630 is configured to determine a valid point from the initial track points according to a positional relationship between the initial track points in the initial track. In an embodiment, the effective point determining module 630 may be configured to perform the operation S230 described above, which is not described herein.

The confidence determining module 640 is configured to determine a confidence level of the initial trajectory according to the number of valid points in the initial trajectory. In an embodiment, the confidence determining module 640 may be used to perform the operation S240 described above, which is not described herein.

The target track determining module 650 is configured to determine a target track characterizing a motion state of the object to be detected from a plurality of initial tracks according to the confidence level. In an embodiment, the target track determining module 650 may be configured to perform the operation S250 described above, which is not described herein.

According to an embodiment of the present disclosure, the object detection method apparatus 600 based on infrared sequential images of this embodiment further includes a frame difference image obtaining module.

The frame difference image obtaining module is used for carrying out image difference value calculation on the infrared sequence images of different frames in the multi-frame infrared sequence images to obtain frame difference infrared sequence images.

According to an embodiment of the present disclosure, the candidate point set module includes a frame difference candidate point unit.

And the frame difference candidate point unit is used for carrying out threshold segmentation on the frame difference infrared sequence image to obtain a candidate point set in the frame difference infrared sequence image.

According to an embodiment of the present disclosure, the initial trajectory obtaining module includes: a target candidate point determining unit and an initial trajectory determining unit.

And the target candidate point determining unit is used for determining the candidate points meeting the threshold rule as target candidate points in the case that at least two candidate points in the candidate point set meet the threshold rule.

And the initial track determining unit is used for determining an initial track according to the position relation among the target candidate points.

According to an embodiment of the present disclosure, a target candidate point determination unit includes: a dual threshold subunit and a Shan Yuzhi subunit.

The double-threshold subunit is used for selecting the reference candidate point in the reference frame infrared sequence image from the candidate points of the multi-frame infrared sequence image, and the distance between the reference candidate point in the reference frame infrared sequence image and the adjacent candidate point in the adjacent frame infrared sequence image is larger than or equal to a first distance threshold and smaller than or equal to a second distance threshold, wherein the first distance threshold is smaller than the second distance threshold.

Shan Yuzhi subunit, configured to, among the respective candidate points in the multi-frame infrared sequence image, make a distance between a reference candidate point in the reference frame infrared sequence image and an adjacent candidate point in the adjacent frame infrared sequence image be less than or equal to a third distance threshold.

According to an embodiment of the present disclosure, the effective point determination module includes: an image pixel point determining unit and a kth effective point determining unit.

The image pixel point determining unit is used for calculating the kth initial track length between the kth initial track point and the 0 th initial track point of the initial track aiming at the kth initial track point in the initial track, wherein the kth initial track length is characterized based on the number of image pixel points passing from the 0 th initial track point to the kth track point in the initial track, k is more than or equal to 1, and the 0 th initial track point is the initial track point of the initial track.

A kth effective point determining unit configured to determine a kth initial trajectory point as an effective point in a case where a kth initial trajectory length characterizes a number of image pixel points passing from a 0 th initial trajectory point to a kth trajectory point, which is larger than a number of initial trajectory points passing from the 0 th initial trajectory point to the kth trajectory point in the initial trajectory.

According to an embodiment of the present disclosure, the target detection method apparatus 600 based on the infrared sequence image of this embodiment further includes a track confidence establishment module.

The track confidence establishment module is used for establishing a track confidence formula based on the number of effective points of the initial track, wherein the track confidence formula is shown as formula (1).

According to an embodiment of the present disclosure, the target trajectory determination module includes: the device comprises a confidence curve determining unit, a confidence threshold determining unit and a result comparing unit.

The confidence coefficient curve determining unit is used for determining a confidence coefficient curve according to the confidence coefficient corresponding to each of the plurality of initial tracks.

The confidence threshold determining unit is used for determining a confidence threshold according to the curvature change relation of the confidence curve.

And the result comparison unit is used for determining a target track from the initial tracks according to the comparison result between the confidence coefficient threshold value and the confidence coefficients.

Any of the alternative point aggregation module 610, the initial trajectory derivation module 620, the valid point determination module 630, the confidence determination module 640, and the target trajectory determination module 650 may be combined in one module to be implemented, or any of them may be split into multiple modules, according to embodiments of the present disclosure. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the alternative point set module 610, the initial trajectory derivation module 620, the effective point determination module 630, the confidence determination module 640, and the target trajectory determination module 650 may be implemented, at least in part, as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or as hardware or firmware in any other reasonable manner of integrating or packaging the circuit, or as any one of or a suitable combination of any of the three. Alternatively, at least one of the candidate point set module 610, the initial trajectory derivation module 620, the effective point determination module 630, the confidence determination module 640, and the target trajectory determination module 670 may be implemented, at least in part, as a computer program module that, when executed, performs the corresponding functions.

As shown in fig. 7, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.

In the RAM 703, various programs and data necessary for the operation of the electronic apparatus 700 are stored. The processor 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. The processor 701 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 702 and/or the RAM 703. Note that the program may be stored in one or more memories other than the ROM 702 and the RAM 703. The processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the present disclosure, the electronic device 700 may further include an input/output (I/O) interface 705, the input/output (I/O) interface 705 also being connected to the bus 704. The electronic device 700 may also include one or more of the following components connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 702 and/or RAM 703 and/or one or more memories other than ROM 702 and RAM 703 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowcharts. The program code means for causing a computer system to carry out the method for infrared sequential image-based object detection provided by the embodiments of the present disclosure when the computer program product is run in the computer system.

The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed over a network medium in the form of signals, downloaded and installed via the communication section 709, and/or installed from the removable medium 711. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be provided in a variety of combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. An infrared sequence image-based target detection method comprises the following steps:

threshold segmentation is carried out on the obtained infrared sequence image to obtain an alternative point set, wherein the infrared sequence image is suitable for representing a moving target object;

Processing the candidate point set by using a distance screening condition to obtain a plurality of initial tracks;

determining effective points from the initial track points according to the position relation between the initial track points in the initial track;

determining the confidence coefficient of the initial track according to the number of the effective points in the initial track; and

and determining a target track representing the motion state of the object to be detected from a plurality of initial tracks according to the confidence degree.

2. The method of claim 1, further comprising:

carrying out image difference calculation on infrared sequence images of different frames in the multi-frame infrared sequence images to obtain frame difference infrared sequence images;

the method for obtaining the candidate point set comprises the following steps of:

and carrying out threshold segmentation on the frame difference infrared sequence image to obtain the alternative point set in the frame difference infrared sequence image.

3. The method of claim 1, the processing the set of candidate points using distance screening conditions to obtain a plurality of initial trajectories, comprising:

in the case that at least two candidate points in the candidate point set meet a threshold rule, determining the candidate points meeting the threshold rule as target candidate points;

And determining the initial track according to the position relation between the target candidate points.

4. A method according to claim 3, said determining a target candidate point set if the initial candidate point set meets a threshold rule, comprising:

the threshold rule characterizes:

among the candidate points of the infrared sequence images, the distance between the reference candidate point in the infrared sequence image of the reference frame and the adjacent candidate point in the infrared sequence image of the adjacent frame is larger than or equal to a first distance threshold value and smaller than or equal to a second distance threshold value, wherein the first distance threshold value is smaller than the second distance threshold value; or alternatively

And among the candidate points of the infrared sequence images of the multiple frames, the distance between the reference candidate point in the infrared sequence image of the reference frame and the adjacent candidate point in the infrared sequence image of the adjacent frame is smaller than or equal to a third distance threshold.

5. The method of claim 1, wherein the determining the valid point from the initial trajectory points according to the positional relationship of the initial trajectory points in the initial trajectory comprises:

for a kth initial track point in the initial track, calculating a kth initial track length between the kth initial track point and a 0 th initial track point of the initial track, wherein the kth initial track length is characterized based on the number of image pixel points passing from the 0 th initial track point to the kth track point in the initial track, wherein k is more than or equal to 1, and the 0 th initial track point is a starting track point of the initial track; and

And determining the kth initial track point as the effective point when the kth initial track length represents the number of the image pixel points passing from the 0 th initial track point to the kth track point and is larger than the number of the initial track points passing from the 0 th initial track point to the kth track point in the initial track.

6. The method of claim 1, wherein the prior to determining the confidence level, further comprises:

and establishing a track confidence coefficient formula based on the number of the effective points of the initial track.

The track confidence coefficient formula is shown as formula (1):

7. The method according to claim 1 or 6, wherein said determining a target trajectory characterizing a motion state of the object to be detected among a plurality of said initial trajectories according to said confidence level comprises:

determining a confidence coefficient curve according to the confidence coefficient corresponding to each of the plurality of initial tracks;

determining a confidence coefficient threshold according to the curvature change relation of the confidence coefficient curve; and

and determining the target track from a plurality of initial tracks according to the comparison result between the confidence threshold and a plurality of confidence levels.

8. An infrared sequence image-based target detection method device comprises the following steps:

the system comprises an alternative point set module, a target object detection module and a target object detection module, wherein the alternative point set module is used for carrying out threshold segmentation on an acquired infrared sequence image to obtain an alternative point set, and the infrared sequence image is suitable for representing a moving target object;

9. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-7.

10. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method according to any of claims 1-7.