KR102011765B1 - Method and apparatus for aiming target - Google Patents

Abstract

Provided are a method and device for an aiming target. The aiming device detects a target object from an input image, estimates a size of the detected target object, and estimates a range of the target object by using an estimated target object size, a preset actual target object size, and a focal length of an optical system used to acquire the image. An aiming network is displayed corresponding to the estimated range of the target object. Therefore, the present invention is capable of accurately aiming a target.

Description

Method and apparatus for aiming a target {Method and apparatus for aiming target}

The present invention relates to an aiming method, and more particularly, to a method and apparatus for aiming a target.

The bullet fired from the sniper weapon flies toward the target in a gentle parabola. Therefore, the aiming point can vary depending on the distance the target is from the sniper, so in order to hit the target accurately, you must aim at the point that matches the range of the target.

A sight is used in the sniper firearm to aim the target precisely at a range. A sight is an optical device that allows you to zoom in on a target and make aiming accurate. In these sights, a reticle is applied to aim the target. The mesh line represents the pattern required for the center of the sight so that the center or distance of the target can be easily found. The mesh may be embodied in a form that is engraved by the aiming lens constituting the aiming mirror.

1 is a diagram illustrating the types of meshes.

As the mesh applied to the sight of the sniper firearm, as shown in FIG. 1, there are a stadia mesh (FIG. 1A) and a cross-sectional aiming mesh (FIG. 1B). Stadia mesh is used to measure the range of a target by using the change in the size of objects caused by perspective. Aim-by-range crossnets display all the aiming points along the range in the sight of the sight so that the user can select the appropriate aiming point for the distance.

In order to aim at an existing sniper weapon, the user must directly estimate the range by using the Stadia mesh, for example, using the Stadia mesh to estimate the range, and aim the target at the point corresponding to the estimated range.

In addition, the steady mesh and the collimating mesh should be displayed on the screen of the sight, where the two meshes occupy a considerable proportion on the screen, which may reduce the visibility of the user.

In addition, in order to estimate the range, the user must move the aiming device to position the target in the area where the steadi network is displayed, and after the distance estimation is completed, the aiming device must be moved once again to position the target in the aiming network. This may not only reduce the ease of aiming but also may expose the enemy to movement due to the movement of the aiming device.

Existing methods can be difficult to respond quickly when the target appears because the target must be aimed after the range estimation, and may be inaccurate because the range must be directly determined by the human eye. In addition, the existing method can handle only a special soldier with a high skill, it is difficult for a variety of soldiers to learn the aiming method.

Related prior arts include "Indirect aiming method and system for firearm" disclosed in Korean Patent Application Publication No. 2017-0115189.

An object of the present invention is to provide an aiming method and apparatus for providing an aiming network based on an estimated distance of a target based on detection tracking.

An aiming method according to a feature of the present invention includes a method of aiming a target, the aiming apparatus comprising: detecting a target from an input image; Estimating, by the aiming device, the size of the detected target; Estimating, by the aiming device, the range of the target using the estimated size of the target, a preset actual target size, and a focal length of the optical system used to acquire an image; And displaying the aiming network in correspondence with the estimated range of the target.

The detecting of the target may include detecting a target by extracting pixels having a predetermined temperature or more using temperature data for each pixel of the thermal image when the image is a thermal image.

The detecting of the target by extracting the pixels above the set temperature may include extracting a target candidate group by extracting the pixels above the set temperature; And detecting, as a target, a target candidate corresponding to preset appearance information from the extracted target candidate group.

The detecting of the target may include detecting a target within a region of interest preset in the image.

The estimating of the target may include estimating the size of the target by calculating the size of the detected target from the image.

In this case, estimating the size of the target includes: calculating the size of the detected target for each frame of the image; And obtaining an average value of the sizes of the targets calculated for each frame, and estimating the average value as the size of the target.

의 조건을 토대로 상기 목표물의 사거리를 추정할 수 있으며, 상기 Z는 상기 목표물의 사거리를 나타내고, 상기 x는 상기 추정된 목표물의 크기를 나타내며, 상기 X는 상기 미리 설정된 실제 목표물의 크기를 나타내고, f는 상기 초점 거리를 나타낼 수 있다. In addition, estimating the range of the target,

The range of the target may be estimated based on a condition of Z, Z represents the range of the target, x represents the estimated size of the target, and X represents the size of the preset actual target, f May represent the focal length.

The displaying of the aiming network may further display an estimate of the range of the estimated target with respect to the aiming network.

The aiming method may further include, after displaying the aiming mesh, performing a notification operation when the target is adjacent to the aiming mesh.

The aiming method may further include providing a trigger signal to the firearm to automatically perform a firing operation on the target after displaying the aiming mesh.

According to another aspect of the present invention, an aiming device includes: an input / output unit configured to receive an image; And a processor connected to the input / output unit and configured to perform aiming processing based on the input image, wherein the processor detects a target from the input image, estimates a size of the detected target, and estimates the target. The range of the target is estimated using the size of the target, the preset size of the actual target, and the focal length of the optical system used to acquire an image, and the aiming network is connected to the input / output unit corresponding to the estimated range of the target. Configured to display through.

The image is a thermal image, and the processor extracts a target candidate group by extracting pixels having a predetermined temperature or more using temperature data for each pixel of the thermal image, and extracts a target candidate corresponding to preset appearance information from the extracted target candidate group. It can be configured to detect as a target.

The processor may be configured to calculate the size of the detected target for each frame from the image, obtain an average value of sizes of the target calculated for each frame of the image, and estimate the average value as the size of the target.

의 조건을 토대로 상기 목표물의 사거리를 추정하도록 구성될 수 있으며, 상기 Z는 상기 목표물의 사거리를 나타내고, 상기 x는 상기 추정된 목표물의 크기를 나타내며, 상기 X는 상기 미리 설정된 실제 목표물의 크기를 나타내고, f는 상기 초점 거리를 나타낼 수 있다. The processor,

May be configured to estimate the range of the target based on a condition of Z, wherein Z represents the range of the target, x represents the size of the estimated target, and X represents the size of the preset actual target. , f may represent the focal length.

The processor may be configured to additionally display, through the input / output unit, an estimate of the range of the estimated target with respect to the aiming network.

According to an exemplary embodiment of the present invention, the target range can be estimated based on the distance and size of the target in the personal firearm, and the aiming network can be provided based on the estimated range, so that more accurate aiming can be achieved.

In addition, by only displaying the aiming network corresponding to the estimated range to the aiming scope of the firearm can ensure the user's field of view as possible. As a result, shooting in a short time allows the user to enter an immediate combat posture.

In addition, by generating a notification when the target is close to the aiming network, the user can easily check the status of the target can be conveniently shooting, in particular, the user's fatigue caused by continuing to aim at a target located in the distance Can be reduced.

1 is a diagram illustrating the types of meshes.
2 is a diagram illustrating an example of a thermal image according to an exemplary embodiment.
3 is an exemplary diagram illustrating target detection in a region of interest according to an embodiment of the present invention.
4 is a diagram illustrating a camera model of an image of a target formed on a target and an image sensor according to an exemplary embodiment of the present invention.
5 is an exemplary view showing a line of sight displayed according to the range of the target according to an embodiment of the present invention.
6 is an exemplary view showing a notification generation according to an embodiment of the present invention.
7 is a flow chart of an aiming method according to an embodiment of the present invention.
8 is a structural diagram of an aiming device according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, without excluding other components unless specifically stated otherwise.

Hereinafter, an aiming method and an apparatus according to an exemplary embodiment of the present invention will be described with reference to the drawings.

In an embodiment of the present invention, the distance estimation of the target is performed based on the detection tracking, and the aiming network is displayed at a position corresponding to the estimated distance.

In the embodiment of the present invention, the target is detected and tracked based on the image displayed on the sight in the personal firearm. Personal firearms are taken with the camera, and the captured images are processed and displayed on the sight.

Extract the target from the image displayed by the sight. Detection tracking is performed based on image information. Various techniques can be used as the detection tracking method. If the target and the background are clearly distinguished from the image, a simple detection tracking technique can be applied, but if not, a complex detection tracking technique that uses the image's appearance information such as edges or feature descriptors should be applied. It can be extracted accurately.

On the other hand, when a thermal imaging system (thermal imaging system) is applied to the personal firearm, a thermal image by the thermal equipment is obtained. Thermal imaging equipment is a device that detects inherent radiant energy differences (temperature differences) emitted by objects and backgrounds, and processes them by imaging them.

Thermal images are images obtained by sensing radiant energy radiated from an object having a temperature and converting them into an electrical signal. The higher the temperature of an object is, the higher the level is expressed.

2 is a diagram illustrating an example of a thermal image according to an exemplary embodiment.

As illustrated in FIG. 2, an object having a human shape located at the center of the thermal image has a bright color because it has a temperature, and a background part around a target has a relatively low temperature and has a dark color.

The thermal image having such characteristics is mainly used in a field situation in which it is difficult to recognize a target because there is no light, and the target is more clearly expressed as the relative temperature difference between the target and the background is larger. Therefore, in the case of thermal imaging, the temperature information can be used to effectively track the target detection. For example, when scanning a screen in a thermal image to detect a target, a target may be quickly searched by designating a pixel having a value higher than a threshold temperature arbitrarily set by a user as a candidate group.

Since there may be various temperature sources in the thermal image, after the candidate group is selected, it is necessary to verify whether the candidate group corresponds to the target. Therefore, target detection is performed on an appearance-based basis for the selected candidate groups. Detect and detect candidates that are similar in appearance to the desired target. For example, the target candidate corresponding to the appearance information is extracted using the appearance information of the target previously set in the target candidate group. The appearance information of the preset target may be, for example, information representing the appearance of a general person. When the target-based target detection, for example, a method combining a histogram of oriented gradient (HOG) and a support vector machine (SVM) may be used. HOG is one of the methods of numerically characterizing the appearance of an image, and SVM is one of the classifiers that classify features.

As described above, when extracting a candidate group and performing target detection based on the appearance based on the candidate group, an object similar to the temporarily appearing target may be misdetected as a target or misidentified as a newly appearing target. Target tracking can be performed to prevent this. For example, Kalman filters can be used to track targets. By detecting the detected target every frame, false detection can be minimized.

Meanwhile, when a personal firearm is used in an exhibition situation, an image including various objects may be displayed on the sight in the exhibition situation. Recognizing all objects displayed on the scope as targets or if unwanted objects are considered targets can cause confusion in shooting.

Accordingly, in an embodiment of the present invention, a region of interest is set, and target detection and tracking are performed in the region of interest.

3 is an exemplary diagram illustrating target detection in a region of interest according to an embodiment of the present invention.

As illustrated in the accompanying FIG. 3 on the screen on which the image is displayed, the ROI is set. Targets can be extracted using detection tracking techniques only in the region of interest. Alternatively, the target candidate group may be extracted using the heat source information only for the ROI, and the target candidate may be extracted and tracked based on the appearance from the target candidate group.

Accordingly, it is possible to minimize the error rate for the target detection by recognizing the object that the user thinks as the target. In addition, as the ROI is set, target detection tracking is performed only in the ROI, thereby reducing the detection tracking time. In addition, a significant computer operation is required to process an image when tracking and detecting a target. An operation that processes an image is performed only within the ROI as shown in FIG. 3 instead of the entire image, thereby reducing processing time and reducing power consumption. Can be reduced.

Meanwhile, although the size of the ROI may be fixedly used, the size of the ROI may be changed by a user's arbitrarily manipulation according to the size of the target. In addition, the display of the ROI may be expressed in various forms. For example, as shown in FIG. 3, the ROI may be implemented to rotate “형태” left and right.

In the embodiment of the present invention, as described above, target detection is performed from an image (thermal image), the detected target is tracked, and the size of the detected tracked target is estimated. The range of the target is estimated based on the estimated target size.

The range of the target can be estimated using a digital camera model representing the relationship between the target and the image of the target formed on the image sensor.

4 is a diagram illustrating a camera model of an image of a target formed on a target and an image sensor according to an exemplary embodiment of the present invention.

As shown in FIG. 4, photographing is performed by a camera including a lens and an image sensor, and an image of the target is formed on the image sensor through the lens. In this case, the relationship between the distance Z between the target and the camera and the focal length f between the lens and the image sensor is expressed by the following equation.

Where Z represents the distance between the camera and the target, that is, the range of the target. x represents the size of the image relative to the target imposed on the image sensor. X represents the actual target size.

The size of the image (hereinafter, for convenience of description, referred to as the size of the target image) x for the target attached to the image sensor may be calculated by the target obtained through the detection tracking. Since the size of the detected and tracked target from the image (thermal image) may vary from frame to frame, to obtain a stable distance calculation, the size x on the target is obtained for each frame, the average value of the size x on the acquired target is obtained, and the average value obtained (the target Average value of the size of the image) may be applied in place of x in Equation 1 above. Here, x may be referred to as a size of a target estimated from an image (thermal image).

X, which is the size of the actual target, may be preset. In the conventional aiming method using the Stadia net, the scale of the Stadia net is determined based on the average person's height. Similarly, in an embodiment of the present invention, the size X of the target is preset, and the distance is estimated using the preset X. The preset size X of the target can be arbitrarily set by the user to suit the purpose.

The range Z of the target is estimated based on Equation 1 described above, and then the aiming network is displayed based on the estimated range of the target. For example, in the aiming network, one of the aiming networks shown in FIG. 1B may be displayed on the screen corresponding to the range estimated by the method.

5 is an exemplary view showing a line of sight displayed according to the range of the target according to an embodiment of the present invention.

In the exemplary embodiment of the present invention, the aiming network corresponding to the range of the target estimated using the size of the target and the target size is displayed as illustrated in FIG. 5. Depending on the estimated range of the target, the aiming net can be displayed at the location of the trajectory hit. Here, an estimate of the range can be additionally displayed near the aiming network so that the user can directly check the range.

When the aiming mesh is displayed on the screen corresponding to the range of the target through the distance estimation, the user may aim by moving the firearm so that the target approaches the aiming mesh. In addition, as only the aiming net line according to the estimated target range is displayed, the user's field of view may be maximized. This allows for short aiming and shooting.

On the other hand, when the target is located at a long distance and is displayed small on the screen, the user can easily feel tired and difficult to aim. In this case, it is possible to improve aiming ease by displaying a separate notification when the target approaches the aiming network using the detection tracking result of the target.

6 is an exemplary view showing a notification generation according to an embodiment of the present invention.

For example, in a state where a target is located at a distant distance and an aiming network corresponding to the estimated range for the target is displayed (FIG. 6A), the target is aiming at the target while the user moves the weapon. 6 (b), a warning, that is, a notification operation is performed so that the user can recognize that the target is close to the aiming network (FIG. 6C). Accordingly, the user can easily check the target near the aiming network, so that the user can conveniently shoot. In particular, the user can reduce fatigue caused by continuously aiming the target located at a long distance. Here, the notification operation may be a warning operation for outputting a notification in the form of an audio signal or a notification in the form of a video signal.

In addition, for ease of use, when the target is located on the aiming network, the trigger signal may be provided (transmitted) to the firearm so that the fire is automatically fired. For example, if the target is located within a set range from the center point of the aiming mesh or the target is located at the center point of the aiming mesh, automatic firing of the firearm can be made. At this time, the automatic firing can be performed as soon as the target is located within or within the set range from the center point of the aiming network, or after the set time has elapsed since the position is placed, or after the notification is performed after being positioned. have.

7 is a flow chart of an aiming method according to an embodiment of the present invention.

Aiming apparatus according to an embodiment of the present invention is applied to a personal firearm, it can operate in conjunction with the aiming scope of the firearm.

The aiming device detects a target from an image (thermal image) as shown in FIG. 7 (S100). For example, the target candidate group is extracted from the thermal image acquired by the thermal imaging apparatus based on temperature information, and the target candidate is detected from the extracted target candidate group using preset appearance information. In this case, tracking of the detected target candidate may be selectively performed. In addition, the target may be detected using only information corresponding to a predetermined ROI in the image (thermal image).

Next, the size of the detected target is estimated (S110).

The size of the target detected from the image (thermal image) is calculated and used as the estimated size of the target. For example, the size of the target may be calculated using the number of pixels occupied by the target detected in the image (thermal image). Alternatively, the sizes of the targets are calculated and obtained for each frame, the average value of the obtained target sizes is obtained, and the obtained average value is used as the estimated size of the target.

The target range Z is estimated according to Equation 1 using the estimated target size x, the preset target size X, and the focal length f of the optical system used to acquire the image. (S120).

Next, the aiming network is displayed based on the estimated range of the target (S130). In this case, an estimate of the range may be displayed near the aiming network so that the user may directly check the range.

In addition, by selectively notifying when the target approaches the aiming network (S140), the user can easily recognize the access state of the target to facilitate the aiming.

In an embodiment of the present invention, the image may be used to mean a thermal image.

8 is a structural diagram of an aiming device according to an embodiment of the present invention.

As shown in FIG. 8, the aiming device 100 according to the embodiment of the present invention includes a processor 110, a memory 120, and an input / output unit 130.

The processor 110 may be configured to implement the methods described with reference to FIGS. 2 through 7 above. For example, the processor 110 may estimate a range of the target using a target detector for detecting a target from an image (thermal image), a target size estimator for estimating the size of the detected target, and an estimated target size. The range estimation unit may be configured to include a network processing unit for displaying the aiming network based on the estimated range of the target.

The memory 120 is connected to the processor 110 and stores various information related to the operation of the processor 110. The memory 120 may store instructions for execution in the processor 110 or temporarily load the instructions from a storage device (not shown). The processor 110 may execute instructions stored or loaded in the memory 120. The processor 110 and the memory 120 may be connected to each other through a bus (not shown), and an input / output interface (not shown) may also be connected to the bus. In addition, the memory 120 may be configured to store appearance information for target detection, focal length for range estimation, and preset actual target size.

The input / output unit 130 is configured to provide an image (thermal image) to the processor 110, and is configured to output a processing result of the processor 110. The input / output unit 130 may be configured to process and provide image-related data provided from a camera or thermal imaging equipment to the processor 110.

In addition, the input / output unit 130 may be configured to include a display unit for displaying a collimated mesh line according to a processing result of the processor 110. In addition, the input / output unit 130 may be configured to include a notification unit for outputting a notification (warning sound, etc.) according to the processing result of the processor 110.

In addition, the input / output unit 130 may be configured to include a firearm firing unit that performs automatic firing of the firearm according to the processing result of the processor 110.

The above-described embodiments of the present invention are not implemented only through an apparatus (object) and a method, but a program capable of executing a function corresponding to the configuration of the method according to an embodiment of the present invention or a computer on which the program is recorded. It can also be implemented through a recording medium that can be read, such an implementation can be easily implemented by those skilled in the art from the description of the embodiments described above.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims (15)

As a way to aim the target,
The aiming device detecting a target from an input image;
Estimating, by the aiming device, the size of the detected target;
Estimating, by the aiming device, the range of the target using the estimated size of the target, a preset actual target size, and a focal length of the optical system used to acquire an image; And
Displaying a line of sight corresponding to the estimated range of the target;
Aiming method comprising a. The method of claim 1,
Detecting the target
If the image is a thermal image, detecting a target by extracting pixels having a predetermined temperature or more using temperature data for each pixel of the thermal image
Including, the aiming method. The method of claim 2,
Detecting a target by extracting pixels above the set temperature,
Extracting a target candidate group by extracting pixels above the set temperature; And
Detecting, as a target, a target candidate corresponding to preset appearance information from the extracted target candidate group;
Including, the aiming method. The method of claim 1,
Detecting the target
A target method for detecting a target within a predetermined region of interest in the image. The method of claim 1,
Estimating the size of the target
Aim method of estimating the size of the target by calculating the size of the detected target from the image. The method of claim 5,
Estimating the size of the target
Calculating a size of the detected target for each frame of the image; And
Obtaining an average value of the sizes of the targets calculated for each frame, and estimating the average value as the size of the targets
Including, the aiming method. The method of claim 1,
Estimating the range of the target,

Estimating a range of the target based on a condition of, wherein Z represents a range of the target, x represents the estimated size of the target, X represents the size of the preset actual target, and f represents Aim method, indicating the focal length. The method of claim 1,
The step of displaying the aiming network
And an additional indication of the estimated range of the target relative to the aiming mesh. The method of claim 1,
After displaying the aiming mesh
Performing a notification operation when the target is adjacent to the aiming network
Further comprising, aiming method. The method of claim 1,
After displaying the aiming mesh
Providing a trigger signal to the firearm to automatically perform firing of the firearm against the target;
Further comprising, aiming method. As a device to aim the target
An input / output unit configured to receive an image; And
A processor coupled to the input / output unit and configured to perform aiming processing based on the input image
Including;
The processor detects a target from the input image, estimates the size of the detected target, and uses the estimated target size, a preset actual target size, and a focal length of an optical system used to acquire an image. And estimating a range of the target and displaying the aiming network through the input / output unit corresponding to the estimated range of the target. The method of claim 11,
The image is a thermal image,
The processor may be configured to extract a target candidate group by extracting pixels having a predetermined temperature or more using temperature data for each pixel of the thermal image, and to detect, as a target, a target candidate corresponding to preset outline information from the extracted target candidate group. Aiming device. The method of claim 11,
And the processor is configured to calculate the size of the detected target for each frame from the image, obtain an average value of sizes of the target calculated for each frame of the image, and estimate the average value as the size of the target. The method of claim 11,
The processor,

Is configured to estimate the range of the target based on the conditions of
Z represents the range of the target, x represents the estimated size of the target, X represents the size of the preset actual target, f represents the focal length. The method of claim 11,
And the processor is configured to further display, through the input / output unit, an estimate of the range of the estimated target with respect to the aiming network.

Images