CN112990187A - Target position information generation method based on handheld terminal image - Google Patents
Target position information generation method based on handheld terminal image Download PDFInfo
- Publication number
- CN112990187A CN112990187A CN202110436206.9A CN202110436206A CN112990187A CN 112990187 A CN112990187 A CN 112990187A CN 202110436206 A CN202110436206 A CN 202110436206A CN 112990187 A CN112990187 A CN 112990187A
- Authority
- CN
- China
- Prior art keywords
- object obj
- camera
- vehicle
- image
- height
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/17—Image acquisition using hand-held instruments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a target position information generation method based on a handheld terminal image, which comprises the following steps: when a suspicious target is found, a camera of the handheld terminal acquires an image of a target scene to obtain a monitoring image; the server identifies the human object and/or the vehicle object in the monitoring image, estimates the orientation of the human object and the actual distance between the human object and the central point of the camera, and estimates the orientation of the vehicle object and the actual distance between the vehicle object and the central point of the camera. The server generates informative text information. The object is divided into the character object and the vehicle object, and different distance recognition algorithms are adopted for the character object and the vehicle object respectively, so that the accuracy of target object distance recognition is effectively improved.
Description
Technical Field
The invention belongs to the technical field of target identification, and particularly relates to a target position information generation method based on a handheld terminal image.
Background
In the fields of public safety, military, emergency rescue or tourism exploration, along with the continuous popularization of handheld terminals (such as mobile phones), the discovery of outdoor specific targets and the information acquisition thereof can be completely completed by the convenient and universal handheld terminals such as the mobile phones and the like. Particularly, in specific industries related to civil information collection and analysis, such as criminal suspects or discovery, recording and reporting of bad behaviors, and the like, comprehensive analysis is performed by combining images with geographic information, so that reliable information can be obtained for decision or analysis of relevant institutions or team command centers, and the system is simple and convenient to collect and high in transmission speed.
The existing information acquisition system has the problem of low acquisition precision of the target geographical position when acquiring the target geographical position, thereby limiting the popularization and the application of the existing information acquisition system.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a target position information generation method based on a handheld terminal image, which can effectively solve the problems.
The technical scheme adopted by the invention is as follows:
the invention provides a target position information generation method based on a handheld terminal image, which comprises the following steps:
step 1, when finding a suspicious target, the camera of the handheld terminal performs image acquisition on the target scene to obtain a monitoring image tu (a), and meanwhile, the handheld terminal obtains the camera position and posture information when acquiring the monitoring image tu (a), and the method comprises the following steps: position coordinate O (x) of camera center point O0,y0) The azimuth beta of a main optical axis of the camera and a pitch angle k of the main optical axis of the camera; the main optical axis azimuth beta of the camera is an included angle between the main optical axis of the camera and the due north direction;
step 2, the handheld terminal uploads the monitoring image tu (A) and the position and posture information of the camera to a server by using a wireless communication module;
step 3, the server performs object identification on the monitoring image tu (A), and detects whether a person object obj (r) and/or a vehicle object obj (c) exist in the monitoring image tu (A); if not, the suspicious target does not exist in the monitoring image tu (A), and the process is ended; if yes, executing step 4;
step 4, the server identifies a person object obj (r) and/or a vehicle object obj (c) in the monitoring image tu (a); such asIf the character object obj (r) is the character object obj (r), the direction alpha of the character object obj (r) is estimated by adopting the steps 5 to 6rAnd the actual distance S between the character object obj (r) and the camera center point Or；
If the vehicle object obj (c) is the vehicle object obj (c), the direction alpha of the vehicle object obj (c) is estimated by adopting the steps 7 to 8cAnd the actual distance S between the vehicle object obj (c) and the camera center point Oc；
Step 5, estimating the orientation alpha of the character object obj (r)rThe method comprises the following steps:
step 5.1, the server analyzes the monitored image tu (A), and obtains the pixel distance x between the image center point of the person object obj (r) and the monitored image tu (A) on the monitored image tu (A)r；
Step 5.2, obtaining the orientation alpha of the character object obj (r) according to the following formular：
αr＝arctan(xr/f)-β
Wherein:
orientation α of person object obj (r)rComprises the following steps: the connecting line between the character object obj (r) and the central point O of the camera forms an included angle with the true north direction; that is, with the camera center point O as a reference, the deviation angle of the human object obj (r) with respect to the true north direction;
step 6, estimating the actual distance S between the character object obj (r) and the central point O of the camerarThe method comprises the following steps: the server reads the pitch angle k of the main optical axis of the camera, and if the pitch angle k of the main optical axis of the camera is smaller than the pitch angle setting threshold k of the person objectmaxIf yes, executing step 6.1; otherwise, executing step 6.2;
step 6.1, the server analyzes the monitoring image tu (A), recognizes the head pixel height m of the person object obj (r) on the monitoring image tu (A), and sets a threshold value m according to the head pixel height m and the head pixelminThe distance D from the projection point of the character object obj (r) in the main optical axis direction of the camera to the camera center point O is obtained according to the following formularThen step 6.3 is executed;
wherein:
f is the focal length of the camera;
m is the pixel value of the height of the human body, namely: on the monitor image tu (a), the pixel value in the height direction of the minimum bounding rectangle of the human object obj (r) is obtained by analyzing the monitor image tu (a);
H1the general actual height value of the person is a preset fixed value;
H2the height value of the general actual head of a person is a preset fixed value;
step 6.2, obtaining the distance D from the projection point of the character object obj (r) in the main optical axis direction of the camera to the central point O of the camera according to the following formularThen step 6.3 is executed;
step 6.3, obtaining the actual distance S between the character object obj (r) and the camera center point O according to the following formular：
Sr＝Dr/cosδr
Wherein: deltarThe included angle between the connecting line of the character object obj (r) and the central point O of the camera and the main optical axis of the camera; deltar＝αr+β；
Step 7, estimating the orientation alpha of the vehicle object obj (c)cThe method comprises the following steps:
step 7.1, the server analyzes the monitoring image tu (A), and obtains the pixel distance x between the vehicle object obj (c) and the image center point of the monitoring image tu (A) on the monitoring image tu (A)c；
Step 7.2, obtaining the azimuth alpha of the vehicle object obj (c) according to the following formulac：
αc＝arctan(xc/f)-β
Wherein:
orientation α of vehicle object obj (c)cComprises the following steps: the connecting line between the vehicle object obj (c) and the central point O of the camera forms an included angle with the true north direction; that is, with the camera center point O as a reference, the declination angle of the vehicle object obj (c) with respect to the true north direction;
step 8, estimating the actual distance S between the vehicle object obj (c) and the central point O of the cameracThe method comprises the following steps:
step 8.1, the server analyzes the monitoring image tu (A), and identifies the minimum circumscribed rectangle of the vehicle object obj (c) on the monitoring image tu (A), wherein the height of the minimum circumscribed rectangle is the vehicle pixel height h;
if the vehicle pixel height h > λ f, where λ is a scaling factor, a known fixed value, then step 8.2 is performed; otherwise, executing step 8.3;
step 8.2, this indicates the actual distance S of the vehicle object obj (c) from the camera center point OcVery small, i.e.: sc0, that is, the position of the vehicle object obj (c) is approximately at the position of the camera center point O; then step 9 is executed;
step 8.3, obtaining the distance D from the projection point of the vehicle object obj (c) in the main optical axis direction of the camera to the central point O of the camera according to the following formulacThen step 8.4 is performed;
wherein:
l2the width value is a common actual width value of the vehicle and is a preset fixed value;
kminsetting a threshold value for the pitch angle of the vehicle object;
h is the pixel height of the vehicle object obj (c), i.e.: on the monitor image tu (a), the pixel value in the height direction of the minimum circumscribed rectangle of the vehicle object obj (c) is obtained by analyzing the monitor image tu (a);
l1the height value of the vehicle is a common actual height value of the vehicle and is a preset fixed value;
step 8.4, obtain the vehicle object obj (c) andactual distance S of camera center point Oc：
Sc＝Dc/cosδc
Wherein: deltacThe included angle between the connecting line of the vehicle object obj (c) and the central point O of the camera and the main optical axis of the camera; deltac＝αc+β；
Then step 9 is executed;
step 9, if the object is the character object obj (r), the orientation α of the character object obj (r) is determinedrAnd the actual distance S between the character object obj (r) and the camera center point OrCombining with the position coordinate O (x) of the central point O of the camera0,y0) Obtaining the position coordinates of the character object obj (r);
if the vehicle object obj (c) is, the orientation alpha of the vehicle object obj (c) is determinedcAnd the actual distance S between the vehicle object obj (c) and the camera center point OcCombining with the position coordinate O (x) of the central point O of the camera0,y0) Obtaining the position coordinates of the vehicle object obj (c);
step 10, the server generates informative text information, wherein the informative text information comprises the position coordinates of the identified person object obj (r) and/or the position coordinates of the identified vehicle object obj (c).
Preferably, in step 3, the server performs object recognition on the monitoring image tu (a), specifically:
and the server adopts the trained machine learning network to perform object recognition on the monitoring image tu (A).
Preferably, the server performs object recognition on the monitoring image tu (a) by using a trained machine learning network, specifically:
if the server recognizes that the person object obj (r) exists in the monitored image tu (a), the age of the person is further recognized; determining the general actual height value H of the person according to the age of the person1And a value of the general actual head height H of the person2；
If the server identifies that the vehicle object obj (c) exists in the monitoring image tu (A), further identifying the vehicle type; according toType of vehicle, determining value of actual width l of vehicle2And the prevailing actual height value l of the vehicle1。
Preferably, after step 10, the method further comprises:
step 11, after obtaining the position coordinates of the character object obj (r) and/or the vehicle object obj (c), the server obtains the geographic information of the scene where the character object obj (r) and/or the vehicle object obj (c) are located through buffer analysis by using a small map service program and a buffer analysis module based on a target position, and fuses the character object obj (r) and/or the vehicle object obj (c) with the geographic information of the scene to generate the informative text information.
The target position information generation method based on the handheld terminal image has the following advantages:
the object is divided into the character object and the vehicle object, and different distance recognition algorithms are adopted for the character object and the vehicle object respectively, so that the accuracy of target object distance recognition is effectively improved.
Drawings
FIG. 1 is a schematic flow chart of a method for generating target location information based on a handheld terminal image according to the present invention;
FIG. 2 is a schematic diagram of an implementation of a method for generating target location information based on a handheld terminal image according to the present invention;
FIG. 3 is a horizontal projection view of the estimation of the geographic coordinates of a human object;
FIG. 4 is a graph of the relationship between the actual height of a person's head and the height of its imaged head pixels in the vertical direction;
FIG. 5 is a schematic diagram of the relationship between the actual height of the target and its imaging height in the vertical direction;
fig. 6 is a schematic diagram of distance calculation considering the pitch angle k of the main optical axis of the camera.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects solved by the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention belongs to the crossing field of the technical fields of mobile terminal technology, digital map technology, automatic identification technology, public security, military application and the like, and mainly aims at the public safety and military detection process, a handheld terminal and other portable devices are used for recording the geographic position of a ground sensitive target or personnel, the geographic position is quickly uploaded to a remote server through a wireless transmission module, and the remote server automatically generates information for relevant personnel to analyze and judge.
Referring to fig. 1 and 2, the present invention provides a method for generating target location information based on a handheld terminal image, comprising the following steps:
step 1, when finding a suspicious target, the camera of the handheld terminal performs image acquisition on the target scene to obtain a monitoring image tu (a), and meanwhile, the handheld terminal obtains the camera position and posture information when acquiring the monitoring image tu (a), and the method comprises the following steps: position coordinate O (x) of camera center point O0,y0) The azimuth beta of a main optical axis of the camera and a pitch angle k of the main optical axis of the camera; the main optical axis azimuth beta of the camera is an included angle between the main optical axis of the camera and the due north direction;
the handheld terminal includes but is not limited to a mobile phone with a camera, a tablet computer and other terminal devices; the hand-held terminal is provided with a position and posture measuring sensor module used for obtaining the position and posture information of the camera when the monitoring image tu (A) is collected. Specifically, the handheld terminal can obtain the current position coordinates of the shooting point through a built-in or external satellite positioning module thereof, namely: position coordinate O (x) of camera center point O0,y0) (ii) a Through its built-in or external gesture and gyrosensor, when can obtain current shooting, orientation and the attitude information of camera, promptly: the azimuth beta of a main optical axis of the camera and the pitch angle k of the main optical axis of the camera are calculated;
step 2, the handheld terminal uploads the monitoring image tu (A) and the position and posture information of the camera to a server by using a wireless communication module;
step 3, the server performs object identification on the monitoring image tu (A), and detects whether a person object obj (r) and/or a vehicle object obj (c) exist in the monitoring image tu (A); if not, the suspicious target does not exist in the monitoring image tu (A), and the process is ended; if yes, executing step 4;
specifically, the present invention mainly recognizes the two types of targets considering that the personnel and the vehicles belong to the primary targets of public safety and military detection, but the targets recognized by the present invention are not limited to the personnel and the vehicles.
In this step, the server performs object recognition on the monitoring image tu (a), specifically:
and the server adopts the trained machine learning network to perform object recognition on the monitoring image tu (A). For example, a machine learning algorithm is firstly adopted to train the target recognition neural network through a large number of data sets of people and vehicles, and after an ideal target recognition neural network is obtained, the neural network is used for recognizing the target in the shooting scene of the handheld terminal. The trained target recognition neural network can better recognize targets in the scene, for example, whether people in the scene are adults or children can be recognized, and the type of vehicles in the scene, belonging to cars or trucks, can also be recognized.
Specifically, if the server recognizes that the person object obj (r) exists in the monitored image tu (a), the age of the person is further recognized; determining the general actual height value H of the person according to the age of the person1And a value of the general actual head height H of the person2；
If the server identifies that the vehicle object obj (c) exists in the monitoring image tu (A), further identifying the vehicle type; determining the universal actual width value l of the vehicle according to the type of the vehicle2And the prevailing actual height value l of the vehicle1。
Step 4, the server identifies a person object obj (r) and/or a vehicle object obj (c) in the monitoring image tu (a); if the character object obj (r) is the character object obj (r), the orientation alpha of the character object obj (r) is estimated by adopting the steps 5-6rAnd the actual distance S between the character object obj (r) and the camera center point Or；
If the vehicle object obj (c) is the vehicle object obj (c), the direction alpha of the vehicle object obj (c) is estimated by adopting the steps 7 to 8cAnd the actual distance S between the vehicle object obj (c) and the camera center point Oc；
In the invention, when the actual distances between the human object obj (r) and the vehicle object obj (c) and the camera center point O are estimated, the actual distances between the estimated object and the camera center point O are different, so that the sizes of the pixels in the shot monitoring images are different, and the sizes of the pixels in the scene of the object are also changed due to the posture and the orientation of the handheld terminal during shooting. Therefore, the present invention proposes a precise estimation method of the geographical positions of the following human object obj (r) and vehicle object obj (c).
Step 5, estimating the orientation alpha of the character object obj (r)rThe method comprises the following steps:
step 5.1, the server analyzes the monitoring image tu (A), and obtains the pixel distance x from the imaging point of the human object obj (r) on the monitoring image tu (A) to the main optical axis direction of the monitoring image tu (A) on the monitoring image tu (A)r；
Wherein, an imaging point of the human object obj (r) on the monitored image tu (a) is represented as G in fig. 1;
step 5.2, obtaining the orientation alpha of the character object obj (r) according to the following formular：
αr＝arctan(xr/f)-β
Wherein:
orientation α of person object obj (r)rComprises the following steps: the connecting line between the character object obj (r) and the central point O of the camera forms an included angle with the true north direction; that is, with the camera center point O as a reference, the deviation angle of the human object obj (r) with respect to the true north direction;
as shown in fig. 3, a horizontal projection view of the estimation of the geographic coordinates for the human object; wherein, the character object obj (r) is represented by a, and the image center point is represented by O; the image plane of the person object obj (r) after being imaged by the camera is represented by B; the Z axis is the direction of the main optical axis of the camera, and the Y axis is the direction vertical to the focal plane; the X axis is determined by taking the direction of a right-handed system from the Z axis to the Y axis;
step 6, estimating the actual distance S between the character object obj (r) and the central point O of the camerarI.e. the distance from point a to point O in fig. 1, the method is: the server reads the pitch angle k of the main optical axis of the camera, and if the pitch angle k of the main optical axis of the camera is smaller than the pitch angle setting threshold k of the person objectmaxIf yes, executing step 6.1; otherwise, executing step 6.2;
the invention estimates the actual distance S between the character object obj (r) and the central point O of the camerarThe main conception is as follows:
firstly, whether the pitch angle k of the main optical axis of the camera is smaller than the pitch angle setting threshold k of the human object is consideredmaxIf so, indicating that the target and the observer position are considered to be in approximately the same horizontal plane, then step 6.1 is performed; otherwise, it indicates that there is a large height difference between the observer position and the target position, for example, the observer is located at a high position, and the target is photographed, in which case, for the purpose of accurate estimation, the influence of the main optical axis pitch angle k on the distance estimation needs to be considered, and the step 6.2 is performed.
Step 6.1, the server analyzes the monitoring image tu (A), recognizes the head pixel height m of the person object obj (r) on the monitoring image tu (A), and sets a threshold value m according to the head pixel height m and the head pixelminThe distance D from the projection point of the character object obj (r) in the main optical axis direction of the camera to the camera center point O is obtained according to the following formularThen step 6.3 is executed;
wherein:
f is the focal length of the camera;
m is the pixel value of the height of the human body, namely: on the monitor image tu (a), the pixel value in the height direction of the minimum bounding rectangle of the human object obj (r) is obtained by analyzing the monitor image tu (a);
H1the general actual height value of the person is a preset fixed value; for example, take 1.7 meters;
H2the height value of the general actual head of a person is a preset fixed value; for example, take0.56 m;
the implementation concept of the step 6.1 is as follows:
when the object is a human object obj (r), a threshold value m is set according to the height m of the head pixel and the head pixelminThereby determining the distance between the human object obj (r) and the camera;
specifically, the method comprises the following steps:
if m > mminThen the representative character object obj (r) is closer to the camera, at this time, the general actual head height value H of the person is adopted2And head pixel height m, estimating the distance between the head pixel height m and a photographer, and adopting the following principle: the general actual head height values H of different character objects when they are closer to the camera2The difference is not large, and the actual head height values of different character objects can be ignored, so that H2The preset fixed value meets the precision requirement; meanwhile, the height m of the head pixel is large, and the requirement of accurate measurement on an image is met.
And when m is less than or equal to mminAt this time, the representative person object obj (r) is far from the camera, and at this time, the general actual height value H of the person is adopted1And the distance between the pixel value M of the height of the person and the photographer is estimated according to the principle: when the person object is far away from the camera, the imaging size of the person image on the image is small, so that the value of the head pixel height M with a small size cannot be accurately measured, and therefore, the pixel value M of the person height with a large pixel is taken as a calculation target to ensure the accuracy requirement.
In practical application, the head pixel sets the threshold value mminIs determined by:
referring to fig. 4, a diagram of the relationship between the actual height of the person's head and the height of the imaged head pixels in the vertical direction is shown. Wherein the height of the head pixel is represented by m, and the actual height of the head of the person is represented by HtAnd (4) showing. The actual height value of the head of an adult is generally 54-58 cm, and the median value is 56 cm. If the head is imaged too small in the image, the recognition difficulty is generated, so the minimum height of the head on the image is taken as 5 pixels according to experience, namely when the height of the head is less than 5 pixels, the head is not processed independently, and the whole height is directly taken。
Due to Dr＝H1f2sin(arctan(xr/f))/(xrM) derivation principle, and Dr＝H2f2sin(arctan(xr/f))/(xrm) are derived in the same manner, and only D will be described belowr＝H1f2sin(arctan(xr/f))/(xrM) reasoning principle:
1) it should be emphasized that, in the present invention, FIG. 3 is a schematic diagram of the target position and its image in the horizontal direction, i.e., the plane of the X-Z axis; and fig. 5 is a schematic diagram of the relationship between the actual height of the target and the imaging height thereof in the vertical direction.
When the target moves from the original position to the main optical axis direction in parallel, the change ratio in the horizontal direction is the same as the change ratio in the vertical direction.
If the target actual height is known, assume the prevalent actual height value H of the person1Its actual height in the main optical axis direction is H'1Thus, there is the following formula (1):
H′1＝DrH1/Sr (1)
namely: the actual height in the direction of the main optical axis, being equal to the original height, multiplied by the ratio Dr/Sr。
2) Based on the schematic diagram in the horizontal direction of fig. 3, it can be seen that:
Dr/Sr＝f/(xr/sinδr) (2)
3) combining equation (1) and equation (2), the following equation (3) is obtained:
H′1＝DrH1/Sr＝fH1sinδr/xr (3)
4) regardless of the camera pose, there is the following geometric relationship:
M/H′1＝f/Dr (4)
thus: having the following formula (5):
Dr＝fH′1/M (5)
5) combining equation (5) and equation (3), equation (6) is obtained:
Dr＝f2H1sinδr/xrM (6)
also, since in FIG. 3, δr＝arctan(xr/f), the following relationship is thus obtained:
Dr＝H1f2sin(arctan(xr/f))/(xrM) (7)
step 6.2, obtaining the distance D from the projection point of the character object obj (r) in the main optical axis direction of the camera to the central point O of the camera according to the following formularThen step 6.3 is executed;
as shown in fig. 6, a schematic diagram of distance calculation considering the pitch angle k of the main optical axis of the camera is shown. In fig. 6, the distance is calculated using the height of the person as a reference. As can be seen from FIG. 6, D is obtained when the pitch angle k of the main optical axis of the camera is taken into considerationrEqual to D when the pitch angle k of the main optical axis of the camera is not consideredrAnd multiplied by cosk.
Step 6.3, obtaining the actual distance S between the character object obj (r) and the camera center point O according to the following formular：
Sr＝Dr/cosδr
Wherein: deltarThe included angle between the connecting line of the character object obj (r) and the central point O of the camera and the main optical axis of the camera; deltar＝αr+β；
Step 7, estimating the orientation alpha of the vehicle object obj (c)cThe method comprises the following steps:
step 7.1, the server analyzes the monitoring image tu (A), and obtains the pixel distance x between the vehicle object obj (c) and the image center point of the monitoring image tu (A) on the monitoring image tu (A)c；
Step 7.2, obtaining the azimuth alpha of the vehicle object obj (c) according to the following formulac：
αc＝arctan(xc/f)-β
Wherein:
orientation α of vehicle object obj (c)cComprises the following steps: the connecting line between the vehicle object obj (c) and the central point O of the camera forms an included angle with the true north direction; that is, with the camera center point O as a reference, the declination angle of the vehicle object obj (c) with respect to the true north direction;
step 8, estimating the actual distance S between the vehicle object obj (c) and the central point O of the cameracThe method comprises the following steps:
step 8.1, the server analyzes the monitoring image tu (A), and identifies the minimum circumscribed rectangle of the vehicle object obj (c) on the monitoring image tu (A), wherein the height of the minimum circumscribed rectangle is the vehicle pixel height h;
if the vehicle pixel height h > λ f, where λ is a scaling factor, a known fixed value, then step 8.2 is performed; otherwise, executing step 8.3;
when the invention identifies the distance of the vehicle object obj (c), the main conception is as follows:
by determining the λ value, the distance between the vehicle object obj (c) and the camera is determined:
specifically, the method comprises the following steps:
if the vehicle pixel height h is more than lambdaf, the actual distance S between the vehicle object obj (c) and the central point O of the camera is showncThe distance is very small, and the vehicle height is not easy to be recognized due to the limitation of the shooting angle and the state when the distance is too short, and at this time, according to the approximation processing, the position of the vehicle object obj (c) is approximately located at the position of the camera center point O.
If the vehicle pixel height h is less than or equal to λ f, the actual distance S between the vehicle object obj (c) and the camera center point O is showncFurther, this case further distinguishes two cases:
in the first case, when the camera main optical axis pitch angle k is small, that is: k is less than or equal to kminIndicating a situation where the target and the observer position are considered to be approximately in the same horizontal plane, in which case the prevailing actual height value l of the vehicle is used1As a reference in distance calculation;
in the second case, when the pitch angle k of the main optical axis of the camera is large, i.e. the camera is in a state of a large pitch angle k：k＞kminIn this case, it is indicated that there is a large height difference between the observer position and the target position, for example, the observer is located at a high position, and the target is photographed in a plan view, and in this case, the entire vehicle body can be photographed because the distance is long, and therefore, the general actual width value l of the vehicle is used2As a reference in distance calculation.
Wherein λ may be taken to be 0.15. λ can be determined by:
1) suppose that the distance D from the projection point of the vehicle object obj (c) in the direction of the main optical axis of the camera to the central point O of the cameracLess than 10 meters, i.e.: when D is presentcIf the distance is less than 10, the distance is considered to be too close, the vehicle height is not easy to identify due to the limitation of the shooting angle and the state, and the following approximate processing is carried out:
due to H/H'c＝f/Dc
Wherein:
h is the vehicle pixel height;
H′cthe height of the vehicle in the direction of a main optical axis is 1.5 meters according to the calculation of a conventional vehicle;
2) thus, it is possible to obtain: h > 0.15f, i.e.: λ is 0.15.
Step 8.2, this indicates the actual distance S of the vehicle object obj (c) from the camera center point OcVery small, i.e.: sc0, that is, the position of the vehicle object obj (c) is approximately at the position of the camera center point O; then step 9 is executed;
step 8.3, obtaining the distance D from the projection point of the vehicle object obj (c) in the main optical axis direction of the camera to the central point O of the camera according to the following formulacThen step 8.4 is performed;
wherein:
l2the width value is a common actual width value of the vehicle and is a preset fixed value; for example, take 1.7 meters;
kminpitching angle for vehicle objectSetting a threshold value; in practical application, the angle is 15 degrees.
h is the pixel height of the vehicle object obj (c), i.e.: on the monitor image tu (a), the pixel value in the height direction of the minimum circumscribed rectangle of the vehicle object obj (c) is obtained by analyzing the monitor image tu (a);
l1the height value of the vehicle is a common actual height value of the vehicle and is a preset fixed value;
step 8.4, obtaining the actual distance S between the vehicle object obj (c) and the camera center point O according to the following formulac：
Sc＝Dc/cosδc
Wherein: deltacThe included angle between the connecting line of the vehicle object obj (c) and the central point O of the camera and the main optical axis of the camera; deltac＝αc+β；
Then step 9 is executed;
step 9, if the object is the character object obj (r), the orientation α of the character object obj (r) is determinedrAnd the actual distance S between the character object obj (r) and the camera center point OrCombining with the position coordinate O (x) of the central point O of the camera0,y0) Obtaining the position coordinates of the character object obj (r);
if the vehicle object obj (c) is, the orientation alpha of the vehicle object obj (c) is determinedcAnd the actual distance S between the vehicle object obj (c) and the camera center point OcCombining with the position coordinate O (x) of the central point O of the camera0,y0) Obtaining the position coordinates of the vehicle object obj (c);
step 10, the server generates informative text information, wherein the informative text information comprises the position coordinates of the identified person object obj (r) and/or the position coordinates of the identified vehicle object obj (c).
Step 11, after obtaining the position coordinates of the character object obj (r) and/or the vehicle object obj (c), the server obtains the geographic information of the scene where the character object obj (r) and/or the vehicle object obj (c) are located through buffer analysis by using a small map service program and a buffer analysis module based on a target position, and fuses the character object obj (r) and/or the vehicle object obj (c) with the geographic information of the scene to generate the informative text information.
It should be emphasized that the above information generation process is performed in the server, which is limited to the software computing capability of the current handheld terminal, and if the handheld terminal has a strong data processing capability, the information generation process can be directly performed in the handheld terminal, and then the generated information and the monitoring image are uploaded to the server together. The invention is not limited in this regard.
In practical application, if the information needs to be uploaded to a server to generate information, before the information is uploaded to the server by taking a picture by the handheld terminal, a user can be required to specify a sensitive target on the image, and the operation can be specified through simple interaction; when the server carries out the target identification algorithm, only the target related to the designated position on the image is extracted, and other targets are ignored, so that the generated information can be more clear.
The invention provides a target position information generation method based on a handheld terminal image, which integrates the performances of various aspects of the handheld terminal, including photographing, positioning, orienting and posture measuring, combines the existing massive geographic information in a geographic information system and the map analysis function of the geographic information system, combines the functions of target identification, character fusion and the like in the existing machine learning method, integrates a set of intelligent information generation system convenient to use, and is convenient for relevant departments to collect sensitive or relevant interest point target information through mobile phone terminals of information collectors and even common users.
The target position information generation method based on the handheld terminal image has the following advantages that:
the object is divided into the character object and the vehicle object, and different distance recognition algorithms are adopted for the character object and the vehicle object respectively, so that the accuracy of target object distance recognition is effectively improved.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and improvements can be made without departing from the principle of the present invention, and such modifications and improvements should also be considered within the scope of the present invention.
Claims (4)
1. A target position information generating method based on a handheld terminal image is characterized by comprising the following steps:
step 1, when finding a suspicious target, the camera of the handheld terminal performs image acquisition on the target scene to obtain a monitoring image tu (a), and meanwhile, the handheld terminal obtains the camera position and posture information when acquiring the monitoring image tu (a), and the method comprises the following steps: position coordinate O (x) of camera center point O0,y0) The azimuth beta of a main optical axis of the camera and a pitch angle k of the main optical axis of the camera; the main optical axis azimuth beta of the camera is an included angle between the main optical axis of the camera and the due north direction;
step 2, the handheld terminal uploads the monitoring image tu (A) and the position and posture information of the camera to a server by using a wireless communication module;
step 3, the server performs object identification on the monitoring image tu (A), and detects whether a person object obj (r) and/or a vehicle object obj (c) exist in the monitoring image tu (A); if not, the suspicious target does not exist in the monitoring image tu (A), and the process is ended; if yes, executing step 4;
step 4, the server identifies a person object obj (r) and/or a vehicle object obj (c) in the monitoring image tu (a); if the character object obj (r) is the character object obj (r), the orientation alpha of the character object obj (r) is estimated by adopting the steps 5-6rAnd the actual distance S between the character object obj (r) and the camera center point Or；
If the vehicle object obj (c) is the vehicle object obj (c), the direction alpha of the vehicle object obj (c) is estimated by adopting the steps 7 to 8cAnd the actual distance S between the vehicle object obj (c) and the camera center point Oc；
Step 5, estimating the orientation alpha of the character object obj (r)rThe method comprises the following steps:
step 5.1, the server analyzes the monitored image tu (A), and obtains the pixel distance x between the image center point of the person object obj (r) and the monitored image tu (A) on the monitored image tu (A)r；
Step 5.2, obtaining the square of the character object obj (r) according to the following formulaBit alphar：
αr＝arctan(xr/f)-β
Wherein:
orientation α of person object obj (r)rComprises the following steps: the connecting line between the character object obj (r) and the central point O of the camera forms an included angle with the true north direction; that is, with the camera center point O as a reference, the deviation angle of the human object obj (r) with respect to the true north direction;
step 6, estimating the actual distance S between the character object obj (r) and the central point O of the camerarThe method comprises the following steps: the server reads the pitch angle k of the main optical axis of the camera, and if the pitch angle k of the main optical axis of the camera is smaller than the pitch angle setting threshold k of the person objectmaxIf yes, executing step 6.1; otherwise, executing step 6.2;
step 6.1, the server analyzes the monitoring image tu (A), recognizes the head pixel height m of the person object obj (r) on the monitoring image tu (A), and sets a threshold value m according to the head pixel height m and the head pixelminThe distance D from the projection point of the character object obj (r) in the main optical axis direction of the camera to the camera center point O is obtained according to the following formularThen step 6.3 is executed;
wherein:
f is the focal length of the camera;
m is the pixel value of the height of the human body, namely: on the monitor image tu (a), the pixel value in the height direction of the minimum bounding rectangle of the human object obj (r) is obtained by analyzing the monitor image tu (a);
H1the general actual height value of the person is a preset fixed value;
H2the height value of the general actual head of a person is a preset fixed value;
step 6.2, obtaining the distance D from the projection point of the character object obj (r) in the main optical axis direction of the camera to the central point O of the camera according to the following formularThen step 6.3 is executed;
step 6.3, obtaining the actual distance S between the character object obj (r) and the camera center point O according to the following formular：
Sr＝Dr/cosδr
Wherein: deltarThe included angle between the connecting line of the character object obj (r) and the central point O of the camera and the main optical axis of the camera; deltar＝αr+β；
Step 7, estimating the orientation alpha of the vehicle object obj (c)cThe method comprises the following steps:
step 7.1, the server analyzes the monitoring image tu (A), and obtains the pixel distance x between the vehicle object obj (c) and the image center point of the monitoring image tu (A) on the monitoring image tu (A)c；
Step 7.2, obtaining the azimuth alpha of the vehicle object obj (c) according to the following formulac：
αc＝arctan(xc/f)-β
Wherein:
orientation α of vehicle object obj (c)cComprises the following steps: the connecting line between the vehicle object obj (c) and the central point O of the camera forms an included angle with the true north direction; that is, with the camera center point O as a reference, the declination angle of the vehicle object obj (c) with respect to the true north direction;
step 8, estimating the actual distance S between the vehicle object obj (c) and the central point O of the cameracThe method comprises the following steps:
step 8.1, the server analyzes the monitoring image tu (A), and identifies the minimum circumscribed rectangle of the vehicle object obj (c) on the monitoring image tu (A), wherein the height of the minimum circumscribed rectangle is the vehicle pixel height h;
if the vehicle pixel height h > λ f, where λ is a scaling factor, a known fixed value, then step 8.2 is performed; otherwise, executing step 8.3;
step 8.2, this case is indicative of the vehicle object obj (c) being located at a distance O from the camera center pointActual distance ScVery small, i.e.: sc0, that is, the position of the vehicle object obj (c) is approximately at the position of the camera center point O; then step 9 is executed;
step 8.3, obtaining the distance D from the projection point of the vehicle object obj (c) in the main optical axis direction of the camera to the central point O of the camera according to the following formulacThen step 8.4 is performed;
wherein:
l2the width value is a common actual width value of the vehicle and is a preset fixed value;
kminsetting a threshold value for the pitch angle of the vehicle object;
h is the pixel height of the vehicle object obj (c), i.e.: on the monitor image tu (a), the pixel value in the height direction of the minimum circumscribed rectangle of the vehicle object obj (c) is obtained by analyzing the monitor image tu (a);
l1the height value of the vehicle is a common actual height value of the vehicle and is a preset fixed value;
step 8.4, obtaining the actual distance S between the vehicle object obj (c) and the camera center point O according to the following formulac：
Sc＝Dc/cosδc
Wherein: deltacThe included angle between the connecting line of the vehicle object obj (c) and the central point O of the camera and the main optical axis of the camera; deltac＝αc+β；
Then step 9 is executed;
step 9, if the object is the character object obj (r), the orientation α of the character object obj (r) is determinedrAnd the actual distance S between the character object obj (r) and the camera center point OrCombining with the position coordinate O (x) of the central point O of the camera0,y0) Obtaining the position coordinates of the character object obj (r);
if the vehicle object obj (c) is, the orientation alpha of the vehicle object obj (c) is determinedcAnd the vehicle object obj (c) and the image pickupActual distance S of head center point OcCombining with the position coordinate O (x) of the central point O of the camera0,y0) Obtaining the position coordinates of the vehicle object obj (c);
step 10, the server generates informative text information, wherein the informative text information comprises the position coordinates of the identified person object obj (r) and/or the position coordinates of the identified vehicle object obj (c).
2. The method for generating the target location information based on the handheld terminal image as claimed in claim 1, wherein in the step 3, the server performs the object recognition on the monitoring image tu (a), specifically:
and the server adopts the trained machine learning network to perform object recognition on the monitoring image tu (A).
3. The method as claimed in claim 2, wherein the server performs object recognition on the monitored image tu (a) by using a trained machine learning network, specifically:
if the server recognizes that the person object obj (r) exists in the monitored image tu (a), the age of the person is further recognized; determining the general actual height value H of the person according to the age of the person1And a value of the general actual head height H of the person2；
If the server identifies that the vehicle object obj (c) exists in the monitoring image tu (A), further identifying the vehicle type; determining the universal actual width value l of the vehicle according to the type of the vehicle2And the prevailing actual height value l of the vehicle1。
4. The method for generating object location information based on handheld terminal image as claimed in claim 1, further comprising after step 10:
step 11, after obtaining the position coordinates of the character object obj (r) and/or the vehicle object obj (c), the server obtains the geographic information of the scene where the character object obj (r) and/or the vehicle object obj (c) are located through buffer analysis by using a small map service program and a buffer analysis module based on a target position, and fuses the character object obj (r) and/or the vehicle object obj (c) with the geographic information of the scene to generate the informative text information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110436206.9A CN112990187B (en) | 2021-04-22 | 2021-04-22 | Target position information generation method based on handheld terminal image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110436206.9A CN112990187B (en) | 2021-04-22 | 2021-04-22 | Target position information generation method based on handheld terminal image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112990187A true CN112990187A (en) | 2021-06-18 |
| CN112990187B CN112990187B (en) | 2023-10-20 |
Family
ID=76341646
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110436206.9A Active CN112990187B (en) | 2021-04-22 | 2021-04-22 | Target position information generation method based on handheld terminal image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112990187B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114061607A (en) * | 2021-11-12 | 2022-02-18 | 浙江数智交院科技股份有限公司 | Improved navigation system and navigation method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016062076A1 (en) * | 2014-10-22 | 2016-04-28 | 中兴通讯股份有限公司 | Camera-based positioning method, device, and positioning system |
| WO2018130016A1 (en) * | 2017-01-10 | 2018-07-19 | 哈尔滨工业大学深圳研究生院 | Parking detection method and device based on monitoring video |
| CN111354046A (en) * | 2020-03-30 | 2020-06-30 | 北京芯龙德大数据科技有限公司 | Indoor camera positioning method and positioning system |
-
2021
- 2021-04-22 CN CN202110436206.9A patent/CN112990187B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016062076A1 (en) * | 2014-10-22 | 2016-04-28 | 中兴通讯股份有限公司 | Camera-based positioning method, device, and positioning system |
| CN105588543A (en) * | 2014-10-22 | 2016-05-18 | 中兴通讯股份有限公司 | Camera-based positioning method, device and positioning system |
| WO2018130016A1 (en) * | 2017-01-10 | 2018-07-19 | 哈尔滨工业大学深圳研究生院 | Parking detection method and device based on monitoring video |
| CN111354046A (en) * | 2020-03-30 | 2020-06-30 | 北京芯龙德大数据科技有限公司 | Indoor camera positioning method and positioning system |
Non-Patent Citations (1)
| Title |
|---|
| 张扬;李守成;陈华: "基于智能前视摄像头的ACC目标车辆探测", 制造业自动化, vol. 39, no. 9 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114061607A (en) * | 2021-11-12 | 2022-02-18 | 浙江数智交院科技股份有限公司 | Improved navigation system and navigation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112990187B (en) | 2023-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109887040B (en) | Moving target active sensing method and system for video monitoring | |
| US20210073573A1 (en) | Ship identity recognition method based on fusion of ais data and video data | |
| WO2017169491A1 (en) | Imaging device and focus control method | |
| US10019624B2 (en) | Face recognition system and face recognition method | |
| US11210796B2 (en) | Imaging method and imaging control apparatus | |
| CN110633629A (en) | Power grid inspection method, device, equipment and storage medium based on artificial intelligence | |
| WO2022000300A1 (en) | Image processing method, image acquisition apparatus, unmanned aerial vehicle, unmanned aerial vehicle system, and storage medium | |
| CN109905641B (en) | Target monitoring method, device, equipment and system | |
| WO2021168838A1 (en) | Position information determining method, device, and storage medium | |
| WO2018032860A1 (en) | Measurement method and apparatus | |
| CN106960027B (en) | The UAV Video big data multidate association analysis method of spatial information auxiliary | |
| CN112990187B (en) | Target position information generation method based on handheld terminal image | |
| CN112488022B (en) | Method, device and system for monitoring panoramic view | |
| KR20150031530A (en) | Method and apparatus for video surveillance by using surveillance apparatus of unmanned aerial vehicle | |
| US20230048952A1 (en) | Image registration method and electronic device | |
| JP4752319B2 (en) | Image recognition apparatus and image recognition method | |
| CN114638880B (en) | Planar ranging method, monocular camera and computer readable storage medium | |
| CN112883809B (en) | Target detection method, device, equipment and medium | |
| WO2021138856A1 (en) | Camera control method, device, and computer readable storage medium | |
| CN117897737A (en) | Unmanned aerial vehicle monitoring method and device, unmanned aerial vehicle and monitoring equipment | |
| CN110705533A (en) | AI recognition and grabbing system for inspection report | |
| KR101300166B1 (en) | Apparatus and method for detecting iris | |
| CN114600162A (en) | Scene lock mode for capturing camera images | |
| CN110726407A (en) | Positioning monitoring method and device | |
| CN115909387B (en) | Engineering lofting method based on enhanced image processing technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |