CN113673417A - Method and system for assisting airplane ground taxiing based on image comparison - Google Patents
Method and system for assisting airplane ground taxiing based on image comparison Download PDFInfo
- Publication number
- CN113673417A CN113673417A CN202110953659.9A CN202110953659A CN113673417A CN 113673417 A CN113673417 A CN 113673417A CN 202110953659 A CN202110953659 A CN 202110953659A CN 113673417 A CN113673417 A CN 113673417A
- Authority
- CN
- China
- Prior art keywords
- image
- database
- similarity
- aircraft
- visual element
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 230000000007 visual effect Effects 0.000 claims description 53
- 230000003247 decreasing effect Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 9
- 230000010365 information processing Effects 0.000 description 6
- 230000001960 triggered effect Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000010977 unit operation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/06—Traffic control systems for aircraft, e.g. air-traffic control [ATC] for control when on the ground
- G08G5/065—Navigation or guidance aids, e.g. for taxiing or rolling
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Aviation & Aerospace Engineering (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a method for assisting airplane ground sliding based on image comparison, which comprises the following steps: the method comprises the steps that images of the surrounding environment of the airplane are collected through an image collecting device when the airplane slides on the ground; retrieving a database image corresponding to the acquired image from a database based on the identification information of the environment; comparing the acquired image with the database image to obtain image similarity; if the image similarity is greater than or equal to the similarity threshold, determining that the aircraft does not deviate from the preset taxi route; and determining that the aircraft deviates from the predetermined taxi route if the image similarity is less than the similarity threshold.
Description
Technical Field
The invention relates to the field of civil aircraft cockpit auxiliary airport guidance, in particular to a method and a system for assisting aircraft ground taxiing based on image comparison.
Background
In the field of airport guidance, in order to perfect airborne auxiliary ground sliding, a relevant system is installed in a side display, and the sliding efficiency of a unit is improved. The existing airport guidance technology mainly utilizes image recognition to judge whether an airplane is in the middle of a taxiway or not, and prevents the airplane from deviating from the taxiway through unit operation (even system automation). Airport guidance technology is mainly based on GPS positioning, and positioning deviation can be caused once GPS signals are abnormal or the resolution is not high enough. This may result in providing the unit with a position indication with a deviation, which may affect the unit judgment, and may cause the unit to spend more effort in judging.
In response to the deficiencies of the prior art airport guidance techniques, it is desirable to provide an improved method and system for assisting aircraft ground taxiing based on image comparison.
Disclosure of Invention
The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.
The invention provides a method for assisting airplane ground taxiing based on image comparison, which comprises the following steps:
the method comprises the steps that images of the surrounding environment of the airplane are collected through an image collecting device when the airplane slides on the ground; retrieving a database image corresponding to the acquired image from a database based on the identification information of the environment; comparing the acquired image with the database image to obtain image similarity; if the image similarity is greater than or equal to the similarity threshold, determining that the aircraft does not deviate from the preset taxi route; and determining that the aircraft deviates from the predetermined taxi route if the image similarity is less than the similarity threshold.
In some embodiments, image-matching the captured image and the database image to obtain image similarity further comprises: extracting one or more visual elements in the acquired image and assigning a corresponding comparison weight to each visual element; comparing each visual element of the acquired image with the corresponding visual element of the database image respectively to obtain corresponding one or more visual element similarities; combining the one or more visual element similarities based on the comparison weight of each visual element to obtain an image similarity.
In some embodiments, the one or more visual elements comprise one or more of: the digital identification pattern of runway number, warning identification, road surface width, fork road surface shape, object shape near fork, light identification, and sky line outline.
In some embodiments, the identification information of the environment includes: the number information of the runway or taxiway, the longitude and latitude information of the airplane and the height of the ground of the airplane.
In some embodiments, capturing an image by the image capture device further comprises: when the airplane is detected to be in a preset position, triggering the image acquisition device to acquire an image, wherein the preset position comprises one or more of the following: gallery bridges, parking ramps, runway turnouts and taxiway turnouts; and when the airplane is detected to be performing the predetermined operation, triggering the image acquisition device to acquire the image, wherein the predetermined operation comprises one or more of the following operations: and (5) braking and turning.
In some embodiments, each database image in the database has an image confidence level, and the method further comprises: if the image similarity of the acquired image and the corresponding database image is greater than or equal to the similarity threshold, increasing the image reliability of the database image; if the image similarity of the acquired image and the corresponding database image is smaller than the similarity threshold value, the image reliability of the database image is decreased progressively; and updating the database by replacing the database image with the corresponding captured image when the image confidence level of the database image is less than the confidence level threshold.
The invention also provides a system for assisting the airplane to slide on the ground based on image comparison, which comprises the following steps: an image acquisition unit configured to acquire an image of an environment around an aircraft while the aircraft is taxiing on the ground; an image retrieval unit configured to retrieve a database image corresponding to the acquired image from a database based on the identification information of the environment; an image comparison unit configured to perform image comparison on the acquired image and the database image to obtain an image similarity; and a determination unit configured to: if the image similarity is greater than or equal to the similarity threshold, determining that the aircraft does not deviate from the preset taxi route; if the image similarity is less than the similarity threshold, determining that the aircraft deviates from the preset taxi route.
In some embodiments, the image alignment unit is further configured to: extracting one or more visual elements in the acquired image and assigning a corresponding comparison weight to each visual element; comparing each visual element of the acquired image with the corresponding visual element of the database image respectively to obtain corresponding one or more visual element similarities; combining the one or more visual element similarities based on the comparison weight of each visual element to obtain an image similarity.
In some embodiments, the image acquisition unit is further configured to: automatically performing image acquisition when the aircraft is detected to be in a predetermined location, wherein the predetermined location comprises one or more of: gallery bridges, parking ramps, runway turnouts and taxiway turnouts; automatically performing image acquisition when it is detected that the aircraft is performing a predetermined operation, wherein the predetermined operation includes one or more of: and (5) braking and turning.
In some embodiments, each database image in the database has an image confidence level, and the system further comprises a database update unit configured to: if the image similarity of the acquired image and the corresponding database image is greater than or equal to the similarity threshold, increasing the image reliability of the database image; if the image similarity of the acquired image and the corresponding database image is smaller than the similarity threshold value, the image reliability of the database image is decreased progressively; and updating the database by replacing the database image with the corresponding captured image when the image confidence level of the database image is less than the confidence level threshold.
The present invention also provides a computer-readable storage medium storing a computer program executable by a processor to perform the aforementioned method for assisting ground taxiing of an aircraft based on image comparison.
The technical scheme of the invention is that the acquired image is marked by utilizing identification information of the longitude and latitude information, the scene height and the like of the airplane, the database is searched by utilizing the identification information, the acquired image is compared with the image in the database to obtain the image similarity, and whether the airplane deviates from the preset taxi route is judged based on the image similarity. The technical scheme of the invention for assisting the airplane to slide on the ground based on image comparison can avoid the positioning deviation caused by abnormal GPS signals and improve the accuracy and reliability of airport guidance.
Drawings
The features, nature, and advantages of the present invention will become more apparent from the detailed description set forth below when taken in conjunction with the drawings. In the drawings, like reference numerals are used to designate corresponding parts throughout the several views. It is noted that the drawings described are only schematic and are non-limiting. In the drawings, the size of some of the elements may be exaggerated and not drawn on scale for illustrative purposes.
Fig. 1 shows a logic diagram of the system for assisting the ground taxiing of an aircraft based on image comparison according to the present invention.
FIG. 2 illustrates a flow chart of an exemplary method of the present invention for assisting ground taxiing of an aircraft based on image comparison.
FIG. 3 sets forth a flow chart illustrating an exemplary method of the present invention for updating a database based on image trustworthiness.
FIG. 4 is a block diagram illustrating a system for assisting ground taxiing of an aircraft based on image comparison in accordance with the present invention.
Fig. 5 shows a block diagram of an apparatus comprising the system for assisting ground taxiing of an aircraft based on image comparison of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings in conjunction with specific embodiments. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the described exemplary embodiments. It will be apparent, however, to one skilled in the art, that the described embodiments may be practiced without some or all of these specific details. In other exemplary embodiments, well-known structures have not been described in detail to avoid unnecessarily obscuring the concepts of the present disclosure. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Meanwhile, the various aspects described in the embodiments may be arbitrarily combined without conflict.
The conventional airport guidance technology based on GPS positioning is easily influenced by GPS signals, and positioning deviation can be caused under the condition that the GPS signals are abnormal or the resolution ratio is not high enough, so that the judgment of a unit is influenced.
The invention provides an improved method and system for assisting the ground taxi of an airplane based on image comparison, which avoids the defects of an airport guiding technology based on GPS positioning and realizes airport guiding with higher precision and more reliability.
FIG. 1 illustrates a logic diagram of a system 100 for assisting ground taxiing of an aircraft based on image comparison in accordance with the present invention.
The aircraft can know the position of the aircraft on the taxi way at the airport through an aircraft navigation signal (102). When the system detects that the aircraft is at a particular location in an airport (such as a bridge, apron, runway or taxiway fork, etc.), an image capture device (106) on the aircraft (such as a camera mounted at the tail, wing tip, nose, etc.) may be triggered to capture images of the environment surrounding the aircraft. Additionally, the image capture device may also be triggered to capture images of the aircraft environment when the system detects that the aircraft is performing a particular aircraft action (104).
The acquired image may be obtained through image acquisition. Meanwhile, the acquired image can be marked by utilizing the identification information of the surrounding environment of the airplane. For example, the captured image may be annotated with location information of the aircraft (e.g., latitude and longitude information of the aircraft, a scene altitude, etc.) to obtain an image with location information (108).
Image retrieval may then be performed in a database (110) using the location information carried by the image to obtain a database image corresponding to the captured image. The two images may then be image-compared. Details regarding image retrieval and image alignment are described in detail in fig. 2.
The comparison may be sent to an information processing center (112) that determines whether the aircraft deviates from the predetermined taxi path based on the image comparison. Meanwhile, the information processing center can also determine whether the database needs to be updated according to the image comparison result. Details regarding database updates will be described in detail in FIG. 3.
FIG. 2 illustrates a flow chart of an exemplary method 200 of the present invention for assisting ground taxiing of an aircraft based on image comparison.
The method 200 begins at 202, where an image of an environment surrounding an aircraft is captured by an image capture device while the aircraft is taxiing on the ground at 202. For example, image acquisition may be performed by a camera mounted on an aircraft.
In some embodiments, image acquisition may be triggered by a particular location. In particular, the image acquisition means may be triggered to acquire an image when the aircraft is detected to be in a particular position (such as a corridor bridge, an apron, a runway or a fork of a taxiway, etc.).
In other embodiments, the image acquisition may be triggered by a specific action. In particular, when it is detected that the aircraft is performing a particular action (such as braking, turning, etc.), the image acquisition device may be triggered to perform image acquisition.
In some embodiments, identification information of the environment surrounding the aircraft may also be recorded when the image of the environment is acquired. The identification information may include runway or taxiway number information, latitude and longitude information of the aircraft, and a scene height of the aircraft. Further, the identification information may also include the time of day, weather conditions, and the like. In a preferred embodiment, the captured image may be annotated with identification information and stored together.
At 204, a database image corresponding to the captured image is retrieved from a database based on the identification information of the environment. In particular, a database image corresponding to the captured image may be retrieved from a database based on latitude and longitude information and the scene altitude of the aircraft. If multiple database images (i.e., database images having the same longitude and latitude information and scene height as the captured image) are retrieved from the database based on the longitude and latitude information and the scene height, one of the database images having the closest runway or taxiway number information to the captured image, time of day, weather conditions, and other identifying information may be selected therefrom.
At 206, the captured image and the database image are image-compared to obtain image similarity. Image similarity represents the degree of similarity between the captured image and the database image.
Image similarity of the acquired image to the database image may be obtained in different ways. In a preferred embodiment, the image similarity of the acquired image to the database image may be obtained based on visual elements. Specifically, the image similarity may be obtained in the following manner.
First, one or more visual elements in the captured image are extracted at 208 and each visual element is assigned a respective alignment weight. Visual elements may include one or more of: a digital identification pattern of a runway number, a warning identification, a road surface width, a fork road surface shape, a shape of an object near a fork, a light identification, a skyline profile, and the like. In some implementations, the visual elements may be assigned alignment weights based on the type of visual element. For example, the comparison result of the figure graph is generally reliable, and the comparison result of the figure graph with the branch shape may be less reliable due to the shooting angle and the like. Therefore, it is possible to assign a high comparison weight to a visual element such as a digital identification pattern, and a relatively low comparison weight to a visual element such as a fork road surface shape or a fork-vicinity object shape.
Next, each visual element of the captured image is compared to a corresponding visual element of the database image, respectively, to obtain a corresponding one or more visual element similarities, at 210. Various methods may be employed to compare visual elements, such as histogram comparison, perceptual hashing algorithms, keypoint matching, and the like.
Finally, the one or more visual element similarities are combined based on the comparison weight for each visual element to obtain an image similarity at 212. For example, the image similarity may be obtained by multiplying each visual element similarity by a corresponding comparison weight, and summing the respective multiplication results.
After obtaining the image similarity, method 200 proceeds to 214. At 214, it is determined whether the image similarity is greater than or equal to a similarity threshold, where the similarity threshold may be predetermined based on measured experience or obtained through theoretical calculations. If the image similarity is greater than or equal to the similarity threshold ("yes" at 214), then it is determined that the aircraft has not deviated from the predetermined taxi path (216). If the image similarity is less than the similarity threshold ("NO" of 214), then it is determined that the aircraft deviates from the predetermined taxi path (218).
Additionally, the image similarity and the determination of whether the aircraft deviates from the predetermined taxi path may be provided to a display device on the aircraft for reference by the crew, or to other aircraft (not shown in fig. 2).
FIG. 3 sets forth a flow chart illustrating an exemplary method 300 of updating a database based on image confidence according to the present invention.
The method 300 begins at 302, where each database image in the database is assigned a respective initial image confidence level at 302. For example, an initial image confidence level may be assigned to each database image based on factors such as the speed and reliability of recognition of the database image, the shading of the light, and so on.
At 304, it is determined whether the image similarity of the captured image to the corresponding database image is greater than or equal to a similarity threshold.
If the image similarity of the acquired image to the corresponding database image is greater than or equal to the similarity threshold ("yes" at 304), the image confidence level for the database image is incremented (306).
If the image similarity of the captured image to the corresponding database image is less than the similarity threshold ("no" at 304), the image confidence of the database image is decremented (308).
At 310, it is determined whether the image confidence level of the database image is greater than or equal to a confidence threshold, where the confidence threshold may be predetermined based on measured experience.
If the image confidence level for all database images in the database is greater than or equal to the confidence threshold ("yes" at 310), the existing database is maintained (312).
If there is an image confidence in the database that is less than the confidence threshold for the database image ("no" at 310), the database is updated. In particular, the database (314) may be updated by replacing the database image with the corresponding acquired image.
For example, when the image similarity between an image captured by a first airplane at a certain geographic location and a corresponding database image is less than a similarity threshold, the image reliability of the database image may be decreased. Subsequently, when the image similarity between the image acquired by the second airplane at the same geographic position and the database image is still smaller than the similarity threshold value, the image reliability of the database image can be continuously decreased. Conversely, if the image similarity between the image acquired by the second airplane at the same geographic position and the database image is greater than or equal to the similarity threshold, the image reliability of the database image is increased, and so on. When the image similarity between the images acquired by enough airplanes at the same geographic position and the database image is smaller than the similarity threshold, the image reliability of the database image is smaller than the reliability threshold, and at the moment, the database can be updated by replacing the database image with the corresponding acquired image. In this way, it can be ensured that the images in the database are sufficiently trustworthy so that the image comparison results are sufficiently reliable.
It should be noted that the method of fig. 3 is only one example of a database update. In different implementations, other methods may also be employed to update the database. For example, if an aircraft determines to deviate from a predetermined taxi route while passing a certain location, the aircraft may actively request (e.g., request from an information processing center) an update of the database. The information processing center, upon receiving the request, may mark the corresponding database image in the database, but temporarily leave the database unchanged. If another aircraft is later determined to deviate from the predetermined taxi route while passing through the same position, the other aircraft updates the database again to the information processing center. The information processing center will now determine that the corresponding database image is erroneous and update the database by replacing the corresponding database image with the most recent captured image.
FIG. 4 illustrates a block diagram of a system 400 for assisting ground taxiing of an aircraft based on image matching in accordance with the present invention.
Referring to fig. 4, the system 400 may include an image acquisition unit 402, an image retrieval unit 404, an image comparison unit 406, a determination unit 408, and a database update unit 410. Each of these units may be connected to or communicate with each other, directly or indirectly, over one or more buses 412.
In embodiments of the present invention, image acquisition unit 402 may be configured to: images of the environment surrounding the aircraft are acquired while the aircraft is taxiing on the ground. In some embodiments, image acquisition unit 402 may be further configured to: automatically performing image acquisition when the aircraft is detected to be in a predetermined location, wherein the predetermined location comprises one or more of: gallery bridges, parking ramps, runway turnouts and taxiway turnouts; automatically performing image acquisition when it is detected that the aircraft is performing a predetermined operation, wherein the predetermined operation includes one or more of: and (5) braking and turning.
The image retrieval unit 404 may be configured to: a database image corresponding to the captured image is retrieved from a database based on the identification information of the environment. The identification information of the environment may include runway or taxiway number information, latitude and longitude information of the aircraft, a scene height of the aircraft, and the like.
The image alignment unit 406 may be configured to: and carrying out image comparison on the acquired image and the database image to obtain the image similarity. In some embodiments, the image alignment unit 406 may be further configured to: extracting one or more visual elements in the acquired image and assigning a corresponding comparison weight to each visual element; comparing each visual element of the acquired image with the corresponding visual element of the database image respectively to obtain corresponding one or more visual element similarities; combining the one or more visual element similarities based on the comparison weight of each visual element to obtain an image similarity.
The determining unit 408 may be configured to: if the image similarity is greater than or equal to the similarity threshold, determining that the aircraft does not deviate from the preset taxi route; if the image similarity is less than the similarity threshold, determining that the aircraft deviates from the preset taxi route.
In some embodiments, each database image in the database has an image confidence. In such embodiments, database update unit 410 may be configured to: if the image similarity of the acquired image and the corresponding database image is greater than or equal to the similarity threshold, increasing the image reliability of the database image; if the image similarity of the acquired image and the corresponding database image is smaller than the similarity threshold, the image reliability of the database image is decreased; when an image confidence level of a database image exists in the database that is less than a confidence level threshold, the database is updated by replacing the database image with the corresponding captured image.
While specific elements of system 400 are shown in FIG. 4, it should be understood that these elements are exemplary only and not limiting. In different implementations, one or more of the units may be combined, divided, removed, or additional units may be added. For example, in some implementations, the image alignment unit 406 and the determination unit 408 may be combined into a single unit. In some implementations, the system 400 may include a display unit (not shown) to display the image comparison results and to prompt the crew whether the aircraft deviates from the predetermined taxi route.
Fig. 5 shows a block diagram illustrating a device 500 comprising the system for assisting aircraft ground taxiing based on image comparison of the present invention.
The apparatus illustrates a general hardware environment in which the present invention may be applied in accordance with an exemplary embodiment of the present invention.
A device 500 will now be described with reference to fig. 5, which is an exemplary embodiment of a hardware device that may be applied to aspects of the present invention. Device 500 may be any machine configured to perform processing and/or computing, and may be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a Personal Digital Assistant (PDA), a smartphone, or any combination thereof. The above-described system may be implemented in whole or at least in part by the device 500 or a similar device or system.
The processor 502 may be any type of processor and may include, but is not limited to, a general purpose processor and/or a special purpose processor (e.g., a special purpose processing chip), an intelligent hardware device (e.g., a general purpose processor, a DSP, a CPU, a microcontroller, an ASIC, an FPGA, a programmable logic device, discrete gate or transistor logic components, discrete hardware components, or any combination thereof). In some cases, the processor 502 may be configured to operate a memory array using a memory controller. In other cases, a memory controller (not shown) may be integrated into processor 502. The processor 502 may be responsible for managing the bus and general processing, including the execution of software stored on memory. The processor 502 may also be configured to perform various functions described herein related to assisting aircraft ground taxiing based on image comparison. For example, the processor 502 may be configured to: the method comprises the steps that images of the surrounding environment of the airplane are collected through an image collecting device when the airplane slides on the ground; retrieving a database image corresponding to the acquired image from a database based on the identification information of the environment; comparing the acquired image with the database image to obtain image similarity; if the image similarity is greater than or equal to the similarity threshold, determining that the aircraft does not deviate from the preset taxi route; and determining that the aircraft deviates from the predetermined taxi route if the image similarity is less than the similarity threshold.
The detailed description set forth above in connection with the appended drawings describes examples and is not intended to represent all examples that may be implemented or fall within the scope of the claims. The terms "example" and "exemplary" when used in this specification mean "serving as an example, instance, or illustration," and do not mean "superior or superior to other examples.
Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, usage of such phrases may not refer to only one embodiment. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean "one and only one" unless specifically so stated, but rather "one or more. The term "some" means one or more unless specifically stated otherwise. All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims.
It is also noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged.
While various embodiments have been illustrated and described, it is to be understood that the embodiments are not limited to the precise configuration and components described above. Various modifications, substitutions, and improvements apparent to those skilled in the art may be made in the arrangement, operation, and details of the devices disclosed herein without departing from the scope of the claims.
Claims (13)
1. A method for assisting ground taxiing of an aircraft based on image comparison, comprising:
the method comprises the steps that when an airplane slides on the ground, an image of the surrounding environment of the airplane is collected through an image collecting device;
retrieving a database image corresponding to the acquired image from a database based on the identification information of the environment;
comparing the acquired image with the database image to obtain image similarity;
determining that the aircraft does not deviate from a predetermined taxi path if the image similarity is greater than or equal to a similarity threshold; and
determining that the aircraft deviates from the predetermined taxi route if the image similarity is less than the similarity threshold.
2. The method of claim 1, wherein image-matching the captured image with the database image to obtain image similarity further comprises:
extracting one or more visual elements in the acquired image and assigning a corresponding comparison weight to each visual element;
comparing each visual element of the acquired image with the corresponding visual element of the database image respectively to obtain corresponding one or more visual element similarities;
combining the one or more visual element similarities based on the comparison weight for each visual element to obtain the image similarity.
3. The method of claim 2, wherein the one or more visual elements comprise one or more of: the digital identification pattern of runway number, warning identification, road surface width, fork road surface shape, object shape near fork, light identification, and sky line outline.
4. The method of claim 1, wherein the identification information of the environment comprises: runway or taxiway number information, latitude and longitude information of the airplane and the scene height of the airplane.
5. The method of claim 1, wherein capturing an image by the image capture device further comprises:
triggering the image acquisition device to acquire an image when the aircraft is detected to be in a predetermined position, wherein the predetermined position comprises one or more of the following: gallery bridges, parking ramps, runway turnouts and taxiway turnouts; and
triggering the image acquisition device to acquire an image when it is detected that the aircraft is performing a predetermined operation, wherein the predetermined operation comprises one or more of: and (5) braking and turning.
6. The method of claim 1, wherein each database image in the database has an image confidence level, and further comprising:
if the image similarity of the acquired image and the corresponding database image is greater than or equal to the similarity threshold, increasing the image reliability of the database image;
if the image similarity of the acquired image and the corresponding database image is smaller than the similarity threshold, the image reliability of the database image is decreased progressively; and
updating the database by replacing a database image with a corresponding captured image when an image confidence level of the database image is less than a confidence level threshold.
7. A system for assisting ground taxiing of an aircraft based on image comparison, comprising:
an image acquisition unit configured to acquire an image of an environment around an aircraft while the aircraft is taxiing on the ground;
an image retrieval unit configured to retrieve a database image corresponding to the acquired image from a database based on the identification information of the environment;
an image comparison unit configured to perform image comparison on the acquired image and the database image to obtain an image similarity; and
a determination unit configured to: determining that the aircraft does not deviate from a predetermined taxi path if the image similarity is greater than or equal to a similarity threshold; determining that the aircraft deviates from the predetermined taxi route if the image similarity is less than the similarity threshold.
8. The system of claim 7, wherein the image alignment unit is further configured to:
extracting one or more visual elements in the acquired image and assigning a corresponding comparison weight to each visual element;
comparing each visual element of the acquired image with the corresponding visual element of the database image respectively to obtain corresponding one or more visual element similarities;
combining the one or more visual element similarities based on the comparison weight for each visual element to obtain the image similarity.
9. The system of claim 8, wherein the one or more visual elements comprise one or more of: the digital identification pattern of runway number, warning identification, road surface width, fork road surface shape, object shape near fork, light identification, and sky line outline.
10. The system of claim 7, wherein the identification information of the environment comprises: runway or taxiway number information, latitude and longitude information of the airplane and the scene height of the airplane.
11. The system of claim 7, wherein the image acquisition unit is further configured to:
automatically performing image acquisition when the aircraft is detected to be in a predetermined location, wherein the predetermined location comprises one or more of: gallery bridges, parking ramps, runway turnouts and taxiway turnouts;
automatically perform image acquisition when it is detected that the aircraft is performing a predetermined operation, wherein the predetermined operation includes one or more of: and (5) braking and turning.
12. The system of claim 7, wherein each database image in the database has an image confidence level, and further comprising a database update unit configured to:
if the image similarity of the acquired image and the corresponding database image is greater than or equal to the similarity threshold, increasing the image reliability of the database image;
if the image similarity of the acquired image and the corresponding database image is smaller than the similarity threshold, the image reliability of the database image is decreased progressively; and
updating the database by replacing a database image with a corresponding captured image when an image confidence level of the database image is less than a confidence level threshold.
13. A computer-readable storage medium storing a computer program executable by a processor to perform the method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110953659.9A CN113673417A (en) | 2021-08-19 | 2021-08-19 | Method and system for assisting airplane ground taxiing based on image comparison |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110953659.9A CN113673417A (en) | 2021-08-19 | 2021-08-19 | Method and system for assisting airplane ground taxiing based on image comparison |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113673417A true CN113673417A (en) | 2021-11-19 |
Family
ID=78543897
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110953659.9A Pending CN113673417A (en) | 2021-08-19 | 2021-08-19 | Method and system for assisting airplane ground taxiing based on image comparison |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113673417A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104157105A (en) * | 2014-08-24 | 2014-11-19 | 无锡北斗星通信息科技有限公司 | Airplane state detecting and alarming system on runway |
| CN105447496A (en) * | 2014-08-01 | 2016-03-30 | 深圳中集天达空港设备有限公司 | Docking airplane model identification and verification method and system based on machine vision |
| CN108205321A (en) * | 2017-12-20 | 2018-06-26 | 成都纵横自动化技术有限公司 | It is sliding to run correction control method and relevant apparatus |
| CN108446630A (en) * | 2018-03-20 | 2018-08-24 | 平安科技（深圳）有限公司 | Airfield runway intelligent control method, application server and computer storage media |
| CN108766036A (en) * | 2018-05-30 | 2018-11-06 | 中国航空无线电电子研究所 | Airborne taxiway and runway visualization guiding and alarm device |
| CN109552650A (en) * | 2017-09-26 | 2019-04-02 | 霍尼韦尔国际公司 | For showing the method and system of the alignment symbology of the deviation between instruction the machine runway route course and tracking |
| CN111352434A (en) * | 2018-12-20 | 2020-06-30 | 波音公司 | Apparatus and method for supporting an aircraft approaching an airport runway at an airport |
| CN112084901A (en) * | 2020-08-26 | 2020-12-15 | 长沙理工大学 | GCAM-based high-resolution SAR image airport runway area automatic detection method and system |
| CN112560922A (en) * | 2020-12-10 | 2021-03-26 | 中国航空工业集团公司沈阳飞机设计研究所 | Vision-based foggy-day airplane autonomous landing method and system |
| CN113212789A (en) * | 2021-06-16 | 2021-08-06 | 王宇灿 | Auxiliary system for airplane taxiing |
-
2021
- 2021-08-19 CN CN202110953659.9A patent/CN113673417A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105447496A (en) * | 2014-08-01 | 2016-03-30 | 深圳中集天达空港设备有限公司 | Docking airplane model identification and verification method and system based on machine vision |
| CN104157105A (en) * | 2014-08-24 | 2014-11-19 | 无锡北斗星通信息科技有限公司 | Airplane state detecting and alarming system on runway |
| CN109552650A (en) * | 2017-09-26 | 2019-04-02 | 霍尼韦尔国际公司 | For showing the method and system of the alignment symbology of the deviation between instruction the machine runway route course and tracking |
| CN108205321A (en) * | 2017-12-20 | 2018-06-26 | 成都纵横自动化技术有限公司 | It is sliding to run correction control method and relevant apparatus |
| CN108446630A (en) * | 2018-03-20 | 2018-08-24 | 平安科技（深圳）有限公司 | Airfield runway intelligent control method, application server and computer storage media |
| CN108766036A (en) * | 2018-05-30 | 2018-11-06 | 中国航空无线电电子研究所 | Airborne taxiway and runway visualization guiding and alarm device |
| CN111352434A (en) * | 2018-12-20 | 2020-06-30 | 波音公司 | Apparatus and method for supporting an aircraft approaching an airport runway at an airport |
| CN112084901A (en) * | 2020-08-26 | 2020-12-15 | 长沙理工大学 | GCAM-based high-resolution SAR image airport runway area automatic detection method and system |
| CN112560922A (en) * | 2020-12-10 | 2021-03-26 | 中国航空工业集团公司沈阳飞机设计研究所 | Vision-based foggy-day airplane autonomous landing method and system |
| CN113212789A (en) * | 2021-06-16 | 2021-08-06 | 王宇灿 | Auxiliary system for airplane taxiing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3581890B1 (en) | Method and device for positioning | |
| KR102273559B1 (en) | Method, apparatus, and computer readable storage medium for updating electronic map | |
| US6952632B2 (en) | Method of guiding an aircraft in the final approach phase and a corresponding system | |
| JP4874607B2 (en) | Object positioning device | |
| CN112052826A (en) | Intelligent enforcement multi-scale target detection method, device and system based on YOLOv4 algorithm and storage medium | |
| US20220068019A1 (en) | Data processor, data processing method, and data processing program | |
| CN113424240B (en) | Road identification device | |
| Rosenbaum et al. | Real-time image processing for road traffic data extraction from aerial images | |
| WO2019061111A1 (en) | Path adjustment method and unmanned aerial vehicle | |
| CN112349150B (en) | Video acquisition method and system for airport flight guarantee time node | |
| JP2017102672A (en) | Geographic position information specification system and geographic position information specification method | |
| JP2012117944A (en) | Navigation device | |
| CN116097128A (en) | Method and device for determining the position of a vehicle | |
| Ji et al. | Validation of global airport spatial locations from open databases using deep learning for runway detection | |
| JP6828448B2 (en) | Information processing equipment, information processing systems, information processing methods, and information processing programs | |
| CN115240154A (en) | Method, device, equipment and medium for extracting point cloud features of parking lot | |
| CN108921896B (en) | Downward vision compass integrating dotted line characteristics | |
| CN113673417A (en) | Method and system for assisting airplane ground taxiing based on image comparison | |
| CN106874928B (en) | Automatic judgment method and system for tracking target emergency key event | |
| WO2023284421A1 (en) | Parking lot list generation method, device, service apparatus and storage medium | |
| JP2020024696A (en) | Prediction device, image recognition device, and prediction system | |
| US11815362B2 (en) | Map data generation apparatus | |
| US20130243339A1 (en) | Picture data provision system | |
| JP7094432B1 (en) | Information processing system, information processing device, program, and information processing method | |
| US20240035847A1 (en) | Parking structure mapping system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |