KR20130019546A - The distance measurement system between a real object and a virtual model using aumented reality - Google Patents

Abstract

PURPOSE: A system and a method using augmented reality for measuring a distance between a real object and a virtual model are provided to inspect installation suitability by measuring a spaced distance between a real object and an external store by using an augmented reality method on a real time basis, to reduce development costs, and to remarkably simplify a design process. CONSTITUTION: A system using augmented reality for measuring a distance between a real object and a virtual model comprises air, a real object(1) such as a vehicle or a warship and virtual object synchronization interface(2). The virtual object synchronization interface includes a recognition device(3), an AR simulator(4), an input device(5), an external store computation mock-up(6), and a display device(7). The recognition device utilizes a small camera to provide augmented reality images like the real object to a real target object. The simulator operates a program in which a ranging algorithm is implemented after composing real object images and virtual model images of an external store. The ranging algorithm measures a distance between the external store and the real object by using a marker. [Reference numerals] (1) Mounting target object(real); (2) Virtual object synchronization interface; (3) Recognition device(compact camera); (4) AR simulator(UMPC); (5) Input device; (6) Mounting object mock-up; (7) Display device(goggles or HMD); (AA) Mounting object indicator(reference marker); (BB) Mounting object image(virtual); (CC) Virtual mounting object image display; (DD) User I/F menu display; (EE) Measured value/calculated value delivery; (FF) Information input; (GG) Marker image delivery(operator's position/view direction determination); (HH) Command input; (II) Block diagram of a system for measuring the distance between a real object and a virtual object using augmented reality

Description

The distance measurement system between a real object and a virtual model using aumented reality}

The present invention relates to a distance measuring system and method between the real and the virtual model using augmented reality, in particular in the design of aircraft aircraft parts or external mounting of the aircraft or in the design of automobiles, heavy equipment or ships, existing aircraft, warships or ground Virtual model of external fixtures mounted on real objects using Augmented Reality and 3D modeling and simulation in a virtual development environment when developing external fixtures mounted on real objects such as vehicles The present invention relates to a distance measuring system and a method between a real model and a virtual model using augmented reality, which calculates the mounting suitability by calculating the separation distance of the model.

Existing Augmented Reality and Computerized Digital Mockup technologies support digital 3D R & D modeling and simulation using augmented reality and augmented reality virtual mockup and integrated virtual development environment and technology. The trend is developing into virtual tense technologies.

Existing augmented reality and computational mockups have applied only algorithms and techniques suitable for simple 3D shape implementation / reinforcement, component interference / fitting, and design review functions in computers or smartphones, but when external mountings are mounted on fixed or mobile real objects. There is no report on a system for measuring a distance from a real object using augmented reality and automatically converting a position of a virtual device according to a change in position of an aircraft.

In general, when developing an external mounting of a car or an aircraft, a prototype of a car or aircraft model modeled by computer aided design (CAD) and a virtual CAD model under design are examined to see if interference between the two models is appropriate and the separation distance is appropriate. After fabrication and mounting on the real object, the inspection should be carried out in the same way. In this case, there is no problem when it is the same as the design intention, but when it is necessary to redesign by the part which is not considered, a considerable time is required as well as the cost of reproducing the prototype. In addition, when there is no CAD object model of the vehicle, that is, a vehicle or an aircraft, an unnecessary time for modeling is required. If there are several objects to be mounted, the time increases proportionally.

In the military field, some developed countries are conducting research on applying the mounting suitability related to the newly-developed mounting equipment (mounting) to the actual equipment (real thing) .In the civil field, augmented reality has recently been applied to mobile phones. In both fields, there are no application areas for aircraft, warships, and automobiles, and augmented reality technology can be applied to design reviews in various fields in the future, which is an important technology and tool for future modeling and simulation.

An object of the present invention for solving the problems of the prior art is the augmented reality (Computer Augmented Reality) and computerized digital mockup technology when designing the external mounting to be mounted on a real object, such as aircraft, warships, and ground vehicles 3D R & D modeling and simulation to check the suitability of the virtual model of the external installation mounted on the real object by using 3D R & D modeling and simulation. According to the distance between the real equipment and virtual equipment, the virtual equipment and the real ground using augmented reality to synthesize the real and virtual models in real time using augmented reality, and then separate the two objects using a marker Measure distances and provide designers with digitally based 3D integrated modeling and simulation In the design stage of the external attachment attached to the object by Sean, the problems and errors occurring in the final production stage are verified between augmented reality and the real model using the augmented reality simulation. To provide a distance measurement system.

Another object of the present invention is to provide a distance measuring method between a real model and a virtual model using augmented reality applied to the system.

In order to achieve the object of the present invention, the distance measurement system between the real model and the virtual model using augmented reality, a real object of an aircraft, a warship or a ground vehicle; And augmented reality-based modeling and simulation (M & S) to verify in advance the mounting suitability of the real and the external mounting by using three-dimensional design data in the advanced equipment design stage for the external mounting mounted on the real object. It is composed of virtual object synchronization interface that analyzes the fitting suitability by applying the modeling & simulation technology to the CAD data of the same size as the real object before mounting the external mounting equipment and calculating the separation distance between the real and the external mounting. The virtual object synchronization interface may include a cognitive device using a small camera for providing an image, such as a real object, to an object to be mounted in augmented reality; AR simulation is performed by using augmented reality to synthesize a virtual model image of the real image and the external mounting, and then use a marker to measure the distance between the two objects of the real and external mounting. Device; An input device using a marker to determine an operator's position / line of sight; A mounting computer mockup for loading three-dimensional CAD design data (computation mock-up data) of a mounting apparatus for newly mounting on the real object image to which the marker is attached; And a display device for outputting a distance measurement result of a virtual model of a real object such as an augmented reality image and an aircraft and a mounted object on a screen.

In order to achieve the another object of the present invention, the distance measuring method between a real object using augmented reality and a virtual model of the mounting, (a) real such as aircraft, warships or ground vehicles AR marker is attached to the real object through USB communication of the camera by attaching a marker, which is an input device, to the object and using a camera used as a recognition device. Transmitting to; (b) loading the 3D CAD design data (computation mock-up data of the mounting) of the mounting equipment for newly mounting on the real image with the marker from the mounting computer mock-up to the AR simulator; (c) comparing and analyzing the real image transmitted from the camera by the AR simulation device with an ID previously defined through an image processing algorithm, finding a match with the analyzed ID and the marker ID, and matching the matched marker ID. Determining position, size, and direction according to the computerized mockup of the 3D design data, synthesizing the real image with the marker attached to the actual size and the virtual model image of the mounting, and performing augmented reality rendering; (d) Augmented by the AR simulation apparatus based on a marker attached to the real image using a distance measuring algorithm of a real object using augmented reality and a virtual object of a mounting object. Measuring the shortest distance of the virtual object (mounting equipment) of the mounting, and measuring the distance to the ground; And (e) outputting a distance measurement result of a virtual object of a real object such as an augmented reality image and an aircraft and a mounting object on a screen of the display apparatus.

In order to achieve the above object of the present invention, a computer-readable recording medium having recorded thereon a program for realizing a distance measuring method between a real object using augmented reality and a virtual object of a mounting is (a) Attaching a marker, which is an input device, to a real object such as an aircraft, a warship, or a land vehicle, and using a camera used as a recognition device, a marker portion on a real image by the input device. Capturing and transmitting the real image to the AR simulation device through the camera's USB communication; (b) loading a three-dimensional CAD design data (computation mock-up data of the mounting) of the mounting equipment for newly mounting on the real image with the marker from the mounting computer mock-up to the AR simulator; (c) comparing and analyzing the real image transmitted from the camera by the AR simulation device with an ID previously defined through an image processing algorithm, finding a match with the analyzed ID and the marker ID, and matching the matched marker ID. Determining the position, size, and direction of the 3D design data according to the computerized mock-up of the 3D design data, synthesizing the real image with the marker attached to the actual size and the virtual model image of the mounting, and performing augmented reality rendering; (d) Augmented by the AR simulation apparatus based on a marker attached to the real image using a distance measuring algorithm of a real object using augmented reality and a virtual object of a mounting object. A function of measuring a shortest distance of a virtual object (mounting equipment) of the mounting and measuring a distance to the ground; And (e) a function of outputting a distance measurement result of a virtual object of a real object such as an augmented reality image and an aircraft and a mounting object on a screen of the display apparatus.

Distance measuring system and method between real and virtual model using augmented reality according to the present invention is a virtual development environment in the design of aircraft aircraft parts in the field of defense, external mounting of the aircraft or in the design of automobiles, heavy equipment or ships in the civil field External mounting on real objects such as aircraft, warships and military vehicles using 3D R & D modeling and simulation using Augmented Reality and Computerized Digital Mockup technology The distance between the real and virtual equipment, the virtual equipment, and the virtual ground modeled by the CAD of the water, is a distance-measuring algorithm that uses augmented reality to separate the real and external objects in real time. Measure distances to check mounting suitability, greatly simplifying development costs and the design process Increase the precision by reducing development time, reduce the cost of variable design changes at the equipment design stage, reduce the cost of prototyping by efficiently coping with the life cycle before R & D, and reduce the cost of remanufacturing in case of design errors To provide.

The distance measuring system and method between the real model and the virtual model using augmented reality according to the present invention is designed in advance to check the mounting suitability of the virtual model (virtual model) of the external mounting mounted on the real thing, such as aircraft, warships or ground vehicles From the design stage to the operator's side, problems and errors from the design stage to the final production stage are clearly reviewed and verified through a simulation using augmented reality in advance. Based on this, new and advanced equipment system considering the actual aircraft structure It reduces design changes in design, reduces R & D costs, and significantly shortens the process from weapon system design to production.

In the military aspect, the distance measuring system and method between the real model and the virtual model using augmented reality are visually analyzed for interference with a real aircraft, a warship or a ground vehicle by using a CAD model of an external mounting mounted on a real equipment (real). By providing a technique to calculate the separation distance from the ground or other structures, it is easier to explain to military practitioners during design review and to reduce the disagreement with users during design.

In terms of economics, the distance measurement system between the real model and the virtual model using augmented reality verifies the mounting suitability between the real model and the virtual model of the external mounting, and utilizes the virtual model of the mounting for fitting suitability which was only possible through the production of prototypes after the existing design. This reduces prototyping costs and reduces the cost of remanufacturing in case of design errors.

1 is a block diagram of a system for measuring a distance between a real object and a virtual object using augmented reality according to the present invention.
2 is a photograph showing the detailed H / W components of the real and virtual object synchronization interface using augmented reality.
3 is a flowchart of a virtual object synchronization system using augmented reality.
4 is a flowchart illustrating a distance measuring algorithm between a real object and a virtual object using augmented reality.
FIG. 5 is a diagram illustrating a picture of a handheld viewer.
6 is a view showing a picture of the See-Through HMD + USB camera system.
7 is a photograph for explaining the suitability for mounting the external mounting using augmented reality (Augmented Reality).
8 is a plan view showing the position information of a normal aircraft tire (Tire) and the mounting equipment.
9 is a plan view showing the aircraft tilt calculation during aircraft Right Tire Flat.
Figure 10 is a side view showing the aircraft tilt calculation in the aircraft Right Tire Flat.
FIG. 11 is a diagram illustrating an inclination calculation of an aircraft after moving to an origin point in an aircraft right tire flat.
12 is a plan view showing the inclination calculation of the aircraft when the aircraft Left Tire Flat.
FIG. 13 is a plan view illustrating aircraft tilt calculation during an aircraft NLG tire flat. FIG.
14 is a side view showing the aircraft tilt calculation in the aircraft NLG Tire Flat.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention will be described in detail the configuration and operation.

1 is a block diagram of a system for measuring a distance between a real object and a virtual object using augmented reality according to the present invention.

The distance measurement system between real and virtual objects using augmented reality is used to review aircraft aircraft parts design in the defense industry, to review the design of external aircraft attachments, and to design cars, heavy equipment, or ships in civil fields. do.

The distance measuring system between the real and virtual objects using augmented reality is composed of a total of seven, the real thing of the existing aircraft, warship or ground vehicle (①); And advanced military equipment for automobiles, heavy equipment, or ships in the design of aircraft aircraft components in the defense industry, in the design of external attachments for aircrafts, or in civil applications for external installations mounted on real objects such as aircraft, warships or ground vehicles. In order to verify in advance the suitability of installation of real-world and external installations such as newly developed aircraft using three-dimensional design data in the virtual development environment during design, equipment using augmented reality-based modeling and simulation (M & S) technology is applied. It consists of a virtual object synchronization interface (②) that analyzes the fitting suitability by calculating the 3D CAD data of the same size as the real object before mounting and calculating the separation distance between the virtual model of the real and the external mounting.

The main function is to solve the problem of size and distance from surrounding objects when installing new attachment equipment on the target object (①) of aircraft, warship or ground vehicle. The CAD data of the same size as the real object can be represented as Augmented Reality through the virtual object synchronization interface (②).

Virtual object synchronization interface (②) is a recognition device (③) using a small camera for providing an image, such as a real (real object) in augmented reality to the mounting object (①); After synthesizing a virtual model image of an external mounting and a real image of an aircraft, a warship, a ground vehicle using the augmented reality from the cognitive device and measuring the distance between the two objects of the real and the external mounting using a marker (Marker) An AR simulator for driving a program in which a distance measuring algorithm is implemented; An input device ⑤ using a marker to determine the operator's position / looking direction; Attachment computational mockup (⑥) for loading the three-dimensional CAD design data (computational mockup data) of the attachment equipment for newly mounting to the real object image attached to the marker (Marker); And a head mounted display (HMD), a handheld viewer, a monitor, etc., and augmented reality images, which are composed of real and external mounting images, and mounted with real objects such as aircraft, warships, and ground vehicles. It is composed of a display device (⑦) which outputs the distance measurement result with the virtual object of water on the screen, and the real object using augmented reality that drives them in conjunction with the virtual object of the external mounting. Perform a ranging algorithm.

The virtual object synchronization interface (②) is a recognition device that attaches a marker to the real image (①) of an aircraft, a warship, and a ground vehicle on which the equipment is to be mounted by an input device (⑤) and uses a small camera. Acquire images using (③). The acquired image is recognized by the program of the AR simulator (④) and at the same time, the 3D equipment data of the designer designed by the designer is equipped with the computerized mockup (⑥). It will appear as augmented reality in (①), and it will be augmented using a display device (⑦) such as HMD (Head Mounted Display) or HandHeld Viewer to review from various directions You can watch the reality video.

The recognition device (③) uses a small camera, supports 30hz or more in real time input and output in consideration of weight or size in order to receive real-time video and to allow the operator to move freely. It has Focus and Contrast functions.

AR simulator (④) acquires real-time images related to aircraft, warships, and ground vehicles from cameras used as recognition devices (③) in real time, and markers on real-life (①) images by input device (⑤). Load the computerized mockup data of the fixture (mounting equipment) from the computerized mockup (⑥) and synthesize the real image and the attached image with the marker at the actual size to perform augmented reality rendering, A data input / output module for extracting a separation distance of the outputting device and outputting a mounting suitability result report file to the display device (⑦); Camera calibration (camera calibration, feature point detection and real-time tracking), marker recognition and tracking, marker geometry calculation (3D position, orientation calculation through feature point matching, gesture and event recognition of recognition device (③), 2D image and graphic model An image processing module providing a composition (augmented reality synthesis by applying geometric information of a virtual model of a real image and a mounting object); and measuring a distance between a model in which a virtual model of the real image and the mounting object is matched with a surrounding object (real), And a suitability inspection module for determining whether or not mounting suitability is calculated by calculating a separation distance between the physical equipment and the mounting equipment.

AR simulation device (laptop) (④) uses a computer or laptop, augmented the real image (screen image) input from the recognition device (③) in real time through image processing, and using Augmented Reality (Augmented Reality) It performs the program function of real-time processing the distance between the real image (①) attached to the marker and the mounting object and the distance to the ground.

Marker (⑤) defines the place to be the reference point for the augmented position and distance measurement of 3D image, and installs the input device (⑤) at the point source to be defined, When measuring the actual distance between the object and the virtual object (virtual object), the operator provides a reference point in the distance measurement by installing a marker (Marker) defined in the desired destination (Destination).

The computerized mockup (⑥) is mounted on the real object such as aircraft using 3D CAD design data of X, Y, and Z axes before the production of the product to be mounted on a real object such as an aircraft, a warship, or a military vehicle. Provides 3D digital CAD data of the mounting for virtual viewing as if the water equipment was mounted.

The display device (⑦) uses any one of See-Through type HMD (Head Mounted Display), HandHeld Viewer, or Monitor. And outputting a distance measurement result between the real object and the virtual object of the mounting on a screen.

2 is a photograph showing the detailed H / W components of the real and virtual object synchronization interface using augmented reality.

Detailed H / W components of the virtual object synchronization interface of real and external mounting using augmented reality include handheld viewer, HMD, rangefinder, notebook, battery, space mouse, LCD monitor, marker, cable, USB hub It consists of a backpack.

The present invention relates to a distance measuring system capable of verifying the mounting suitability between a virtual model of the existing real installations, such as aircraft and automobiles and the external mounting mounted thereto. The distance measurement system consists of three parts: one camera (cognitive device 3), a small computer used as an AR simulator 4, and a display device. Each function captures a real object with a reference marker from a camera, calculates correlations between virtual models from images captured on a small computer used as an AR simulator 4, and uses a See-Through type. The display results of the display device 7 such as a head mounted display (HMD), a handheld viewer, a monitor, and the like, and output a result of combining a real model and a virtual model.

Marker (Marker) was used to print the capital letters of the alphabet on the cube. The virtual model was used by converting the data modeled by CATIA CAD software. The main object is a wheeled vehicle, such as a car or an aircraft, that uses a virtual model to test the mounting suitability before prototyping, when designing the external mounting required for such a real object. In addition, it is possible to simulate the situation when there is a problem with some tires of the aircraft. In other words, when a small amount of air is in the tire, an external mounting is developed by using augmented reality technique, and the distance from the ground or other structures can be stored as a video. The present invention is a system that can save time compared to the cost of the development of the external mounting attached to the real, the future is a structure that can be reused in various real or external mounting.

The present invention proposes a technique that can easily calculate the separation distance using augmented reality technique to check the mounting suitability of the actual vehicle or aircraft and the system to be developed by using a CAD model without a prototype. It is a state that can not be found at home and abroad of the distance measuring system with features similar to the present invention. Most augmented reality techniques are widely used in virtual reality and broadcasting, and have been developed using complex and expensive systems in limited environments. However, the present invention can be operated by a very simple operation in which two users photograph one part to be tested and instruct a marker to a part that the other person wants to measure. This method can be used in various weapon systems as well as in automobiles and aircraft in the private sector.

3 is a flowchart illustrating a method for measuring a distance between a real object and a virtual object using augmented reality in a virtual object synchronization system using augmented reality.

According to the present invention, a method for measuring a distance between a real object using augmented reality and a virtual object of a mounting apparatus includes: (a) a marker which is an input device to a real object such as an aircraft, a warship or a ground vehicle; (Marker) attached to the image using a camera used as the recognition device (3) by the input device (5) to take a marker (Marker) part of the real image through the USB communication of the camera AR simulation device ( 4) sending; (b) loading the three-dimensional CAD design data (computation mock-up data of the mounting) of the mounting equipment for newly mounting on the real image with the marker from the mounting computer mock-up to the AR simulation apparatus 4; (c) comparing and analyzing the real image transmitted from the recognition device (camera) 3 by the AR simulation device 4 with an ID previously defined through an image processing algorithm, and matching the ID and marker ID analyzed and compared. 3D design data corresponding to the matched marker ID is determined, and the position, size, and direction are judged according to the computerized mockup of the fixture, and the actual image with the marker attached to the actual size and the virtual model image of the fixture are synthesized. Performing rendering; (d) The AR simulation apparatus 4 refers to a marker attached to the real image using a distance measuring algorithm of a real object using augmented reality and a virtual object of a mounting object. Measuring the shortest distance of the virtual object (mounting equipment) of the mounting to which the augmentation is carried out, and measuring the distance to the ground; And (e) outputting a result of distance measurement between a real object such as an augmented reality image and an aircraft and a virtual object of a mounted object on a screen of the display apparatus 7.

The computer-readable recording medium which records a program for realizing a distance measuring method between a real object using augmented reality and a virtual object of the mounting includes (a) an aircraft, a warship or a land vehicle. Attach a marker as an input device to a real object and take a marker on the real image by the input device 5 by using a camera used as a recognition device 3. Transmitting a real image to the AR simulation apparatus 4 through communication; (b) loading the 3D CAD design data (computation mock-up data of the mounting) of the mounting equipment for newly mounting on the real image with the marker from the mounting computer mock-up to the AR simulation apparatus 4; (c) comparing and analyzing the real image transmitted from the recognition device (camera) 3 by the AR simulation device 4 with an ID previously defined through an image processing algorithm, and matching the ID and marker ID analyzed and compared. 3D design data corresponding to the matched marker ID is determined, and the position, size, and direction are judged according to the computerized mockup of the fixture, and the actual image with the marker attached to the actual size and the virtual model image of the fixture are synthesized. The ability to perform rendering; (d) The AR simulation apparatus 4 refers to a marker attached to the real image using a distance measuring algorithm of a real object using augmented reality and a virtual object of a mounting object. Measuring the shortest distance of the virtual object (mounting equipment) of the mounted augmented with a function of measuring the distance to the ground; And (e) a function of outputting distance measurement results of a virtual object of a real object such as an augmented reality image and an aircraft and a mounting object on a screen of the display apparatus 7.

The operation principle of the virtual object synchronization interface (②) is to attach a marker, which is an input device (⑤), instead of the equipment to be mounted on an aircraft, a ship, or a ground vehicle. Marker) is photographed and the real image is transmitted to the AR simulator (④) through the camera's USB communication.

The mounting computer mockup (⑥) loads the three-dimensional CAD design data (virtual model image) of the mounting equipment to be newly mounted on the real image attached with the marker to the AR simulator 4.

The AR simulator 4 compares and analyzes the real image transmitted from the cognitive device (camera) 3 with a predefined ID through an image processing algorithm, finds a match with the analyzed ID and the marker ID, and matches Based on the computerized mockup of the 3D design data corresponding to the marker ID, the position, size, and direction are determined to synthesize the real image having the marker attached to the actual size and the virtual model image of the mounting, and perform augmented reality rendering.

The AR simulation apparatus 4 uses a distance measuring algorithm between a real object using augmented reality and a virtual object of the mounting, and augments the mounting based on a marker attached to the real image. Measure the shortest distance of the virtual object (mounting equipment) and the distance to the ground.

The AR simulator (④) compares and analyzes the real image transmitted from the camera, which is a cognitive device, with a predefined ID through an image processing algorithm, finds a match with the analyzed ID and the marker ID, and matches the marker ID. Based on the computerized mockup (⑥) of the 3D design data for the ID, the position, size, and direction are judged to synthesize and augment the real image and the virtual model image of the fixture. The operator uses the Cubic Marker that supports 360 degrees when creating the marker so that the viewer can observe the desired position instead of the specific place.

The shortest distance was measured using a distance measurement algorithm of real (aircraft) and virtual mounting equipment (equipment) using a marker-based Augmented Reality technique.

The AR simulator 4 outputs a distance measurement result between augmented reality image and a virtual object of a real object such as an aircraft and a mounted object on a screen of the display device 7.

As shown in FIG. 4, the AR simulator 4 has an operator attached to a real image using a distance measuring algorithm between a real object using augmented reality and a virtual object of a mounting object. Based on the marker (Marker) measures the distance to the virtual object (mounting equipment) of the augmented mounting, and also provides a distance measuring function to the ground. The distance measurement algorithm between the real object and the virtual object of the installation using augmented reality measures the distance from the point where the marker is attached to the actual object to be mounted by the operator. The distance is calculated in real time through calculation processing such as numerical calculation and image correction.

Even in unexpected situations such as the position of the aircraft and the tire flat, the distance measurement algorithm automatically calculates the distance between the virtual machine and the real machine, the virtual machine and the ground in real time by the marker position and attitude correction algorithm.

The algorithm for measuring the distance between the real object and the virtual object of the installation using augmented reality attaches a marker to the object instead of the mounting equipment to be mounted on the aircraft and digitally computes it according to the position and orientation of the marker. Synthesize and enhance the 3D virtual equipment data of the mounting generated by the mockup, and calculate the position information of the vertex position of the augmented 3D virtual equipment data and the position information of the marker that is the distance measurement standard in real time. It is an algorithm that extracts and stores the location coordinates and distance of the shortest distance when distance scan is performed on several points, and can update in real time.

In order to process the whole process in a batch, the operator can use a light touch monitor and USB as a display device (⑦) as shown in FIG. By using a handheld viewer system in which a camera is integrally integrated, an operator can carry and carry an augmented image while carrying the hand. As shown in FIG. 6, a see-through type head mounted display (HMD) is shown. And equipped with a USB camera, you can conveniently shoot and view the screen at the same time.

FIG. 5 is a diagram illustrating a picture of a handheld viewer.

The handheld viewer used as the display device (⑦) is an integrated jig system of camera and touch screen considering the ergonomic user interface, and the integrated input / output jig for the user to view the equipment to be mounted on the aircraft as augmented reality. By integrating the system input method USB camera and input / output touch screen for program operation into one-piece, you can check the part you want to see on the touch screen at the same time. There is a touch screen for setting options and displaying results, and it is a production jig with a USB camera attached to the outside.

6 is a view showing a picture of the See-Through HMD + USB camera system.

7 is a photograph for explaining the suitability for mounting the external mounting using augmented reality (Augmented Reality).

The distance measurement system between the real object and the virtual object using augmented reality is a digital-based three-dimensional R & D modeling and augmentation using Augmented Reality and Computerized Digital Mockup technology in a virtual development environment. Real equipment (real) and virtual equipment (virtual) between virtual models modeled by CAD (Computer Aided Design) of external mountings mounted on real objects such as aircrafts and automobiles existing by simulation In addition, the distance between the virtual equipment and the actual ground is measured by the distance arithmetic algorithm to determine the mounting suitability by measuring the separation distance in real time using the augmented reality technique.

Referring to FIG. 7, an aircraft exterior mounting compatibility simulation system technology using augmented reality technology and modeling & simulation (M & S) technology is newly developed by using three-dimensional design data from advanced equipment design stage. A system technique was used to verify the compatibility of the external installation with the aircraft by applying Augmented Reality-based modeling and simulation (M & S) technology.

8 is a plan view showing the position information of a normal aircraft tire (Tire) and the mounting equipment. 9 is a plan view showing the aircraft tilt calculation during aircraft Right Tire Flat. Figure 10 is a side view showing the aircraft tilt calculation in the aircraft Right Tire Flat. FIG. 11 is a diagram illustrating an inclination calculation of an aircraft after moving to an origin point in an aircraft right tire flat.

12 is a plan view showing the inclination calculation of the aircraft when the aircraft Left Tire Flat. FIG. 13 is a plan view illustrating aircraft tilt calculation during an aircraft NLG tire flat. FIG. 14 is a side view showing the aircraft tilt calculation in the aircraft NLG Tire Flat.

8 to 14, when using the aircraft as a real object (real object), the automatic position calculation and distance measurement algorithm of the virtual mounting equipment according to the aircraft tire flat situation is equipped with new and advanced equipment system considering the aircraft situation It is an algorithm that performs real-time calculation and distance measurement to the ground in order to determine how far the mounting equipment (attachment) to be attached to the actual equipment (real thing) and the ground is separated from the aircraft tire flat situation.

There are 8 types of aircraft tire flats: 1) normal, 2) Right Tire Flat, 3) Left Tire Flat, 4) Nose Tire Flat, 5) Left-Right Tire Flat, 6) Left-Nose Tire Flat, 7) Right-Nose Tire Flat, 8) All Tire (Left-Right-Nose) Flat In accordance with the situation, it is possible to process the location of the mounting equipment and distance from the ground in real time.

8 is a plan view showing the position information of a normal aircraft tire (Tire) and the mounting equipment. FIG. 8 is a diagram illustrating a positional relationship between an external mounting, a front landing gear (NLG) tire, and a main landing gear (MLG) left and right tires. Each position coordinate is converted and used.

9 is a plan view showing the aircraft tilt calculation during aircraft Right Tire Flat. When some tires are actually flat, the aircraft rotates around the other normal tires, but because it simulates a flat situation that does not actually occur in the virtual reality system, the aircraft and its external attachments remain where they are. This is achieved by rotating it at the expected angle. At this time, the coordinates of the rotation axis and the center point to rotate the ground should be obtained based on the coordinate system of the external mounting.

: 항공기 Right 타이어 중심부와 Left 타이어 중심부 까지의 거리(Z축거리),here,

Is the distance between the center of the right tire of the aircraft and the center of the left tire (Z-axis distance),

: 항공기 Right 타이어에서 NLG 타이어 까지의 X축 거리,

: X axis distance from aircraft right tire to NLG tire,

: NLG 타이어와 Left 타이어의 선상에 Right 타이어가 직각을 이루는 점 까지의 거리,

= Distance to right point of right tire on line of NLG tire and left tire,

D _height : X-axis length from the right angle of the right tire to the left tire on the line of the NLG tire and the left tire,

),D _p : X-axis and Z-axis length from the right angle of the right tire on the line of the NLG tire and the left tire to the left tire (

),

θ: The angle seen from the top view of the NLG tire and the left and right tires.

(NLG 타이어와 Left 타이어의 선상에 Right 타이어가 직각을 이루는 점 까지의 거리) 임을 알 수 있다. 이에 필요한 각각의 길이/각도값은 항공기 형상에 의해 결정되며, 항공기 고유의 입력값으로 사용된다. When the right MLG tires are flat, the aircraft rotates around a straight line connecting the left MLG (Main Landing Gear) and the NLL (Node Landing Gear), The radius of rotation required for the angle calculation

(The distance to the right angle of the right tire on the line of the NLG tire and the left tire). Each length / angle value required for this is determined by the aircraft shape and is used as the aircraft's own input.

In order to know the orthogonal points, the coordinates of each distance and the reference point are obtained by using the city method as shown above.

과

을 피타고라스 정리를 이용하여 θ를 구할 수 있다. First, to find θ (the angle from the top view of the NLG tire and the Left and Right tires),

and

Can be obtained using Pythagorean theorem.

))를 구할 수 있다.D _P with the θ obtained here (the angle from the top view of the NLG tire and the left and right tires) (X and Z axis lengths from the right angle of the right tire on the line of the NLG tire and the left tire to the left tire)

)).

We can get D _height and D _weight with θ and D _P.

Here, D _height is the X-axis length from the point where the right tire is perpendicular to the line of the NLG tire and the left tire to the left tire,

)를 나타낸다. D _p is the length of the X and Z axes from the right angle of the right tire on the line of the NLG tire and the left tire to the left tire.

).

If the D _height and the D _weight obtained from the above equations are calculated from the origin, the coordinates of the reference point of the point orthogonal to the right tire are obtained based on the left and NLG tires.

Once the reference point is found, the reference point should be mapped to the origin. Since the rotation transformation of the object is based on the origin, the reference point is mapped to the origin and then rotated to the axis to fit the axis.

First, the position is moved to the origin using the rotation transformation matrix. When the point (x, y, z) is moved by (J, K, L), the following transformation matrix is used.

When the movement is finished, it should be rotated to fit each axis.

First, rotate it about the Y axis to fit the X axis.

The Y-axis rotation transformation matrix is as follows.

After rotating on the Y axis, the heights of the left and NLG tires are different, so the position and angle according to the height should be calculated.

Figure 10 is a side view showing the aircraft tilt calculation in the aircraft Right Tire Flat.

이다.The distance between the center of rotation of the front landing gear (NLG) and the center of rotation of the main landing gear (MLG) corresponds to the tire radius (D _N , D _L ).

to be.

이므로, θ₁ 값을 구할 수 있다.

Therefore, the value of θ ₁ can be obtained.

Where D _N is the height from the ground to the center of the NLG tire,

D _L : the height from the ground to the center of the Left (Right) tire,

: Left(Right)타이어와 NLG타이어의 높이 차이,

Is the height difference between the left (Right) tire and the NLG tire,

θ ₁ : Angle according to the height of NLG tire and Left (Right) tire when viewed from left or right,

: Left 타이어와 Right타이어의 중심부를 NLG타이어와 연결했을 때의 실제 길이

이다.

: Actual length when the center of left tire and right tire are connected with NLG tire

to be.

Rotate the Z axis by the angle θ ₁ according to the height difference between the left and NLG tires. The Z-axis rotation transformation matrix is

Once the axis has been aligned by movement and rotation, it should be rotated by the angle when the right tire is flat.

At this time, the angle θ ₂ can be obtained as follows.

FIG. 11 is a diagram illustrating an inclination calculation of an aircraft after moving to an origin point in an aircraft right tire flat.

)는 다음 식에 의해 구한다. The distance to the point where the right tire is perpendicular to the line of the NLG tire and the left tire (

) Is obtained by the following equation.

: NLG 타이어와 Left 타이어의 선상에 Right 타이어가 직각을 이루는 점 까지의 거리,

: 항공기 Right 타이어 중심부와 Left 타이어 중심부 까지의 거리(Z축거리), D_P :NLG타이어와 Left 타이어의 선상에 Right 타이어가 직각을 이루는 점으로부터 Left 타이어까지의 X축과 Z축길이(

), θ₂ : Right 타이어 Flat시에 발생되는 각도, R_L: 지면에서 Left(Right) 타이어의 중심까지의 높이를 나타낸다. here,

= Distance to right point of right tire on line of NLG tire and left tire,

: Distance between the center of the right tire of the aircraft and the center of the left tire (Z-axis distance), D _P : X-axis and Z-axis length from the right angle of the right tire to the left tire on the line of the NLG tire and the left tire (

), θ ₂ : Angle at right tire flat, R _L : Height from the ground to the center of Left (Right) tire.

The right tire can be flattened by rotating the X-axis by θ ₂ obtained above. X-axis rotation can be expressed by the following matrix equation.

Rotate θ ₂ on the X axis, then move and rotate to its original position to determine the position of the external attachment and the shortest distance to the ground. To move and rotate to the original position, rotate in the reverse order of the conversion.

First, rotate it by -θ _{1 on the} Z-axis, rotate it by -θ _{1 on the} Y-axis, and then move it by (-J, -K, -L). The position of the water object is determined and the shortest distance is obtained by measuring the distance between the object's information and the ground.

12 is a plan view showing the inclination calculation of the aircraft when the aircraft Left Tire Flat. In the left tire flat, since the Z-axis of the external attachment is located at the center of the left and right tires, the angles θ, θ ₁ and θ ₂ are the same as those of the right tire flat. Therefore, the rotation and the movement direction may be calculated with opposite signs.

: 항공기 Right 타이어 중심부와 Left 타이어 중심부 까지의 거리(Z축거리), here,

Is the distance between the center of the right tire of the aircraft and the center of the left tire (Z-axis distance),

: 항공기 Right 또는 Left 타이어에서 NLG 타이어 까지의 X축 거리,

Is the X-axis distance from the aircraft's right or left tires to the NLG tires,

: NLG타이어와 Right 타이어의 선상에 Left 타이어가 직각을 이루는 점 까지의 거리,

= Distance to the point where the left tire is perpendicular to the line of the NLG tire and the right tire,

D _height : X-axis length from the point where the left tire is perpendicular to the line of the NLG tire and the right tire to the right tire,

), D _P : X-axis and Z-axis length from the point where the left tire is perpendicular to the line of the NLG tire and the right tire to the right tire (

),

θ: Angle seen from the top view of NLG tires with left and right tires

FIG. 13 is a plan view illustrating an inclination calculation of an aircraft in an aircraft NLG tire flat, and FIG. 14 is a side view illustrating an inclination calculation of an aircraft in an aircraft NLG tire flat.

In NLG Tire Flat, the reference point should be based on the center of the left and right tire segments. After positioning to the midpoint of the reference point using the movement matrix, the NLG tire should be rotated to a flat height. Here, the rotation angle can be obtained as follows.

Here, θ _1: an angle of the tire and NLG Left (Right) tires in the back,

θ ₂ : Angle generated when the NLG tire is flat and the final angle with the tire behind it is θ ₁ + θ ₂ ,

R _L : Height from the ground to the center of the left (right) tire

RN: Height from ground level to center of NLG tire

Therefore, the external mounting position information of the NLG tire flat can be rotated by θ _{2 on the} -Z axis and the reference point P is moved to the original position.

As described above, the method of the present invention is implemented as a program, and can be read by using software and a device of a personal computer (CD-ROM, RAM, ROM, floppy disk, flash memory, hard disk, magneto-optical disk, etc.). Can be stored in.

As described above, although the preferred embodiment of the present invention has been described by way of example, the scope of the present invention is not limited to only this specific embodiment, and thus the present invention is defined in the spirit and scope of the present invention. Modifications, changes or improvements in various forms within the scope of the invention should be construed as being included for all technical ideas within the scope equivalent to the claims below.

1: physical mounting target 2: virtual object synchronization interface
3: cognitive device 4: AR simulator
5: Input device 6: Attachment computer mockup
7: display device

Claims (15)

In the distance measurement system between real and virtual model using augmented reality,
Real objects of aircraft, warships or ground vehicles; And
Augmented reality-based modeling and simulation (M & S) to verify in advance the mounting suitability of the real and the external mounting using three-dimensional design data in the advanced equipment design stage for the external mounting mounted on the real object. Modeling & Simulation) is applied to the virtual object synchronization interface that analyzes the mounting suitability by calculating the distance between the real and the external mounting as the augmented reality of the CAD data of the same size before mounting the external mounting equipment
The virtual object synchronization interface may include a cognitive device using a small camera for providing an image, such as a real object, to an object to be mounted in augmented reality; After the synthesis of the real image using the augmented reality from the cognitive device and the virtual model image of the external mounting after the distance measuring algorithm for measuring the distance between the two objects of the real and the external mounting using a marker An AR simulator for driving; An input device using a marker to determine an operator's position / line of sight; A mounting computer mockup for loading three-dimensional CAD design data (computation mock-up data) of a mounting apparatus for newly mounting on the real object image to which the marker is attached; And a visual display device for outputting a distance measurement result of a virtual model of a real object such as an aircraft and an augmented reality on a screen, and a distance measuring system using the augmented reality. The method of claim 1,
The recognition device,
It uses the small camera and supports 30Hz or more in real time input / output in terms of performance in consideration of weight or size to receive real-time video and free the operator's movement, and supports Auto WhiteBalance, Focus, and Contrast. (Contrast) Distance measurement system between the real and virtual models using augmented reality, characterized in that it has a function. The method of claim 1,
The AR simulator,
Acquire physical images related to aircraft, warships, and ground vehicles from the camera used as the recognition device, attach markers to the physical images by the input device, and attach the equipment (mounting equipment) from the mounting computer mockup. Performs augmented reality rendering by loading the mockup data of the computer and the actual image with the marker attached to the actual size, extracting the separation distance between two objects, and outputting the mounting suitability result report file to the display device. Data input / output module;
Camera calibration (camera calibration, feature point detection and real-time tracking), marker recognition and tracking, marker geometry calculation (3D position, orientation calculation through feature point matching), gesture and event recognition of the cognitive device, 2D image and graphic model synthesis An image processing module for providing augmented reality synthesis by applying geometric information of a virtual model of a real image and a mounted object; And
The physical image and the virtual model of the mounting is composed of a conformity test module for determining the suitability of the mounting by measuring the distance between the matched model and the surrounding object (real) and the separation distance between the physical equipment and the mounting equipment Distance measurement system between real and virtual models using augmented reality. The method of claim 1,
The AR simulator,
Using a computer or a laptop, the screen image input from the cognitive device is augmented in real time through image processing, and the distance between the real image on which the marker is attached and the fixture is measured in real time by using augmented reality. Distance measurement system between the real and virtual models using augmented reality, characterized in that to perform a function of processing. The method of claim 1,
The input device,
It is installed at the point to be defined as the reference point for measuring the augmented position and distance of the 3D image, and the operator wants to measure the actual distance between the real object and the virtual object such as an aircraft. A distance measuring system between a real model and a virtual model using augmented reality, characterized in that a reference point is provided during distance measurement by installing a marker defined in a destination. The method of claim 1,
The mounting computer mockup
Before the production of the fixture to be mounted on real objects, such as aircraft, warships, military vehicles, etc., the 3D CAD design data was used to mount virtually as if the fixture equipment was mounted on a real object such as an aircraft. Distance measurement system between real and virtual models using augmented reality, characterized in that to provide three-dimensional digital CAD data of the water. The method of claim 1,
The display device,
See-Through type HMD (Head Mounted Display), Handheld Viewer, Monitor, any one of the equipment to be mounted on the real thing (mounting) to see the image augmented reality to output the image And providing a function of outputting a distance measurement result between the real object and the virtual object of the mounting unit on a screen. The method of claim 7, wherein
The handheld viewer,
It is an integrated jig system of camera and touch screen considering ergonomic user interface, and it is an integrated jig for user to see the equipment to be mounted on the aircraft as augmented reality.It is a system input method of USB camera and input / output method for program operation. The integrated touch screen allows users to check the part they want to see on the touch screen at the same time as shooting, with the handles on both sides of the jig, and the touch screen for program execution, option setting, and the result display on the front, and USB on the outside. Distance measuring system between the real model and the virtual model using augmented reality, characterized in that using the production jig attached to the camera. The method according to any one of claims 1, 3, and 4,
The AR simulator,
Using a marker-based Augmented Reality technique using augmented reality, characterized in that to measure the shortest distance using a distance measurement algorithm of the real (aircraft) attached to the marker and the virtual mounting equipment (equipment) Distance measuring system between real and virtual models. In the distance measuring method between a real object and a virtual model of the mounting using augmented reality,
(a) Attaching a marker as an input device to a real object such as an aircraft, a warship or a land vehicle, and using a camera used as a recognition device, a marker portion is placed on a real image by the input device. Photographing and transmitting the real image to the AR simulation device through the USB communication of the camera;
(b) loading the 3D CAD design data (computation mock-up data of the mounting) of the mounting equipment for newly mounting on the real image with the marker from the mounting computer mock-up to the AR simulator;
(c) comparing and analyzing the real image transmitted from the camera by the AR simulation device with an ID previously defined through an image processing algorithm, finding a match with the analyzed ID and the marker ID, and matching the matched marker ID. Determining position, size, and direction according to the computerized mockup of the 3D design data, synthesizing the real image with the marker attached to the actual size and the virtual model image of the mounting, and performing augmented reality rendering;
(d) Augmented by the AR simulation apparatus based on a marker attached to the real image using a distance measuring algorithm of a real object using augmented reality and a virtual object of a mounting object. Measuring the shortest distance of the virtual object (mounting equipment) of the mounting, and measuring the distance to the ground; And
(e) outputting a result of distance measurement between a real object such as an augmented reality image and an aircraft and a virtual object of a mounting unit on a screen of the display device;
Distance measurement method between the real and virtual models using augmented reality comprising a. The method of claim 10,
In step (d),
The algorithm for measuring the distance between a real object and a virtual object using the augmented reality is to mount a marker on a real object photographed by a camera instead of the equipment to be mounted on an aircraft, and to position the marker on the real image. Synthesize and enhance the 3D virtual equipment data of the mounting generated by the digital computer mockup according to the direction and direction, and calculate the vertex position information of the enhanced 3D virtual equipment data and the position information of the marker which is the distance measurement standard in real time. The distance measurement method between the real model and the virtual model using augmented reality, characterized in that for extracting the distance from the coordinates, and storing the position coordinates and distance of the shortest distance when the distance scan (Distance Scan). The method of claim 10,
The step (d)
The distance measuring algorithm between the real object using the augmented reality and the virtual object of the mounting is a distance from the point where the marker is attached to the position of the mounting designated by the operator with respect to the actual aircraft. The distance calculation algorithm calculates the distance in real time through the calculation process of numerical calculation and image correction, and the distance measurement algorithm is automatically adjusted by the marker position and attitude correction algorithm even in case of unexpected situation such as the position of the aircraft and the tire flat. The distance measurement method between the real model and the virtual model using augmented reality, characterized in that to calculate in real time the distance measurement between the virtual equipment (mounting) and the actual equipment (real), the virtual equipment and the ground. The method of claim 10,
Step (e),
For convenience of operator's movement, augmented image is displayed by using a handheld viewer system that is integrated with a light touch monitor and USB camera so that users can shoot with the camera while viewing the recorded image. Method for measuring the distance between the real model and the virtual model using augmented reality characterized in that by mounting a USB camera on the HMD (Head Mounted Display) to conveniently shoot and output to the screen. The method of claim 10,
In case of using the aircraft as the real object, the algorithm of automatic position calculation and distance measurement of the virtual mounting equipment according to the aircraft tire flat situation is 1) steady state, 2) right tire flat, 3) left tire flat, 4) Nose Tire Flat, 5) Left-Right Tire Flat, 6) Left-Nose Tire Flat, 7) Right-Nose Tire Flat, 8) All Tire (Left-Right-Nose) Flat The distance calculation can be processed in real time, and it is to verify whether the new and advanced equipment system is installed in consideration of the aircraft tire flat situation, and the equipment to be attached to the actual equipment (mounting) of the aircraft in the aircraft tire flat situation. A distance measuring method between the real model and the virtual model using augmented reality, characterized in that the distance to the ground and real-time calculation to determine how far apart from the ground. A computer-readable recording medium having recorded thereon a program for implementing a distance measuring method between a real object using augmented reality and a virtual object of a mounting device,
(a) Attaching a marker as an input device to a real object such as an aircraft, a warship or a land vehicle, and using a camera used as a recognition device, a marker portion is placed on a real image by the input device. Capturing and transmitting a real image to an AR simulation device through a USB communication of a camera;
(b) loading a three-dimensional CAD design data (computation mock-up data of the mounting) of the mounting equipment for newly mounting on the real image with the marker from the mounting computer mock-up to the AR simulator;
(c) comparing and analyzing the real image transmitted from the camera by the AR simulation device with an ID previously defined through an image processing algorithm, finding a match with the analyzed ID and the marker ID, and matching the matched marker ID. Determining the position, size, and direction of the 3D design data according to the computerized mock-up of the 3D design data, synthesizing the real image with the marker attached to the actual size and the virtual model image of the mounting, and performing augmented reality rendering;
(d) Augmented by the AR simulation apparatus based on a marker attached to the real image using a distance measuring algorithm of a real object using augmented reality and a virtual object of a mounting object. A function of measuring a shortest distance of a virtual object (mounting equipment) of the mounting and measuring a distance to the ground; And
(e) a function of outputting a distance measurement result of a virtual object of a real object such as an augmented reality image and an aircraft and a mounting object on a screen of a display apparatus;
Readable recording medium having recorded thereon a program for realizing the program.

Images