KR20120015802A - System and method for managing vehicle parts using augmented reality - Google Patents

Abstract

PURPOSE: A vehicle component management system and a method thereof using augmented reality are provided to efficiently manage the components of a vehicle by supplying the malfunction information of a vehicle component. CONSTITUTION: A wireless communication unit(21) receives the malfunction information of a vehicle component from a component information manager(14). An augmented reality processor(22) recognizes each component from an image through an augmented reality technique. The augmented reality processor generates an augmented reality image.

Description

Vehicle parts management system using AR and its method {SYSTEM AND METHOD FOR MANAGING VEHICLE PARTS USING AUGMENTED REALITY}

The present invention relates to a vehicle parts management system and method using augmented reality, and more particularly, to manage the failure information of the vehicle parts in conjunction with the failure diagnosis system of the vehicle, the failure information of the vehicle parts using augmented reality technology The present invention relates to a vehicle parts management system using augmented reality and a method thereof.

Augmented reality is a technology that superimposes a virtual object on the real world seen by the user. It is also called Mixed Reality (MR) because it combines the virtual world with additional information in real time into one image.

Augmented reality, a concept that complements the real world with the virtual world, uses a virtual environment made of computer graphics, but the main role is the real environment. Computer graphics serve to provide additional information needed for the real world. This means that the distinction between the real environment and the virtual screen is blurred by overlapping the 3D virtual image on the live image that the user sees.

Virtual reality technology immerses the user in the virtual environment and cannot see the real environment. However, augmented reality technology, which is a mixture of real environment and virtual objects, provides the user with a better sense of reality and additional information. For example, when the camera is illuminated with a smartphone camera, information such as the location and phone number of a nearby store is displayed as a stereoscopic image.

Various applications using such augmented reality technology have been provided, but it has not been used in applications that manage and provide fault information of vehicle parts.

In order to solve the problems of the prior art as described above, the present invention manages the failure information of the vehicle parts in conjunction with the failure diagnosis system of the vehicle, by providing a failure information of the vehicle parts using augmented reality technology, An object of the present invention is to provide a vehicle parts management system using augmented reality and a method for managing the parts of the vehicle.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention which are not mentioned can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

The apparatus of the present invention for achieving the above object, the component information storage unit for storing the failure information of the vehicle component; And a part information management unit which collects the fault information of the vehicle parts, stores the part information in the parts information storage unit, and provides the failure information of the vehicle parts to the parts information providing device through wireless communication.

In addition, the method of the present invention for achieving the above object comprises the steps of collecting and storing failure information of the vehicle parts; Providing fault information of the vehicle component to an apparatus for providing component information through wireless communication; Generating, by the component information providing apparatus, each component in the image using an augmented reality technology and generating an augmented reality image in which failure information is displayed on the image of the corresponding component; And displaying the augmented reality image on a screen.

The present invention as described above, in conjunction with the vehicle failure diagnosis system to manage the failure information of the vehicle parts, by providing a failure information of the vehicle parts by using augmented reality technology, to efficiently manage the parts of the vehicle It works.

In addition, the present invention has the effect of effectively managing the parts of the vehicle, to ensure the safety of the driver, as well as to prevent unnecessary replacement of parts by some unscrupulous repairman.

1 is a configuration diagram of an embodiment of a vehicle parts management system using augmented reality according to the present invention;
2 is an example of an augmented reality image according to the present invention,
3 is a flow chart of an embodiment of a vehicle parts management method using augmented reality according to the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, It can be easily carried out. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram of an embodiment of a vehicle parts management system using augmented reality according to the present invention.

As shown in FIG. 1, a vehicle parts management system using augmented reality according to the present invention includes a vehicle parts management device (for example, a cubis system) 10 and a parts information providing device (for example, a mobile terminal) 20. ).

The vehicle parts management device 10 is mounted on a vehicle to manage failure information of the vehicle parts in cooperation with a vehicle failure diagnosis system (for example, Mozen Telematics system), and communicates with the parts information providing device 20 to manage the failure information. Transmit fault information of a vehicle part. Here, the vehicle failure diagnosis system includes a telematics center server and diagnoses the vehicle in conjunction with the vehicle control device.

At this time, the vehicle control device includes an engine control unit (ECU) for controlling the overall operation of the engine, a motor control unit (MCU) for controlling the overall operation of the electric motor, and a transmission control unit (transmission control unit) for controlling the transmission. , TCU), a battery management system (BMS) for controlling the operation of the battery, and an air conditioner controller (Full Auto Temperature Controller, FATC) in charge of room temperature control.

The component information providing apparatus 20 displays the failure information of the vehicle component received from the vehicle component management apparatus 10 on the corresponding component of the image acquired through a camera or the like based on the augmented reality technology.

Looking at each of the components in more detail, the vehicle component management apparatus 10 is a first wireless communication unit 11, component information storage unit 12, input / output unit 13, and component information management unit 14 ).

The first wireless communication unit 11 communicates with the component information providing apparatus 20 by using a Bluetooth communication method or a near field communication method, and under the control of the component information management unit 14, the component information storage unit ( The failure information of the vehicle parts stored in 12 is transmitted to the parts information providing apparatus 20, and the replacement information of the vehicle parts is transmitted from the parts information providing apparatus 20 to manage the history of the vehicle parts. In addition, the first wireless communication unit 11 transmits and receives data generated during the authentication process to the device information providing apparatus 20.

The part information storage unit 12 classifies the vehicle parts into an engine, a braking device, a transmission, and the like (large classification), classifies each part into detailed parts (small classification), and manages failure information.

For example, when a failure occurs in the fuel shutoff valve in the engine, the following information is stored.

① Description of the failure: "The current problem with the vehicle is related to the engine. It seems to be related to the wiring or single unit of the fuel shutoff valve, which is an important electronic component of the engine. Has occurred. For your safety and vehicle protection, please immediately go to the nearest service center or service provider and have it checked for maintenance. " Or more abbreviated: "The current problem with the vehicle seems to be an abnormality in the relevant wiring or components of the fuel shutoff valve, which is an important electronic component of the engine. For immediate driver safety and vehicle protection, go immediately to the nearest maintenance partner. And ask for inspection or maintenance. "

② Cause of the failure is stored "Information deterioration and malfunction of shutoff valve to shut off fuel" is stored.

③ It serves as a faulty part and stores the information "It is a device to cut off the fuel line between the fuel tank and the injector at the time of starting stop."

As another example, when an error occurs in the brake disc in the braking device, the following information is stored.

① As a result of the breakdown, the current phenomenon that has occurred in the vehicle is related to the vehicle body control system. It seems that the brake system is overheated. The TCS warning lamp lights up on the instrument panel and the vehicle bodywork control function is suspended. In this case, the ABS brake works normally, so there is no significant effect on the braking function. However, there is no body position control, which may affect driving stability. When the brake system is at normal temperature, the TCS warning lamp goes out and the system is operating normally. If the TCS warning light does not go out continuously, please immediately go to the nearest service center or maintenance partner and ask for inspection or maintenance for your safety and vehicle protection. " The abnormality occurred is related to the body attitude control device. The TCS warning lamp lights up on the instrument panel. If the TCS warning light does not go out continuously, please immediately go to the nearest service center or maintenance partner and ask for inspection or maintenance for your safety and vehicle protection. "

② Cause of failure, the information "overheating of brake device or disconnection, short circuit and contact failure of power supply related wiring of ABS and vehicle body attitude control device is expected."

③ It is a part of the faulty part, and it stores the information "It controls the ABS and TCS device."

The failure information differs slightly depending on the vehicle type, and only some types of parts are indicated.

The input / output unit 13 may be implemented as an audio video navigation (AVN) device, for example, and displays a failure information of a part upon request from a driver, and replaces a corresponding part when a driver replaces a specific part. Get input. That is, the driver inputs the corresponding information every time the vehicle parts are replaced to be managed as the history information.

The parts information management unit 14 controls the first wireless communication unit 11, the parts information storage unit 12, and the input / output unit 13, and collects the failure information of the vehicle parts in cooperation with the failure diagnosis system of the vehicle. To the component information providing apparatus 20. It may also be provided to the driver through the input / output unit (13).

In addition, the component information management unit 14 performs the authentication process in communication with the component information providing apparatus 20 by using Bluetooth communication or short-range wireless communication. By providing a, his vehicle information is prevented from being provided to others.

In addition, the component information management unit 14 receives the replacement information of the component from the component information providing apparatus 20 and manages the history of the corresponding component.

Meanwhile, the component information providing apparatus 20 may include a second wireless communication unit 21, an augmented reality processor 22, a controller 23, a display unit 24, and an image acquisition unit (camera) for acquiring a part image of a vehicle ( 25).

The second wireless communication unit 21 communicates with the vehicle parts management apparatus 10 using Bluetooth communication or near field communication, and transmits and receives authentication data and receives failure information of the vehicle parts from the vehicle parts management apparatus 10. The replacement information for the history management of the part is received from the user and transmitted to the vehicle parts management apparatus 10.

The augmented reality processor 22 recognizes a vehicle part from an image acquired by the image acquisition unit 25 and generates an augmented reality image in which fault information is displayed on an image of the corresponding vehicle part using an augmented reality technology. For example, when the engine is recognized from the image, the image and the failure information of the defective part in the engine are displayed. The augmented reality image is shown in FIG. 2 as an example, and the image and the failure information of the corresponding part are displayed on the oval-shaped part.

At this time, the augmented reality processor 22 may recognize the type of the vehicle component using the augmented reality technology and then directly display the failure information on the position of the corresponding component, or may display its name. When displaying the name, the controller 23 provides the corresponding failure information as detailed information when the user touches the corresponding name through the touch pad function.

Here, the augmented reality processor 22 detects the type of the vehicle component in association with the three-dimensional geometric database of the vehicle component, for example. The three-dimensional geometric database refers to a database in which three-dimensional geometrical information of parts and lengths, shapes, and colors of parts are input and stored using a three-dimensional scanner.

In addition, when the augmented reality processor 22 recognizes the vehicle in the image, the augmented reality processor 22 may collectively display an image of the faulty part and the fault information at a corresponding position of the vehicle.

The control unit 23 controls the second wireless communication unit 21, the augmented reality processor 22, the display unit 24, and the image acquisition unit 25, but the failure of the component transmitted through the second wireless communication unit 21. The augmented reality processor 22 controls the augmented reality processor 22 to generate an augmented reality image displayed on the image of information or failure information of a component stored in a storage unit (not shown) received through the second wireless communication unit 21. .

The display unit 24 has a touch screen function and displays an augmented reality image generated by the augmented reality processor 22, and displays the user's request for detailed information through the touch screen function.

Hereinafter, the augmented reality processor 22 will be described in more detail. Augmented reality technology is generally well known and the present invention provides the driver with management information of each part in the vehicle, for example, in the following manner.

The augmented reality processor 22 includes a parallel processor circuit and a tracker, and the parallel processor circuit includes a first processor and a second processor.

The first processor included in the parallel processor circuit processes the image of the vehicle component photographed by the camera, so that the coordinate values of the camera markers projected at the intersections of the corners representing the pose of the component, the image captured by the camera Projection matrix values for displaying an image of a part on the screen, and normal vector values of planes represented by triangles included in a polygonal pyramid generated by virtually connecting the centers and intersections of the cameras are calculated.

The tracker generates three-dimensional information data by tracking the attitude and position of the part based on the coordinate values, the projection matrix values, and the normal vector values of the markers calculated by the first processor. The tracker may further include a compensator for correcting an error between the posture and the position of the tracked part and the posture and the position of the actual part, and generating three-dimensional information data. According to an embodiment, the compensator is located outside the tracker and The posture and position information of the tracked part may be received from the tracker to correct the error between the posture and the position of the tracked part and the posture and the position of the actual part, and then generate 3D information data.

The second processor included in the parallel processor circuit receives the 3D information data and the vertex data of the target image model appearing together with the component in the 3D virtual space and renders the received 3D information data and the vertex data. performing a rendering operation to generate an augmented reality image in which the part and the target image model appear as a virtual 3D image.

In general, it can be said that the 3D image is divided into a plurality of polygons. For example, assuming that a 3D image is composed of a plurality of triangles, vertices mean vertices of triangles, and vertex data may be location and color information of vertices. In this case, the second processor may receive the vertex data and connect the vertices, and determine the color of the vertex at the pixel or fragment level.

At this time, the controller 23 controls the operations of the parallel processor circuit and the tracker. For example, the operation of the tracker can be stopped while the parallel processor circuit is in operation, and the operation of the parallel processor can be stopped while the tracker is in operation. That is, the controller 23 can activate and deactivate the operations of the parallel processor circuit and the tracker.

3 is a flow chart of an embodiment of a vehicle parts management method using augmented reality according to the present invention.

First, the vehicle component management apparatus 10 stores failure information of the vehicle component (301). That is, the vehicle component management apparatus 10 collects and stores failure information of the vehicle component in cooperation with the failure diagnosis system of the vehicle.

Thereafter, the vehicle component management apparatus 10 provides the failure information of the vehicle component to the component information providing apparatus 20 through wireless communication (302).

Subsequently, the component information providing apparatus 20 recognizes each component in the image by using an augmented reality technology and generates an augmented reality image in which failure information is displayed on the image of the corresponding component (303).

Thereafter, the device information providing apparatus 20 displays the generated augmented reality image on the screen (304).

Therefore, the driver can be provided with the failure information of the parts in his vehicle on the screen through the parts information providing device 20, it is possible to easily manage the parts of the vehicle.

On the other hand, the method of the present invention as described above can be written in a computer program. And the code and code segments constituting the program can be easily inferred by a computer programmer in the art. In addition, the written program is stored in a computer-readable recording medium (information storage medium), and read and executed by a computer to implement the method of the present invention. The recording medium may include any type of computer readable recording medium.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The present invention is not limited to the drawings.

10: vehicle parts management apparatus 11: the first wireless communication unit
12: part information storage unit 13: input / output unit
14: parts information management unit 20: parts information providing device
21: second wireless communication unit 22: augmented reality processing unit
23 control unit 24 display unit
25: image acquisition unit

Claims (5)

A component information storage unit for storing failure information of the vehicle component; And
Part information management unit that collects the failure information of the vehicle parts and stores it in the parts information storage unit, and provides the failure information of the vehicle parts to the parts information providing device through wireless communication
Vehicle parts management system using augmented reality comprising a.
The method of claim 1,
The parts information management unit,
Vehicle parts management system using augmented reality, characterized in that for receiving the replacement information of the parts from the parts information providing device through wireless communication to manage the history of the corresponding parts.
The method of claim 1,
The part information providing device,
A wireless communication unit which receives failure information of a vehicle part from the parts information management unit;
An augmented reality processor for recognizing each part in an image using an augmented reality technology and generating an augmented reality image in which fault information is displayed on an image of the part;
A display unit displaying the augmented reality image; And
Control unit for transmitting the failure information of the vehicle component to the augmented reality processor, and controls the display unit to display the augmented reality image
Vehicle parts management system using augmented reality comprising a.
Collecting and storing failure information of the vehicle component;
Providing fault information of the vehicle component to an apparatus for providing component information through wireless communication;
Generating, by the component information providing apparatus, each component in the image using an augmented reality technology and generating an augmented reality image in which failure information is displayed on the image of the corresponding component; And
Displaying the augmented reality image on a screen;
Vehicle parts management method using augmented reality comprising a.
The method of claim 4, wherein
Managing the history of a corresponding part by receiving replacement information of the part from the part information providing device;
Vehicle parts management method using augmented reality further comprising a.

Images