DE102009008039A1 - Augmented reality image producing method for manufacturing technical component of land vehicle, involves selecting color and/or texture of virtual image component, and comparing component with part of real image to augmented reality image - Google Patents

Abstract

The method involves receiving a real image (RB) by a camera, and determining position of a virtual image component (VIRT) to the real image received by the camera. Color and/or texture of the real image at the position of the virtual image component is determined. Color and/or texture of the virtual image component is selected based on the color and/or texture of the real image at the position of the virtual image component. The virtual image component is compared with a part of the real image to an augmented reality image (ARB). Independent claims are also included for the following: (1) a method for manufacturing a technical component of a motor vehicle or a motor vehicle component (2) a device for manufacturing a technical component of a motor vehicle or a motor vehicle component.

Description

The The invention relates to a method for producing a technical Component, in particular a motor vehicle or a motor vehicle component, by means of an augmented reality system, which displays a display of a virtual image.

The EP 0 949 513 A2 discloses an augmented reality system including a tracker system for determining the relative position between a sensor for determining a pattern of reference points on an object surface and the object surface, the augmented reality system including a processor connected to the sensor.

The WO 01/96829 A1 discloses a method for automotive production wherein an examiner is assisted in performing an inspection for quality defects of a motor vehicle or a motor vehicle component by an Augmented Reality (AR) system that captures and / or posts test-relevant data the examiner outputs, the test in the context of the production of the motor vehicle or the motor vehicle component for detecting quality defects in the motor vehicle or the motor vehicle component in at least one predetermined manufacturing state, wherein a worker in the implementation of individual, the test upstream production steps for Manufacture of other motor vehicles or automotive components via the augmented reality system is supplied based on at least one of the identified quality deficiencies caution, and wherein a post-processor in the implementation of a test downstream Na In the case of a faulty motor vehicle or a defective motor vehicle component, error information about the augmented reality system based on the respectively identified quality defects is supplied to the driver.

According to the WO 01/96829 A1 the term augmented reality is defined as a superposition of a real environment with a computer-generated environment, ie a virtual world. Augmented reality is thus a kind of human-technology interaction that fills in the user's information in his field of vision and thus expands his perception. This is context-dependent, ie adapted to a particular situation in which the user is currently located. In particular, image or text data can be dazzled as information. By overlaying the virtual and the real environment, the user has the option of simply carrying out target / actual comparisons. Furthermore, an augmented reality system can be used for easy visualization of complex information of any kind without overlapping with the real field of view.

It Object of the invention, the cost of production a technical component. It is still a task of the invention, the cost of producing a motor vehicle to lower. It is also an object of the invention, the quality to increase in the production of a motor vehicle and in particular the error rate in the production of a motor vehicle to lower.

The aforementioned The object is achieved by a method for generating an augmented reality image, wherein a real image is captured by a camera, wherein the position of a virtual image component to the means of Camera recorded real image is determined, taking the color and / or the texture of the real image at the position of the virtual Image components is determined, with the color and / or texture of the virtual image component depending on the color and / or the texture of the real image at the position of the virtual Image components is selected, and where the virtual Picture component with at least a part of the real picture to a Augmented reality image is combined. It is especially provided that the virtual part is placed at the determined position becomes.

In particular, a position in the sense of the invention is not punctiform information but an area. The position of a virtual image component relative to the real image captured by the camera is, in particular, the area of the real image to be overlaid by the virtual image component or a slightly larger area (for example 10%). Such an area should be understood in particular to mean the extent of the virtual image component. For example, if the virtual component is a text, the region is not only the area of the real image covered by the text but, in particular, the area of the real image covered by the text field of the text or something (eg, by 10%). ) larger area. The choice of the color and / or the texture of the virtual image component is chosen in particular so that the virtual image component is easily recognizable. For example, if the region of the real image covered by the virtual image component is green, the virtual image component is displayed in red, for example. The aforementioned object is also achieved by a method for displaying a virtual image component on a transparent display, wherein a real image is recorded by means of a camera, which images the surroundings of the display, which a viewer sees when viewing through the display, wherein the Position of a virtual image component to the captured by the camera real Image is determined, wherein the color and / or the texture of the real image is determined at the position of the virtual image component, wherein the color and / or texture of the virtual image component depending on the color and / or the texture of the real image at the position the virtual image component is selected, and wherein the virtual image component is displayed by means of the transparent display. It is provided in particular that the virtual component is placed at the determined position.

In Advantageous embodiment of the invention gives the virtual image component a piece of information relating to the assembly of a component. In a further advantageous embodiment of the invention is the virtual Picture component a part number.

In Furthermore, an advantageous embodiment of the invention becomes an mounting component marked by the virtual image component or marked and the so marked or marked component in a technical component, in a motor vehicle or in a motor vehicle component mounted.

In Furthermore, advantageous embodiment of the invention, the mounting location a component to be mounted by means of the virtual image component marked or marked and the component to be mounted in a technical Component, in a motor vehicle or in a motor vehicle component mounted on the so marked or marked mounting location.

The aforementioned Task is - especially in conjunction with the above Characteristics - moreover, by a method for producing a technical component, in particular of a motor vehicle or a motor vehicle component, solved, wherein a real Image of a component to be assembled and / or a mounting location of the or of a component is recorded by means of a camera, wherein the Position of a virtual image component to that by means of the camera recorded real image is determined, the color and / or the texture of the real image at the position of the virtual image component is determined, with the color and / or texture of the virtual Picture components depending on the color and / or the Texture of the real image at the position of the virtual image component is selected, wherein the virtual image component with at least a part of the real image combined into an augmented reality image or by means of a transparent display, and wherein the component to be mounted in the technical component, in the motor vehicle or in the motor vehicle component is mounted.

In Furthermore, advantageous embodiment of the invention gives the virtual Picture component a information relating to the assembly of a component again. In a further advantageous embodiment of the invention the virtual image component is a part number.

In Furthermore advantageous embodiment is the component to be mounted named or marked by means of the virtual image component and the named or marked component in the motor vehicle or the Motor vehicle component mounted. In a further advantageous embodiment becomes a desired mounting location by means of the virtual Picture components marked or marked and the / a component in the motor vehicle or in the motor vehicle component on the mounted mounting location.

• – ein, insbesonders stationäres oder mobiles, Display,
• – eine dem Display zugeordnete Kamera zur Aufnahme eines realen Bild,
• – ein, insbesondere ein Trackingsystem umfassendes, System zur Bestimmung der Position eines virtuellen Bildbestandteils zu dem mittels der Kamera aufgenommenen realen Bild,
• – ein Bildbearbeitungsmodul zur Bestimmung der Farbe und/oder der Textur des realen Bildes an der Position des virtuellen Bildbestandteils und zur Auswahl der Farbe und/oder der Textur des virtuellen Bildbestandteils in Abhängigkeit der Farbe und/oder der Textur des realen Bildes an der Position des virtuellen Bildbestandteils sowie
• – einen Szenen-Generator zur Kombination des virtuellen Bildbestandteils mit zumindest einem Teil des realen Bildes zu einem Augmented-Reality-Bild

umfasst.The aforementioned object is also achieved, in particular in conjunction with the aforementioned features, by a device for generating an augmented reality image, wherein the device

• - a, in particular stationary or mobile, display,
• A camera associated with the display for capturing a real image,
• A system comprising, in particular, a tracking system for determining the position of a virtual image component relative to the real image recorded by means of the camera,
• An image processing module for determining the color and / or the texture of the real image at the position of the virtual image component and for selecting the color and / or the texture of the virtual image component as a function of the color and / or the texture of the real image at the position of the image virtual image component as well
• A scene generator for combining the virtual image component with at least a part of the real image into an augmented reality image

includes.

• – ein transparentes Display,
• – eine dem transparenten Display zugeordnete Kamera zur Aufnahme eines realen Bildes, das die Umgebung des Displays abbildet, die ein Betrachter bei Durchsicht durch das transparente Display sieht,
• – ein, insbesondere ein Trackingsystem umfassendes, System zur Bestimmung der Position eines virtuellen Bildbestandteils zu dem mittels der Kamera aufgenommenen realen Bild,
• – ein Bildbearbeitungsmodul zur Bestimmung der Farbe und/oder der Textur des realen Bildes an der Position des virtuellen Bildbestandteils und zur Auswahl der Farbe und/oder der Textur des virtuellen Bildbestandteils in Abhängigkeit der Farbe und/oder der Textur des realen Bildes an der Position des virtuellen Bildbestandteils sowie
• – einen Szenen-Generator zur Darstellung des virtuellen Bildbestandteils mittels des transparenten Displays

umfasst.The aforementioned object is - in particular in connection with the aforementioned features - also solved by a device for displaying a virtual image component, wherein the device

• - a transparent display,
• A camera associated with the transparent display for capturing a real image that depicts the environment of the display that a viewer sees when viewing through the transparent display,
• A system comprising, in particular, a tracking system for determining the position of a virtual image component relative to the real image recorded by means of the camera,
• An image processing module for determining the color and / or the texture of the real image at the position of the virtual image component and for selecting the color and / or the texture of the virtual image component as a function of the color and / or the texture of the real image at the position of the virtual image component as well
• - A scene generator for displaying the virtual image component by means of the transparent display

includes.

One Virtual image component in the sense of the invention (s) may be alphanumeric Include or consist of characters. A virtual image component For the purposes of the invention (s) may alphanumeric characters and a include colored background for these characters. One virtual image component within the meaning of the invention (s) may be based on alphanumeric Sign in conjunction with a color background for these signs exist. Is a color background provided, in particular its color depending on the color and / or the texture of the real image at the position of the virtual Image components selected.

motor vehicle in the context of the invention is in particular an individual on the road usable land vehicle. Motor vehicles in the context of the invention especially not on land vehicles with internal combustion engine limited.

Further Advantages and details will become apparent from the following description of exemplary embodiments. Showing:

1 an embodiment of a system for displaying an augmented reality image,

2 an embodiment of a real image,

3 an embodiment of an augmented reality image by superposition of the real image according to. 2 with a virtual image component,

4 an embodiment of an extended scene generator,

5 another embodiment of a real image,

6 an embodiment of an augmented reality image by superposition of the real image according to 5 with a virtual image component,

7 a further embodiment of an augmented reality image by superposition of the real image according to 5 with a virtual image component and

8th an embodiment of a method for producing a motor vehicle or a motor vehicle component.

1 shows a system for displaying an augmented reality image ARB. The system includes one by a user 1 worn headset 10 with a display 11 for displaying the augmented reality image ARB. In the headset 10 is one the display 11 associated camera for capturing a real image taken by the user 1 environment or part of this environment. The real image RB output from the camera is input to a tracking system 12 which outputs the real image RB and the orientation POS of the real image RB. The real image RB and the orientation POS of the real image RB are input to an extended scene generator 13 ,

The system for displaying an augmented reality image ARB also comprises a database 14 with virtual image components. From this database 14 becomes the advanced scene generator 13 a virtual image component VIRT supplied. The extended scene generator 13 selects or changes the color and / or texture of the virtual image component VIRT and combines this with the real image RB to an augmented reality image ARB, which by means of the display 11 is pictured.

2 shows an embodiment of a real image RB, wherein reference numerals 2 denotes a marker which serves to align the real image RB. In the present embodiment, a virtual image component of a production line is superimposed on the real image. That on the display 11 illustrated augmented reality image shows 3 ,

4 shows the extended scene generator 13 a more detailed presentation. The extended scene generator 13 includes a scene generator 131 as commonly provided in augmented reality systems. In addition, the extended scene generator includes 13 an image processing module 132 for determining the color and / or texture of the real image RB at the position of the virtual image component VIRT and for selecting the color and / or texture of the virtual image component VIRT depending on the color and / or texture of the real image RB at the position of the virtual image component VIRT. The image component, possibly altered in terms of its color and / or texture, is designated by reference symbol VIRTCOL and input to the scene generator 131 ,

Both the scene generator 131 , as well as the image processing module 132 include a module for determining the position of the virtual image component VIRT or VIRTCOL to the real image RB recorded by the camera as a function of the orientation POS of the real image RB. However, this module can also be swapped out and from both, that is the scene generator 131 or the image processing module 132 , be used. Together with the tracking system 12 This module forms an exemplary embodiment of a system for determining the position of the virtual image component VIRT or VIRTCOL to the real image RB recorded by means of the camera in the sense of the claims. However, the module itself also forms an exemplary embodiment of a system for determining the position of a virtual image component VIRT or VIRTCOL relative to the real image RB recorded by the camera in the sense of the claims.

The function of the image processing module 132 is with reference to 5 . 6 and 7 explained. It shows 5 an embodiment of a real image that a silver-colored, reflective shelf structure 23 with a partially yellow-colored linkage 25 shows. In the shelf structure are one with reference numerals 22 designated matte red box for the component 1J2 283 963 and one with reference numerals 26 designated matte blue box arranged. With reference number 21 and 24 is a white background called. Becomes a virtual image component 31 - as in 6 shown - the red box 22 as well as the yellow linkage 25 superimposed, it is displayed in white. Becomes the virtual image component 32 against it - as in 7 shown - the silver-colored shelf structure 23 and the white background 21 and 24 superimposed, it is displayed dull red.

8th shows an embodiment of a method for producing a motor vehicle or a motor vehicle component. It is in one step 41 a component - as exemplified in 6 and in 7 represented - by means of a virtual image component 31 respectively. 32 marked. Subsequently, this is done by means of the virtual image component 31 respectively. 32 marked component installed in the motor vehicle or the motor vehicle component. Alternative to the step 41 or in addition, a step 43 be provided, in which a mounting location by means of a virtual image component corresponding to the virtual image components 31 and 32 marked and a component in the step 42 is then installed at the marked as installation site.

It can be provided that the display 11 is designed as a transparent display. In this case, the augmented reality image ARB is not a superimposition of the real image RB with the virtual image component VIRTCOL adapted in its color and / or texture, but the virtual image component VIRTCOL, which is adapted in its color and / or texture, at a specific location is positioned so that it can be displayed at this point by means of the transparent display. The superimposition between reality and virtual image component takes place in the eye of the user 1 ,

1

user

2

marker

10

headset

11

display

12

tracking system

13

extended scene generator

14

database

21 24

white background

22

Red box

23

shelf structure

25

yellow linkage

26

Blueness box

31 32, VIRTCOL

In its color and / or texture adapted virtual image component

41 42, 43

step

131

scene generator

132

Image processing module

ARB

Augmented reality image

POS

alignment a real picture

RB

Reales image

VIRT

virtual Image component

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited patent literature

• EP 0949513 A2 [0002]
• WO 01/96829 A1 [0003, 0004]

Claims (11)

Method for generating an augmented reality image (ARB), wherein a real image (RB) is recorded by means of a camera, wherein the position of a virtual image component (VIRTCOL, 31 . 32 ) to the real image (RB) recorded by the camera, the color and / or the texture of the real image (RB) being determined at the position of the virtual image component (VIRTCOL, 31 . 32 ), wherein the color and / or texture of the virtual image component (VIRTCOL, 31 . 32 ) depending on the color and / or the texture of the real image (RB) at the position of the virtual image component (VIRTCOL, 31 . 32 ), and wherein the virtual image constituent (VIRTCOL, 31 . 32 ) is combined with at least part of the real image (RB) to form an Augmented Reality Image (ARB). Method for displaying a virtual image component (ARB, VIRTCOL) on a transparent display ( 11 ), whereby a real image (RB) is recorded by means of a camera, which surrounds the environment of the display ( 11 ), which a viewer sees when viewing through the display, determining the position of a virtual image component (VIRTCOL) to the real image (RB) captured by the camera, the color and / or texture of the real image (RB ) is determined at the position of the virtual image component (VIRTCOL), the color and / or the texture of the virtual image component (VIRTCOL) being dependent on the color and / or the texture of the real image (RB) at the position of the virtual image component (VIRTCOL ), and wherein the virtual image component (VIRTCOL) by means of the transparent display ( 11 ) is pictured. Method according to claim 1 or 2, characterized in that the virtual image constituent (VIRTCOL, 31 . 32 ) represents an information relating to the assembly of a component. Method according to claim 1, 2 or 3, characterized in that the virtual image constituent (VIRTCOL, 31 . 32 ) is a part number. Method for producing a technical component, in particular of a motor vehicle or a motor vehicle component, characterized in that a component to be mounted by means of the virtual image component (VIRTCOL, 31 . 32 ) is marked or marked according to a method according to one of the preceding claims, and that the component to be mounted is mounted in the technical component, in the motor vehicle or in the motor vehicle component. Method for producing a technical component, in particular a motor vehicle or a motor vehicle component, in particular a method according to claim 5, characterized in that the mounting location of a component to be mounted by means of the virtual image component (VIRTCOL, 31 . 32 ) is marked or marked according to a method according to one of claims 1 to 4, and that the technical component, the component to be mounted in the motor vehicle or in the motor vehicle component is mounted at the mounting location. Method for producing a technical component, in particular a motor vehicle or a motor vehicle component, wherein a real image (RB) of a component to be mounted and / or a mounting location of the component or component is recorded by means of a camera, wherein the position of a virtual image component (VIRTCOL, 31 . 32 ) to the real image (RB) recorded by the camera, the color and / or the texture of the real image (RB) being determined at the position of the virtual image component (VIRTCOL, 31 . 32 ), wherein the color and / or texture of the virtual image component (VIRTCOL, 31 . 32 ) depending on the color and / or the texture of the real image (RB) at the position of the virtual image component (VIRTCOL, 31 . 32 ), the virtual image component (VIRTCOL, 31 . 32 ) is combined with at least part of the real image (RB) to form an augmented reality image (ARB) or by means of a transparent display ( 11 ), and wherein the technical component, the component to be mounted is mounted in the motor vehicle or in the motor vehicle component. A method according to claim 7, characterized in that the virtual image component (VIRTCOL, 31 . 32 ) represents an information relating to the assembly of a component. Method according to claim 7 or 8, characterized in that the virtual image constituent (VIRTCOL, 31 . 32 ) is a part number. Device for generating an augmented reality image (ARB), wherein the device - a stationary or mobile display ( 11 ), - a display ( 11 ) associated camera for recording a real image (RB), - a, in particular a tracking system ( 12 ) system for determining the position of a virtual image component (VIRTCOL, 31 . 32 ) to the real image (RB) recorded by the camera, - an image processing module ( 132 ) for determining the color and / or the texture of the real image (RB) at the position of the virtual image component (VIRTCOL, 31 . 32 ) and to select the color and / or the texture of the virtual image component (VIRTCOL, 31 . 32 ) depending on the color and / or the texture of the real image (RB) at the position of the virtual image component (VIRTCOL, 31 . 32 ) as well as - a scene generator ( 131 ) to the combination of the virtual image component (VIRTCOL, 31 . 32 ) comprising at least a portion of the real image (RB) to an Augmented Reality Image (ARB). Device for displaying a virtual image component, wherein the device - a transparent display ( 11 ), - a transparent display ( 11 ) for capturing a real image (RB), which surrounds the environment of the display ( 11 ), which an observer sees through the transparent display ( 11 ), - a, in particular a tracking system ( 12 ) system for determining the position of a virtual image component (VIRTCOL, 31 . 32 ) to the real image recorded by the camera, - an image processing module ( 132 ) for determining the color and / or the texture of the real image (RB) at the position of the virtual image component (VIRTCOL, 31 . 32 ) and to select the color and / or the texture of the virtual image component (VIRTCOL, 31 . 32 ) depending on the color and / or the texture of the real image (RB) at the position of the virtual image component (VIRTCOL, 31 . 32 ) as well as - a scene generator ( 131 ) for displaying the virtual image component (VIRTCOL, 31 . 32 ) by means of the transparent display ( 11 ).