EP3271116B1 - Device and method for screwing - Google Patents

Description

The present invention relates to a device and a method for screwing objects, in particular in the manufacture and / or assembly of vehicles.

A high degree of automation is achieved today in the manufacture or assembly of motor vehicles. In particular when screwing vehicle parts such as parts of the body or the chassis, the engine or the drive train, a high degree of reliability is of great importance in complying with the torque prescribed at the respective screwing point. Such screw connections are often carried out manually by specialists. It is known to carry out and in particular monitor the screwing process according to the four-eyes principle. One person actuates the screwing tool, such as a torque wrench with the nut attached, and a second person observes and records the torque displayed on the tool. Such a procedure is evidently burdened with a high level of personnel and the associated costs.

Furthermore, a wrench is known which is capable of electrically recording and logging the torque. This allows logging of the screwing cases carried out in their order. However, it is not possible to assign the logged screwing cases to a specific screwing location on the vehicle. In order to be able to understand afterwards at which point on the vehicle or the object to be screwed the respective logged torques occurred, manual assignment, recording, acknowledgment or logging is also required here.

From medical technology, it is known, according to an in-house development by the applicant, to monitor interventions on the body with video, preferably using infrared images. In this system, ISO surfaces are spanned using MIP points, which are superimposed on one another. A registered video overlay of real images and "construction data" is required for the system. Markings on the tool make it easier to assign and understand the operations performed.

From the DE 10 2010 032 553 A1 shows a device for monitoring torque and connection conditions during the manufacturing process in the automotive industry. This device has a rotation sensor which has a projector for projecting a predetermined pattern and an image converter 66 or camera. This generates three-dimensional data so that it is possible to monitor the rotational distance covered by a mechanical fastener. A fastening device is based on this visual information and / or torque data and / or connection data are controlled accordingly.

The DE 10 2012 003 809 A1 relates to a system with which the torque required to open a plastic screw cap of a bottle is determined without contact. This is done by detecting appropriate marks using a camera.

In the EP 1 963 786 B1 describes a system for writing a position of an object. The objects are captured optically.

From the US 2008/0115589 A1 is a torque wrench that has a torque measuring device. This system also has a camera and a visible pointer. The visible pointer is, for example, a light signal that marks a specific connection that is to be tightened with the torque wrench. The tightened connection can be recorded with the camera and it can be used to automatically determine which connection is to be tightened next, so that the user is given an exact sequence. Information about the individual connections can also be saved.

From the US 2008/0061145 A1 goes a connection tool such. B. a screwdriver, with which connecting parts can be connected. A marking is assigned to the connecting parts. The tool has a reading device that can detect this marking. The marking can be a barcode, for example. This information can be compared with corresponding information from a database.

In the US 2014/0137669 A1 describes a torque determination system that determines the corresponding torque from predetermined angular positions. The angular positions can be recorded using an X-ray system.

In the DE 10 2013 017 007 A1 discloses a robot in which the force and torque input exerted by the robot can be determined. Another example is in WO 2013111387 disclosed.

It is an object of the present invention to provide a device and a method for screwing on or to objects, in particular in the manufacture or assembly of vehicles, which make it possible to reduce the torques applied in each case by assigning them to the respectively processed screwing locations with less Record personnel costs reliably and without changes to the object to be screwed.

The above object is solved by the features of independent claims 1 and 12. Advantageous further developments and preferred embodiments of the present invention form the subject of the subclaims.

The present invention is based on the basic idea of monitoring a work area with the object to be screwed and a tool unit which is designed for screwing screw connection elements by means of an image recording unit and analyzing the recorded images in order to at least identify the object and a screwing location at which one Bolting is carried out to recognize and automatically generate a log in which the recognized object and bolting location as well as a torque applied and recorded during the bolting or an applied angle of rotation are related to each other. Optionally, the data records of the protocol, which are referred to in the context of the present invention as a protocol tuple, can also be data which represent the tool unit also recognized by image analysis, and / or a time stamp, which is a time at which the torque or the angle of rotation is applied was, and / or have specifications generated on the basis of a work plan with regard to a screwing location on the object and / or a screwing case and / or a tool selection. The protocol tuples can also be used for monitoring the screwing process, for checking the screwing success and / or progress, for reporting back specifications and / or check results to a user and for guiding the user. Feedback can be provided by optical and / or acoustic signaling and / or image display.

The protocol tuples are preferably only saved when it has been established that the tool unit has been removed from the screwing point; and / or the recorded data with regard to the object and the screwing location and the applied torque or the executed rotation angle, preferably also with respect to the tool unit, correspond to specifications; and / or a completed screwdriving case is acknowledged by a user. A backup can take place, for example, but not only, as a printout, permanent storage in a non-volatile memory or data carrier, transfer to a remote administration point, etc. The torque recorded and contained in the protocol tuple, in particular in the secured protocol tuple, is preferably a maximum torque applied at the screwing location.

The object can also be recognized on the basis of the points which describe a specific body which is part of the object. The points of this body thus represent reference points. Different algorithms are known with which individual bodies are extracted from spatial images, that is to say images that include three-dimensional information about the objects detected, and are described with a data model.

The object is preferably determined or recognized by comparing the reference points with the corresponding points in a three-dimensional model of the object.

Such a model can be derived from design data. However, such a model of the object is preferably created in an empty learning process in which several spatial recordings are made with the image recording unit. The model is then generated from these spatial recordings by superimposing and comparing these recordings. Such a model derived from the spatial images or recordings describes the object in a manner that is very similar to the spatial images that are captured in later operation with the image recording unit.

The use of a stereo camera to generate such spatial images of the object and a comparison with a three-dimensional model of the object allows the stereo camera To aim at the object with different viewing directions and still achieve a reliable comparison with the model, whereby the object and its position in the coordinate system of the stereo camera can be reliably detected, even if the spatial images of the object are captured from different viewing directions.

Any screwing tool can be understood as a tool unit in the sense of the invention, which makes it possible to record, measure, display, recognize, etc. a torque applied during screwing. In particular, the tool unit can be designed as a so-called torque wrench, which is a lever unit with a tool holder has, with an interchangeable tool being received or receivable in or on the tool holder. The interchangeable tool can be a so-called nut or a so-called bit, with or without an extension, and is adapted or adaptable to the screw connection element to be screwed for a particular screwing case. Other examples of suitable tool units include, but are not limited to, a screwdriver or an impact wrench.

The torque can be displayed, for example, but not only, by means of an optical display such as a pointer, a scale, a clock or a digital display and / or by sensing and transmitting measurement data. In the latter case, the sensor data are transmitted to a data processing unit in a wireless or wired manner and evaluated there to record the torque, and in the former case the image supplied by the image recording unit is also evaluated to identify the torque indicated by the optical display. A corresponding sensor can be provided for sensing the torque. The rotation specification output device is preferably arranged in or on the lever unit

In preferred embodiments, the tool unit, in particular the lever unit and / or the exchangeable tool, carry an identification mark. The identification marking can, for example, but not only, have a bar code, a color code, a 2D barcode and / or a hole marking. The identification mark makes it possible to recognize the tool unit on the basis of the identification mark. The identification marking can be attached to an exchangeable marking unit, which can be attached or attached to the tool unit at a suitable location.

A carrier unit, which carries at least the image recording unit, can preferably be designed in the form of a spectacle frame, protective glasses, protective helmets, a headband or part of a piece of clothing such as a shoulder flap, a collar or a belt, or fastening elements for attachment to glasses, protective glasses , a hard hat, a headband, or on a piece of clothing such as a shoulder patch, collar, or belt. This allows a user to wear the image recording unit on the body. When the image recording unit is worn on the head, image acquisition advantageously follows the user's head movements.

The image recording unit is preferably designed to generate a spatial image. For this purpose, the image acquisition unit has in particular a first image acquisition unit and a second image acquisition unit, which are arranged spatially separated from one another.

It goes without saying that the method can be repeated for a large number of screwing locations or screwing cases.

The object is preferably recognized on the basis of predefined, stationary first detection points on the object, the first detection points being recognized by comparison with a previously stored model of the object, and the first detection points preferably being immanent to the object. The location of the screw connection is preferably identified on the basis of predefined, stationary, second detection points on the tool unit, the second detection points being identified by comparison with a previously stored model of the tool unit, the second detection points preferably being immanent to the tool unit. The model can be derived from design data such as CAD data or generated by prior training.

The image is a spatial image, which is recorded by two spatially separate image acquisition units (= stereo camera). A geometry reference defined by the relative spatial position of the two image acquisition units can be used to recognize the object.

• Fig. 1 ein Blockschaubild zur Veranschaulichung einer Vorrichtung zum Verschrauben von Gegenständen gemäß einem Ausführungsbeispiel der vorliegenden Erfindung;
• Fig. 2 eine Ansicht einer Werkzeugeinheit, die in dem Ausführungsbeispiel verwendet werden kann;
• Fig. 3 eine räumliche Ansicht einer auswechselbaren Markierungseinheit zum Anbringen einer Identifikationsmarkierung an der Werkzeugeinheit von Fig. 2;
• Fig. 4 eine räumliche Ansicht einer Bildaufnahmeeinheit mit einer Trägereinheit gemäß einem Ausführungsbeispiel der vorliegenden Erfindung;
• Fig. 5 eine räumliche Ansicht einer Bildaufnahmeeinheit mit einer Trägereinheit gemäß einem weiteren Ausführungsbeispiel der vorliegenden Erfindung;
• Fig. 6 eine Ansicht einer Bildaufnahmeeinheit mit einer Trägereinheit gemäß einem noch weiteren Ausführungsbeispiel der vorliegenden Erfindung;
• Fig. 7 eine schematische Darstellung einer Verschraubungssituation zur Veranschaulichung eines Verfahrens zum Verschrauben von Gegenständen gemäß einem Ausführungsbeispiel der vorliegenden Erfindung;
• Fig. 8 ein Prinzipschaubild, welches ausgewählte Punkte in der in Fig. 7 gezeigten Situation in einem Koordinatensystem schematisch darstellt;
• Fig. 9 ein Blockschaubild zur Veranschaulichung des Verfahrens zum Verschrauben von Gegenständen gemäß der vorliegenden Erfindung;.
• Figur 10 eine Markierungseinheit mit einem zweidimensionalen Barcode.

Further features, objects, advantages and effects of the present inventions will become apparent from the following description of specific exemplary embodiments. To illustrate the exemplary embodiments, reference is made to the accompanying drawing. Here is:

• Fig. 1 a block diagram illustrating an apparatus for screwing objects according to an embodiment of the present invention;
• Fig. 2 a view of a tool unit that can be used in the embodiment;
• Fig. 3 a spatial view of an interchangeable marking unit for attaching an identification mark to the tool unit of Fig. 2 ;
• Fig. 4 a spatial view of an image recording unit with a carrier unit according to an embodiment of the present invention;
• Fig. 5 a spatial view of an image recording unit with a carrier unit according to a further embodiment of the present invention;
• Fig. 6 a view of an image recording unit with a carrier unit according to yet another embodiment of the present invention;
• Fig. 7 is a schematic representation of a screwing situation to illustrate a method for screwing objects according to an embodiment of the present invention;
• Fig. 8 a principle diagram, which selected points in the in Fig. 7 schematically shows the situation shown in a coordinate system;
• Fig. 9 a block diagram illustrating the method for screwing objects according to the present invention.
• Figure 10 a marking unit with a two-dimensional barcode.

The present invention will now be described in detail by means of preferred embodiments with reference to the accompanying drawings. It should be understood that the pictorial representations are purely schematic and are not necessarily to scale. It should also be pointed out that the drawings and the following description concentrate on the features helpful for understanding the invention, without the The scope of the present invention, which is defined solely by the appended claims, is intended to be limited.

Fig. 1 is a block diagram which schematically illustrates a device for screwing objects in a screwing case. To carry out a screw connection on an object 100, a device is used which has several elements, namely a tool unit 200, an image recording unit 300, a data processing unit 400 and an image output unit 500.

As shown in Fig. 1 the object 100 has a plurality of reference points 110 and a screwing location 120. The reference points 110 are unchangeable recognition points on the object 100, which are predetermined by the construction of the object 100. The reference points 110 can also be the points that describe a specific body of the object. Those points which describe an elongated body are preferably also used as reference points. A data model which is stored in a model data storage unit 440 and which enables the recognition and spatial assignment of the reference points 110 on the object 100 from a spatial image is described in more detail below. This model can be derived from design data and / or generated during a learning process in which several spatial recordings are made with the image recording unit 300. The model is preferably generated from the spatial images acquired with the image acquisition unit 300, since such a model is very similar to the spatial images acquired in later operation. The multiple spatial images are superimposed and compared to create the model.

The screw connection point 120 is specified by a work plan and linked to the screw connection case. The screwing location 120, which in Fig. 1 is shown, is preferably only one of many screwing locations that are to be processed according to the work schedule. Furthermore, more than three reference points can be defined on the object 100, but for spatial orientation, at least three reference points 110 are required, which are preferably not on a straight line.

The screwing location 120 can be hidden behind a part of the object 100 both in the model and in spatial images acquired with the image recording unit 300. For automatic recognition of the object 100, it is sufficient if the reference points are visible in the captured image. If the object 100 is identified and its arrangement in space or in the coordinate system of the image recording unit 300 is known, then such hidden screwing locations 120 can also be identified.

The screw locations 120 and the reference points 110 can be entered during the learning process and integrated into the model to be created. The work schedule for learning the screw locations 120 can be processed here.

In the present exemplary embodiment, a screw 150 is provided for screwing onto the object 100. As shown in Fig. 1 a tool unit 200 is used for the screw connection, which has a torque wrench and a nut 250 adapted to the screw or the screw head of the screw 150. The torque wrench has a lever arm 210, a pin 220 for coupling with the nut 250 and a torque indicator 230. A marking unit 240 with an identification marking is attached to the lever arm 210. The identification marking is designed in such a way that it uniquely identifies the torque wrench 200 from other tools and will be described in more detail later using a specific example of the marking unit 240. The torque wrench is understood as a tool unit 200 in the sense of the present invention, with the lever arm 210 being understood as a lever unit, the torque indicator 230 as a rotation specification output device, the pin 220 as a tool holder and the nut 250 as a replaceable tool. The pin 220 has a square cross section in a manner known per se and customary, which mates with a square recess in the nut 250, and can have a spring element (not shown in more detail) on one side, which has a lateral inner wall of the recess in FIG Nut 250 cooperates in order to couple the nut 250 to the pin 220 in a captive but detachable manner. The pin 220 can also be mounted in a ratchet unit (not shown in detail), which enables quasi-continuous screwing without repeated setting down and reattaching of the tool.

In the present exemplary embodiment, the torque display 230 has an optical display 232 and a radio transmitter 234, which enables both a visual reading of the applied torque and a transmission of the torque in the form of measured value data or signals to an appropriately configured radio receiver. Alternatively, instead of a radio transmitter, a cable interface or an attached cable can also be provided in order to transmit measured value data or signals in a wired manner. It should be pointed out that the present invention also works with only one of the optical display and the radio transmitter or the cable interface / the transmission cable. In the context of the present invention, it is essentially a matter of outputting or receiving the torque that is applied with the tool unit during screwing in any form.

Instead of the torque, the angle of rotation of a screw connection can also be recorded. The angle of rotation is detected by detecting the tool unit 200 in its different rotational positions by means of the image acquisition unit 300 and corresponding evaluation by the data processing unit 400. There are assembly instructions which apply to screw connections Prescribe a certain torque or a certain angle of rotation or a combination of both. For example, there are assembly instructions that first prescribe a certain torque for screw connections. When this torque is reached, the screw connection must be screwed on by a predetermined angle (eg 45 ° or 90 °). The rotation specification output device is therefore designed such that it can specify both a torque and an angle of rotation.

As shown in Fig. 1 The image acquisition unit 300 has as essential components a camera unit 310, a control unit 320 and a radio interface 305. The camera unit 310 is designed and aligned to record at least a section of the object 100 with the reference points 110 and the screwing location 120, and supplies image data of the recorded images to the control unit 320. The control unit 320 is configured to receive image data from the camera unit 310 and can be configured to send control data to the camera unit 310 in order to set recording parameters such as, for example, a focus, an aperture, a depth of field or an image section. The control unit 320 is also coupled to the radio interface 305 and supplies the latter with the image data received from the camera unit 310 such that the radio interface 305 can provide the recorded image data to a remote station.

In the present exemplary embodiment, the image recording unit 300 also has a laser pointer 330 as a laser pointing device, a loudspeaker unit 340 as an acoustic output unit and a light-emitting diode unit 350 as an optical output unit. The laser pointer 330, the loudspeaker unit 340 and the light-emitting diode unit 350 are technically coupled to the control unit 320, the control unit 320 being designed to output control data to the laser pointer 330, the loudspeaker unit 340 and the light-emitting diode unit 350.

As shown in Fig. 1 the data processing unit 400 has a radio interface 405, a control unit 410, an image data acquisition unit 420, a measurement data acquisition unit 430, a model data storage unit 440, an image analysis unit 450, a timer unit 460, a logging unit 470, a work plan storage unit 480, a specification unit 485 and a feedback generation unit 495 ,

As in Fig. 1 is shown, the radio interface 405 is designed to establish and maintain a wireless data or signal transmission connection with the rotation specification output device 230 of the tool unit 200 and the radio interface 305 of the image recording unit 300. The control unit 410 is coupled to the radio interface 405 and is designed to deliver image data received from the image acquisition unit 300 via the radio interface 405 to the image data acquisition unit 420 and to supply torque data received from the rotation specification output device 230 via the radio interface 405 to the measurement data acquisition unit 430 and to provide feedback data from of the feedback generation unit 495. The image data acquisition unit 420 is designed to receive image data from the control unit 410 and, if appropriate, to store them temporarily and / or permanently and to make them available to the image analysis unit 450. The model data storage unit 440 is designed to store model data, which design and geometric characteristics of the object 100 with its unchangeable reference points (110, Fig.1 ) to store the tool unit 200 or its identification features and to make it available to the image analysis unit 450. The image analysis unit 450 is designed to automatically recognize the object 100, the tool unit 200 and the screwing location 120 on the object 100 by comparing the image data supplied by the image data acquisition unit 420 with the model data supplied by the model data storage unit 440 and data indicating the recognized screwing location and preferably reflect the recognized tool unit to be provided to the protocol generation unit 470.

The measurement data acquisition unit 430 is designed to acquire torque data and rotation angles received from the control unit 410, to store them temporarily and / or permanently, if necessary, and to make them available to the protocol generation unit 470.

The protocol generation unit 470 is designed to generate respective protocol tuples p from the data provided by the image analysis unit 450 with regard to the screwing location and preferably the recognized tool unit, the torque data / rotation angle data provided by the measurement data acquisition unit 430 and a time stamp provided by the timer unit 460, a log memory 275 for backup to be supplied and made available to the checking unit 490. The protocol tuples generated by the protocol generation unit 470 may also include default data provided by the default generation unit 485. The specification generation unit 485 is designed to generate specification data on the basis of a work plan taken from the work plan storage unit 480 and to make it available to the protocol generation unit 470 and the checking unit 490. For synchronization with the actual workflow, the default generation unit 485 can also be coupled to the timer unit 460 and the image analysis unit 450. For example, the specification generation unit 485 can generate specification data based on a work cycle based on a time supplied by the timer unit 460 or can supply specification data flexibly adapted to an individually selected work sequence using a screwing location supplied by the image analysis unit 450.

The checking unit 490 is designed to check the (not yet secured) protocol tuples p to determine whether or not the specifications provided by the default generating unit 485 have been reached and to provide a corresponding checking result to the feedback generating unit 495. The backup of the protocol tuple p in the log memory 475 can be linked to the condition that the specifications provided by the specification generation unit 485 have been achieved at least with regard to the screwing location and the detected torque or rotation angle. The protocol tuple p can also be saved by manual acknowledgment triggered by a user. Furthermore, a protocol tuple p can be saved automatically when the screwing process is terminated, for example when the tool is removed from the screw 150.

The feedback generation unit 495 is designed to receive protocol tuples provided by the protocol generation unit 470, and to receive default data provided by the default generation unit 485 and check results provided by the checking unit 490, to generate a feedback therefrom and to provide this to the control unit 410. The control unit 410 is also designed to provide feedback received from the feedback generation unit 495 via the radio interface 405 of the image recording unit 300 and an image output unit 500 to be described below.

The control unit 320 of the image recording unit 300 can be designed to convert feedback received from the data processing unit 400 via the radio interface 305 into control signals to the laser pointer 330, the loudspeaker unit 340, the light-emitting diode unit 350 and / or the camera unit 310. Then, for example, the feedback can be output in acoustic form via the loudspeaker unit 340, an optical confirmation or warning signal can be output via the light-emitting diode unit 350 and / or the camera unit 310 can change image recording parameters. If the feedback generated by the feedback generation unit 495 contains default data, the laser pointer unit 330 can also generate a laser beam directed at the screwing location 120 to be processed, in order to guide a person to the corresponding screwing location 120.

The above-mentioned image output unit 500 is provided in the present exemplary embodiment as a tablet PC which has a radio interface 510, a screen unit 520 and a loudspeaker unit 530. On the basis of the feedback received via the radio interface 510, as shown here, default values for the screw 150 to be selected and the torque (tightening torque) prescribed for this can be output.

It is understood that the principle described above enables many variations and options. For example, a green, a red and possibly a yellow light-emitting diode of the light-emitting diode unit 350 of the image recording unit 300 can be controlled in order to convey a work result to a user in a color-coded manner. For example, the correct positioning of the tool unit 200 can be indicated by a yellow illumination and the correct tightening torque reached by a green illumination. Flickering with different frequencies in yellow color can, for example, signal an approximation to the prescribed tightening torque, flickering in red color indicates that the tightening torque is too high. In addition, the work success can be displayed on the screen 520 of the image output unit 500 by means of a text message or a colored background.

An embodiment of the torque wrench (tool unit) 200 is shown in FIG Fig. 2 and the marking unit 240 for use on the tool unit 200 is shown in FIG Fig. 3 shown. It is Fig. 2 a plan view of the tool unit 200 (torque wrench), and Fig. 3 is a perspective view of the marking unit 240 in the form of a cuff.

As in Fig. 2 shown, the torque display 230 is in the form of a digital display. The digital display of the torque display 230 is provided by the camera unit 310 of the image recording unit 300 ( Fig. 1 ) good to see. The marking unit 240 is attached to the lever unit 210 near the tool holder 220. As an alternative to a digital display, the applied torque can also be achieved by means of a pointer instrument or a cantilever that does not deform with the lever unit 210, possibly in conjunction with a scale attached to the lever arm 210, which moves or shifts along the cantilever when the lever arm 210 is deformed , or any other suitable visual representation.

As in Fig. 3 is shown, the marking unit 240 is designed in the form of a double-shell sleeve with an upper part 242 and a lower part 243, which are connected by a series of screws 244 in order to clamp the marking unit 240 to the lever arm 210.

The marking unit 240 also carries an identification marking in the form of a bar code 246 on the one hand and a hole marking or hole coding 248 on the other hand. Both types of marking can be easily read by the camera unit 310 of the image recording unit 300.

Fig. 10 shows an alternative example of an approximately rectangular marking unit 240 with a two-dimensional barcode 247 for coding an identification number for the respective tool and at the corners with a position mark 248, by means of which the position of the tool in space or in the coordinate system of the image recording unit can be detected. The two-dimensional code is mirrored around a center line. The position of the tool in the three-dimensional coordinate system of the camera can easily be determined using the position marks 248.

This marking unit 240 can be printed out with a conventional printer. The marking unit 240 is then glued onto the tool unit 200, in particular the lever arm 210.

After the marking unit 240 has been fastened to the tool unit 200, it is scanned in a learning process, regardless of whether the fastening unit is a sleeve or a glued-on label, so that the position of the marking unit 240 on the tool unit 200 is clearly detected. In later operation, the position of the tool unit 200 in the three-dimensional coordinate system of the image recording unit 300 can then be unambiguously determined based on the detection of the marking unit 240.

A carrier unit can be provided for conveniently carrying the image recording unit 300 by a user, on which the image recording unit 300 can be carried in a compact or distributed manner on the body.

Some specific exemplary embodiments for the image recording unit 300 with carrier unit 600 are shown in FIG FIGS. 4 to 6 shown schematically.

As shown in Fig. 4 The carrier unit 600 is designed in the form of protective glasses in accordance with a specific exemplary embodiment. These protective glasses have a mask unit 622, which encloses protective glasses 624, and an elastic band 626 which lies behind the head, the band 626 being adjustable by a closure unit 628. In this exemplary embodiment, the image recording unit 300 is attached laterally at a transition between the mask unit 622 and the elastic band 626.

As shown in Fig. 5 According to a further specific exemplary embodiment, the carrier unit 600 is designed in the form of safety glasses with a frame 642 which accommodates two protective glasses 644 and two brackets 646, 646, the image recording unit 300 being arranged distributed on the carrier unit 600. A camera unit 310 is attached to the right and left bracket 646, 646, respectively, is a loudspeaker unit 340, the housing of which also contains the control unit 320 and the radio interface 305 ( Fig. 1 ), is attached to the left bracket 646, and is a light-emitting diode unit 350 with three differently colored light-emitting diodes, which are attached to a nose root part of the frame 642. Wiring 648 between camera units 310, light-emitting diode unit 350 and loudspeaker unit 340 with radio interface 305 and control unit 320 is integrated at the upper edge of frame 642.

As shown in Fig. 6 a carrier unit 600 is designed according to a still further exemplary embodiment of the present invention as a protective helmet 660, on the sides of which a camera unit 310 is attached in each case. The camera units 310 can be fixed, they can be attached to a holder integrated on the helmet 660, or they can be integrated with a holder that can be attached to the helmet 660. The further device technology of the image recording unit 300 can be distributed in one of the camera units 310, distributed over both camera units 310 or integrated in the protective helmet 660.

Fig. 7 is a spatial schematic representation of an application situation in the sense of the present invention. A user 700 uses the torque wrench 200 to screw on a screw connection 120 on an object 100, here a motor vehicle. The user 700 wears an image recording unit 300 by means of a carrier unit 600 in the form of a helmet (cf. Fig. 6 ). Three reference points 110, which are structurally predetermined and unchangeable per se, are predetermined on the object 100. On the tool unit 200 (torque wrench) are three reference points 710 constructive and / or by marking unit (240, Fig. 3 ) specified or identifiable.

As in Fig. 7 can be seen, the screw connection 120 is hidden behind construction elements of the object 100 in the illustrated application. Nevertheless, it is possible to determine from the reference points 110 of the object 100 and the reference points 710 of the tool unit 200 at which point on the object 100 the tool unit 200 starts.

This process is explained below using a principle diagram, which is shown in Fig. 8 is shown, are described in more detail. In Fig. 8 The reference points 110 of the object 100 and the reference points 710 of the tool unit 200 are shown in a virtual space 800. As in Fig. 8 As shown, the reference points 110 of the object 100 span a first reference triangle 810, which defines a first reference plane 815. The reference points 710 of the tool unit 200 also span a second reference triangle 820, which defines a second reference plane 825. The reference points 110, 710 are captured by the camera units 310 of the image recording unit 300. Since the camera units 310 are spatially spaced from one another, the obtained image data can be used to assign the reference points 110, 710 to coordinates X1, X2, X3 of the virtual space 800, so that the absolute position of the object 100 and the tool unit 820 in virtual space 800 as well the relative position between the object 100 and the tool unit 200 are fully defined. It goes without saying that the mathematical processes required for this, including the generation of a mathematical model of the object 100 with its reference points 110, a mathematical model of the tool unit 200 with its reference points 710, the generation of a spatial image from two stereoscopic images and their assignment to coordinates in A virtual space is just as well known in technology as the techniques of image analysis, which can be used to compare supplied image data with the previously generated and saved model and therefore do not need to be described here, but instead refer to the relevant prior art can be.

The Figures 5 and 6 each show a carrier unit 600 with two camera units 310, so that these camera units 310 each form a stereo camera. In Figure 4 the image acquisition unit 300 is only shown schematically. This is also a stereo camera. The use of a stereo camera compared to other known image recording units for capturing three-dimensional structures offers the advantage that the three-dimensional images can be generated very quickly. In a prototype of the invention, the image recording unit 300 generates two spatial images per second. It is expedient that at least one spatial image is generated at least every two seconds, preferably at least one, two or at least three images are generated per second. On the other hand, the resolution of a stereo camera is sufficient to be able to clearly indicate the screwing locations 120 to the operator.

The use of the stereo camera thus allows the carrier unit 600 arranged on a body of an operator to provide the image recording unit 300, the operator being able to move freely and being able to assume any position in relation to the object 100 as long as the viewing direction of the image recording unit 300 is towards the object 100 is directed. By comparing the acquired spatial image (s) with the spatial model of the object stored in the model data storage unit 440, the spatial image data can quickly be brought into agreement with the spatial model. As a result, the relative positions of the image recording unit 300 with respect to the object 100 are uniquely identified, so that the three-dimensional coordinate system of the image recording unit 300 is fixed and the position of the object 100 is determined in this coordinate system. Pointer elements, such as the laser pointer 330, are preferably mechanically connected to the image recording unit 300, so that their position in the coordinate system of the image recording unit 300 is also clearly defined.

An operator who carries the carrier unit 600 can thus move freely with respect to the object 100, the screwing locations 120 being displayed or specified automatically on the one hand and the screwing completely (torque and / or angle of rotation; time stamp) being logged on the other hand.

Fig. 9 is a block diagram illustrating process flows in the implementation of an inventive method for screwing objects according to a further embodiment of the present invention.

In Fig. 9 An area 910 symbolizes a physical production level, an area 920 symbolizes a data processing level and an area 930 symbolizes a tool data level. A first interface level 921 creates a link between the physical production level 910 and the data processing level 920, and a second interface level 923 creates a link between the tool level 930 and the data processing level 920. The physical level 910 can essentially with the object 100 in production image acquisition unit 300 and the image output unit 500 in Fig. 1 The data processing level 920 can essentially be compared with the data processing unit 400 of FIG Fig. 1 are equated, and the tool plane 930 can essentially be compared with the tool unit 200. The interface level 921 can with the radio interfaces 305, 405, 510 and associated radio transmission links in Fig. 1 can be compared, and the interface level 923 can essentially be compared with a radio transmitter integrated in the turning specification output device 230 of the tool unit 200 and the radio interface 405 of the data processing unit 400 in Fig. 1 be compared.

An arrow 912 in Fig. 9 symbolizes a production process, for example on a vehicle assembly line during vehicle production. As soon as a vehicle can be picked up at an assembly location, a NOK message with vehicle ID is issued in step 914. Bypassing the data processing level 920, the assembly of the vehicle can of course also be carried out by manual detour (generally represented as step 915) with manual OK setting 916. According to the invention, the assembly, in particular the setting of screw connections on the vehicle, takes into account the data processing level 920. In this case, data from the physical production level 910 is introduced into the data processing level 920 via the interface level 921 and data provided in the data processing level 920 via the interface level 921 in the physical production level 910 can be represented. For this purpose, a query 917 takes place, in the form of a constantly repeating loop, until an OK message 918 is issued by the data processing level 920. After manual OK setting 916 or automatic OK message 918, the vehicle can be returned to the production process 912 from the assembly point.

Several processing blocks are defined in the data processing level 920. These are, in particular, a start block 940, a data load block 950, a recognition block 960, a navigation block 970 and a monitoring block 980. At the tool level 930, essentially a measurement data generation block 990 is to be observed, in which, on request of the navigation block 970 of the data processing level 920, measurement data on the tool ( Tool unit 200, Fig. 1 ) are generated and made available to the monitoring block 980 of the data processing level 920. The request and provision of data between the data processing level 920 and the tool level 930 takes place via the interface level 923.

After implementation of the start block 940, the data loading block 950 is implemented on the data processing level 920, in which data about the vehicle, the tightening situation and the tool are loaded. Then the recognition block 960 is implemented, in which a recognition is first started in a step 962, then a recognition is carried out in a step 964 and after recognition in a step 966 the recognized situation is registered. The start of the recognition in step 962 can be carried out with an activation of the image recognition unit 300 in Fig. 1 , in particular the camera units 310. The execution of the recognition in step 964 corresponds, for example, to the recognition of the vehicle (object 100), the screwing point 120 and the tool unit 200 by image analysis in the image analysis unit 450 on the basis of the image data supplied by the camera unit 310 and captured by the image data acquisition unit 420 in comparison with the data in FIG the model storage unit 440 stored model data, in particular on the basis of the predetermined and unchangeable reference points 110 and the marking unit 240 ( Fig. 1 ) and reference points 710 identifiable on the tool unit 200 ( Fig. 7 ). The registration of the recognition in step 966 corresponds to the generation of a data record with data relating to the recognized vehicle (the recognized object 100), the recognized tool unit 200 and the recognized screwing location 120 ( Fig. 1 ) play. The generated or registered data record is then transferred to navigation block 970.

In the navigation block 970, the fitter navigates to the screwing point in step 972, for example via the laser display device 330 in Fig. 1 , the output of default data via the image output unit 500 and optical and / or acoustic signaling via the loudspeaker unit 340 or the light-emitting diode unit 350 in Fig. 1 can be done. As soon as it has been recognized that the tool (tool unit) 200 with the required screw 150 is attached to the intended screwing point 120, a detection message is sent to the monitoring block 980 in step 974. At the same time, the screwing process is started in step 976, which is a request of measurement data from the tool unit 200 ( Fig. 1 ) includes.

On the basis of the request from the navigation block 970, the processing block 990 generates measurement data on the tool level 930 which corresponds to the torque applied to the tool unit 200 ( Fig. 1 ) correspond and also transmits them to the monitoring block.

In step 982, the monitoring block 980 monitors the screwing process on the basis of the measurement data supplied by the tool level 930 (processing block 990) and ends the monitoring in step 984 when the target is reached. Protocols are then generated in step 986, protocol tuples being generated which generate and store the data record registered in step 966 together with the data which represent the detected torque. The protocol can be transferred in the form of log files and / or video protocols 925 to the interface level 921. Likewise, the monitoring block 980 delivers an OK message 927 to the interface level 921 if the vehicle is still active, and the OK message 927 is converted into the OK message 980 described above via the interface level 921.

It should be noted that the process described above is subject to a predetermined schedule and within the scope of this schedule using a work plan (cf. 480, Fig. 1 ) Bolting cases are processed, whereby a bolting case ID is generated for each bolting case. The tightening case ID and the vehicle ID mentioned above are part of the protocol tuple of the generated protocol.

In the example explained above, the object 100 is a motor vehicle. Other objects such as e.g. Aircraft, medical devices or the like are used.

In the exemplary embodiments described above, an operator or fitter carries the carrier unit 600 on which the camera units 310 are arranged. Within the scope of the invention it is also possible to provide an automatic screwing machine in which the tool unit 200 is automatically moved and actuated to the respective screwing location 120. A robot arm can be provided for this purpose, for example.

The automatic screwing device and the object to be screwed can be moved relative to one another, with at least one spatial image of the object being recorded in the different positions. The spatial resolution of the spatial images captured with the image acquisition unit 300 cannot be sufficient to control the tool unit 200 sufficiently precisely to the screwing location 120 on the basis of a single spatial image of the object 100 alone. If this is the case, further spatial images can be recorded for positioning the tool unit 200 during the positioning process, the tool unit 200 and the object 100 being recorded therein, so that the relative position of the tool unit 200 to the object 100 is determined on the basis of these images. This relative position is determined with a significantly higher spatial resolution than the absolute position of the object 100 in the three-dimensional coordinate system of the camera. With a closed control loop, the tool unit 200 can thus be positioned very precisely and fully automatically at the screwing location 120.

Such a closed-loop control for positioning the tool unit 200 can also be used in a "mobile" system with a portable carrier unit 600, in which case the fitter is given visual, acoustic or other information as to how the tool unit 200 is relative to the object 100 has to position.

It is to be understood that the above description can only include exemplary embodiments of the present invention and the invention is not restricted to the specific embodiments described, but rather is defined in its broadest understanding solely by the appended claims.

Claims (15)

1. A device for screwing objects, comprising:

   - a tool unit (200) designed for screwing screw connection elements (150),

   - a rotation default output means (230) designed to output the torque to be applied when screwing with the tool unit (200) and/or the angle of rotation to be achieved when screwing, and

   - a data processing unit (400) designed to detect torque data or rotational angle data representing the torque applied during screwing or the rotational angle achieved during screwing,

   - an image capturing unit (300) comprising a stereo camera for capturing a spatial image of a working area having at least a portion of an object (100) to be screwed together and providing the captured image data, and

   - the data processing unit (400) comprises an image analysis unit (450) designed to recognize the object (100), the tool unit (200) and a screwing location on the object (100) in the captured image and to generate a protocol tuple (p) including data representing the recognized screwing location, the detected torque data and/or rotational angle data associated with each other, and

   - characterized in that the device comprises a carrier unit (600) carrying at least the image capturing unit (300), the carrier unit being designed to be carried on the body of a user.
2. The device according to claim 1, characterized in that the data processing unit is designed such that the recognition of the object (100) takes place on the basis of at least three reference points (110) of the detected spatial image, wherein the recognition of the object (100) takes place preferably on the basis of a body which is a component of the object and is represented in the spatial image by a plurality of reference points.
3. The device according to claim 1 or 2, characterized in that the protocol tuple (p) further comprises data representing the recognized tool unit (200) and/or a time stamp representing a time at which the torque was applied or the angle of rotation was achieved.
4. The device according to any of claims 1 to 3, characterized in that the tool unit (200) has a lever unit (210) including a tool mount (220) and comprises an exchangeable tool (250) which is received or receivable in or on the tool mount (220), the tool (250) being adapted or adaptable to the screw connection element (150) to be screwed, wherein the rotation default output means (230) is preferably arranged in or on the lever unit (210).
5. The device according to any of the preceding claims, characterized in that the rotation default output means (230) comprises at least one of the following:

   - an optical display (232) such as a pointer, scale, clock or digital display, the image analysis unit (450) of the data processing unit (400) being designed to recognize the torque indicated by the optical display (232) or the rotational angle from the image supplied by the image capturing unit (300);

   - a data transmission unit (234) designed to transmit measurement data representing a sensed torque, wherein a receiving unit (405) of the data processing unit (400) is designed to receive the measurement data supplied by the data transmission unit (234), and wherein a measurement data detection unit (430) of the data processing unit (400) is designed to interpret and detect the received measurement data as the torque data.
6. The device according to any of the preceding claims, characterized in that the tool unit (200), in particular the lever unit (210) and/or the exchangeable tool (250), are provided with an identification mark (246, 248), in particular a bar code, a color code, a 2D bar code and/or a perforation mark, wherein the data processing unit (400) is designed to recognize the tool unit (200) by means of the identification mark (246, 248), the tool unit (200) preferably comprising an exchangeable marking unit (240) which is provided with the identification mark (246, 248).
7. The device according to any of the preceding claims, characterized in that the carrier unit (600) is realized in the form of a spectacle frame, protective goggles, a protective helmet, headband or part of a piece of clothing such as a shoulder flap, collar or belt, or comprises attachment members for attachment to spectacles, protective goggles, a protective helmet, headband or piece of clothing such as to a shoulder flap, collar or belt.
8. The device according to any of the preceding claims, characterized in that the data processing unit (400) comprises a default generation unit (485) which is designed to generate defaults with respect to a screwing location on the object (100) and/or a screwing case and/or a tool selection, preferably based on a previously stored workflow plan, wherein the protocol generation unit (470) is preferably designed to add defaults generated by the default generation unit (485) to the respective protocol tuple (p), and wherein the data processing unit (400) preferably comprises a verification unit (485) configured to verify the achievement of the defaults and generate a verification result; wherein preferably by an acoustic output unit (340) coupled to the data processing unit (400), wherein the data processing unit (400) is designed to transmit the defaults and/or the verification results to the acoustic output unit (340), and the acoustic output unit is designed to output acoustic indication, warning and/or confirmation signals corresponding to the defaults and/or verification results.
9. The device according to claim 7 or 8, further characterized by an optical output unit (330, 350) coupled to the data processing unit (400), wherein the data processing unit (400) is designed to transmit the defaults and/or the verification results to the optical output unit (330, 350), and the optical output unit (330, 350) is designed to output optical indication, marking, warning and/or confirmation signals corresponding to the defaults and/or verification results, wherein the optical output unit (330, 350) preferably comprises one or more light emitting diodes (350) of one or different colors, a laser pointing device (330) and/or a hologram generating means.
10. The device according to any of claims 7 to 9, further characterized by an image output unit (500) coupled to the data processing unit (400), wherein the data processing unit (400) is designed to transmit the defaults and/or the verification results to the image output unit (500), wherein the image output unit (500) is designed to output indications, instructions, warnings or confirmations corresponding to the defaults and/or verification results.
11. The device according to any of the preceding claims, further characterized by a protocol backup unit (475) designed to backup the protocol tuples (p) generated by the protocol generation unit (470), wherein the protocol generation unit (470) is designed to hand over a protocol tuple (p) to the protocol backup unit (475) preferably whenever at least one of the following conditions is satisfied:

    - the tool unit (200) is removed from the screwing location (120);

    - the data detected with respect to the object (100) and the screwing location and the torque applied, preferably also with respect to the tool unit (200), correspond to defaults;

    - a processed screwing case is acknowledged by a user.
12. A method for screwing objects, comprising the steps:

    - capturing a spatial image of a working area with a stereo camera, wherein the captured image comprises at least a portion of an object (100) to be screwed;

    - recognizing the object (100) on the basis of at least three reference points (110) of the captured spatial image, and recognizing a screwing location on the object (100) from the one or more captured images;

    - automatic detection of a torque applied during screwing by the tool unit (200) and/or an achieved angle of rotation;

    - generating a protocol tuple of data representing the screwing location and the torque and/or the angle of rotation associated with each other, wherein

    characterized in that
    a carrier unit (600) carrying at least the image capturing unit (300) is carried on the body of a user.
13. The method according to claim 12, characterized in that the recognition of the object (100) is carried out on the basis of reference points which describe a body that is part of the object, and/or the recognition of the object is carried out by comparing the reference points with the corresponding points in a three-dimensional model of the object (100), preferably wherein the following steps are carried out:

    - recognizing a tool unit (200) for screwing the object (100),

    - creating data representing the recognized tool unit (200), and

    - adding the data to the respective protocol tuple (p).
14. The method according to claim 12 or 13, further characterized by the steps:

    - detecting a time at which the torque and/or the angle of rotation was applied,

    - creating a time stamp that represents the detected time, and

    - adding the time stamp to the respective protocol tuple (p).
15. The method according to any of claims 12 to 14, characterized in that the detection of the torque comprises at least one of the steps:

    - evaluating the image and recognizing a torque indicator (230) at the tool unit (200) in the image;

    - evaluating sensor data which are generated by the tool unit (200) and preferably transmitted to a data processing unit (400) in a wireless or wired manner.

Images