WO2016146705A1

WO2016146705A1 - Device and method for screwing objects

Info

Publication number: WO2016146705A1
Application number: PCT/EP2016/055721
Authority: WO
Inventors: Armin Baumgartner
Original assignee: Nexustec Gmbh
Priority date: 2015-03-17
Filing date: 2016-03-16
Publication date: 2016-09-22
Also published as: DE102015103941A1; EP3271116B1; EP3271116A1

Abstract

The device according to the invention for screwing objects has a tool unit (200), a rotational specification output device (230), and a data processing unit (400) which is designed to detect torque data which describes the torque applied during the screwing process. Furthermore, an image capturing unit (300) is provided which is designed to capture an image of a working region with at least one section of an object (100) to be screwed and to provide the captured image data, and the data processing unit (400) has an image analysis unit (450) which is designed to detect the object (100), the tool unit (200), a screw location, and a corresponding rotational angle on the object (100) in the captured image and to generate a protocol tuple (p) with data that describes the detected screwing location and includes the detected torque data in a mutually assigned manner. A method according to the invention for screwing objects has the following steps: - capturing a spatial image of a working region using a stereo camera,the detected image comprising at least one section of an object (100) to be screwed; detecting the object (100) and a screwing location on the object (100) from the one or more captured images; - automatically detecting a torque applied during the screwing process by means of the tool unit (200); and generating a protocol tuple of data which describes the screwing location and the torque in a mutually assigned manner.

Description

Device and method for screwing objects

The present invention relates to a device and a method for screwing objects, in particular in the manufacture and / or assembly of vehicles.

In the manufacture or assembly of motor vehicles, a high degree of automation is achieved today. In particular, when bolting vehicle parts such as parts of the body or the chassis, the engine or the drive train high reliability in the maintenance of the prescribed torque at the respective screw is of great importance. Such glands are often performed in manual labor by skilled workers. It is known to carry out the implementation and in particular monitoring of the screwing process on the four-eye principle. In this case, a person operates the screwing tool such as a torque wrench with nut attached, and a second person observes and logs the torque displayed on the tool. Such a procedure is obviously burdened with a high workforce and the associated costs.

Furthermore, a wrench is known, which is capable of electrical detection and logging of the torque. This allows logging of the respective screwing cases carried out in their sequence. However, it is thus not possible to assign the recorded bolting cases to a respective bolting location on the vehicle. In order to be able to reconstruct later, at which point on the vehicle or the object to be screwed, the respective recorded torques have occurred, a manual assignment, recording, acknowledgment or logging is also required here.

From medical technology, it is known according to an own development of the applicant to monitor interventions on the body with video, being preferably carried out with infrared images. In this system, ISO surfaces are spanned via MIP points, which are overlaid against each other. The system requires a registered video overlay of real images and "design data." Markings on the tool make it easier to assign and track the operations performed.

DE 10 2010 032 553 A1 discloses a device for monitoring torque and connection conditions during the manufacturing process in the automotive industry. This device comprises a rotation sensor having a projector for projecting a predetermined pattern and an image converter 66 or camera. This generates three-dimensional data, so that it is possible to monitor the rotational distance traveled by a mechanical fastening means. A fastening device is based on this visual information and / or torque data and / or connection data driven accordingly.

DE 10 2012 003 809 A1 relates to a system with which the torque, which is necessary for opening a plastic screw cap of a bottle, is determined without contact. This is done by detecting corresponding marks by means of a camera.

A system for writing a position of an object is described in EP 1 963 786 B1. Here, the objects are optically detected.

US 2008/01 15589 A1 discloses a torque wrench having a torque measuring device. This system further includes a camera and a visible pointer. The visible pointer is, for example, a light signal that marks a particular connection to be tightened with the torque wrench. The camera can record the connected connection and automatically determine which connection is to be tightened next, so that the user is given an exact sequence. Furthermore, information about the individual connections can also be stored. From US 2008/0061 145 A1 a connection tool such. As a screwdriver out, with which connecting parts can be connected. The connecting parts are assigned a marking. The tool has a reading device that can detect this mark. The marker may be, for example, a barcode. This information can be compared with corresponding information of a database.

In US 2014/0137669 A1 a torque determination system is described, which determines the corresponding torque from predetermined angular positions. The angular positions can be detected with an X-ray system. DE 10 2013 017 007 A1 discloses a robot in which the force and torque input exerted by the robot can be determined.

It is an object of the present invention to provide an apparatus and a method for screwing on or to objects, in particular in the manufacture or assembly of vehicles, which make it possible to apply the respectively applied torques to the respective ones processed screwed locations with lower personnel costs and without changes to the object to be screwed reliably log. The above object is solved by the features of the independent claims. Advantageous developments and preferred embodiments of the present invention form the subject of the dependent claims. The present invention is based on the basic idea of monitoring an operating area with the object to be screwed and a tool unit designed for screwing screw connection elements by an image recording unit and analyzing the recorded images to at least the object and a screwing point a screwing is made to recognize and automatically generate a log in which the detected object and Verschraubungsort and applied during the screwing and detected torque or an applied angle of rotation are related to each other. Optionally, the records of the protocol, referred to in the context of the present invention as protocol tuples, may also represent data representing the tool unit also recognized by image analysis, and / or a time stamp representing a time at which the torque or torque is applied Rotation angle has been applied, and / or based on a work plan generated specifications regarding a Verschraubungsorts on the object and / or a Schraubfalls and / or a tool selection. The log tuples can also be used to monitor the bolting operation, to check the bolting success and / or progress, to feed back defaults and / or check results to a user, and to guide the user. Feedback can be made by optical and / or acoustic signaling and / or image display.

The protocol tuples are preferably not backed up until it has been determined that the tool unit has been removed from the screwing point; and / or the data acquired with respect to the object and the Verschraubungsorts and the applied torque or the executed rotation angle, preferably also with respect to the tool unit, comply with specifications; and / or a worn screw is acknowledged by a user. A backup can be, for example, but not limited to, printing, persistent storage in nonvolatile memory or volume, transmission to a remote administrative office, etc. The torque detected and contained in the protocol tuple, in particular in the secured protocol tuple, is preferably a maximum torque applied at the screwing location.

A device for screwing objects according to a first aspect of the present invention comprises

a tool unit that is designed to screw threaded fasteners, a rotation specification output device which is designed to output the torque to be applied during screwing with the tool unit or the angle of rotation to be executed during screwing, and

a data processing unit configured to detect torque data or rotation angle data representing the torque applied when screwing, an image pickup unit having a stereo camera adapted to receive a spatial image of a work area with at least a cutout of an object to be screwed; to provide the captured image data, wherein

the data processing unit has an image analysis unit which is designed to detect the object, the tool unit and a screwing location on the object in the recorded image and a protocol tuple (p) with data representing the recognized screwing location and the detected torque data and / or rotation angle data associated with each other.

Preferably, the data processing unit is designed such that the recognition of the object takes place on the basis of at least three reference points of the acquired spatial image. With three reference points, the object is clearly defined in space or in the three-dimensional coordinate system of the image acquisition unit.

Recognition of the object may also be based on the points describing a particular body that is part of the object. The points of this body thus represent reference points. Various algorithms are known with which individual bodies are extracted from spatial images, that is to say images which comprise three-dimensional information about the detected objects, and are described using a data model.

The detection of the object is preferably done by comparing the reference points with the corresponding points in a three-dimensional model of the object.

Such a model can be derived from design data. Preferably, however, such a model of the object is created in an empty learning process in which a plurality of spatial images are taken with the image acquisition unit. The model is then created from these spatial images by superimposing and aligning these images. Such a model, derived from the spatial images, describes the object in a manner very similar to the spatial images acquired in later operation with the imaging unit.

The use of a stereo camera to produce such spatial images of the object and comparison with a three-dimensional model of the object allows directing the camera with different viewing directions on the object and still achieve a reliable comparison with the model, whereby the object and its position in the coordinate system of the stereo camera can be reliably detected, even if the spatial images are detected by the object from different viewing directions.

A method of bolting articles according to another aspect of the present invention comprises the following steps:

Capturing a spatial image of a workspace with a stereo camera, the captured image comprising at least a portion of an article to be bolted;

Recognizing the object from at least three reference points of the acquired spatial image and a screwing location on the object from the one or more captured images;

automatically detecting a torque applied during screwing by the tool unit and / or an executed angle of rotation;

Generating a protocol tuple of data representing the bolting location and the torque and / or the rotation angle associated with each other.

A tool unit in the sense of the invention can be understood to mean any screwing tool which makes it possible to detect, measure, display, recognize, etc. a torque applied during screwing. In particular, the tool unit can be designed as a so-called torque wrench with a lever unit a tool holder, wherein in or on the tool holder a replaceable tool is received or receivable. The replaceable tool may be a so-called nut or a so-called bit, with or without extension, and is adapted for a respective screw to the screw connection element to be screwed or adaptable. Other examples of suitable tooling units include, but are not limited to, a screwdriver or impact wrench. The indication of the torque may be, for example, but not limited to, by an optical display such as a pointer, a scale, a clock or a digital display, and / or by sensing and transmitting measurement data. In this case, to capture the torque in the latter case, the sensor data is transmitted to a data processing unit wireless or wired and evaluated there, and in the former case, the image supplied by the image pickup unit is also evaluated to recognize the torque indicated by the optical display. For sensing the torque, a corresponding sensor may be provided. The rotation specification output device is preferably arranged in or on the lever unit In preferred embodiments, the tool unit, in particular the lever unit and / or the replaceable tool, carry an identification mark. For example, but not limited to, the identification tag may include a bar code, a color code, a 2D bar code, and / or a punch mark. The identification mark makes it possible to recognize the tool unit by means of the identification mark. The identification mark can be attached to an interchangeable marking unit, which is attached or attachable to a suitable location on the tool unit.

A carrier unit, which carries at least the image pickup unit, may preferably be in the form of a spectacle frame, protective goggles, a protective helmet, a headband or part of a garment such as an epaulette, a collar or a belt or fasteners for attachment to glasses, goggles , a protective helmet, a headband or on a garment such as an epaulette, a collar, a belt. Thus, a user can wear the image pickup unit on the body. When the image pickup unit is worn on the head, image acquisition advantageously follows the head movements of the user.

Preferably, the image capture unit is configured to generate a spatial image. For this purpose, the image acquisition unit in particular has a first image acquisition unit and a second image acquisition unit, which are arranged spatially separated from one another.

It is understood that the method can be repeated for a variety of Verschraubungsorten or screwing. The object is preferably identified by means of predetermined, stationary, first detection points on the object, wherein the first detection points are recognized by comparison with a previously stored model of the object, and wherein the first detection points are preferably inherent in the object. The detection of the Verschraubungsorts preferably takes place on the basis of predetermined, stationary second Er- points on the tool unit, the second detection points are detected by comparison with a previously stored model of the tool unit, the second detection points are preferably the tool unit immanent. In each case, the model can be derived from design data such as CAD data or generated by prior learning.

The image is a spatial image, which is captured by two spatially separated image acquisition units (= stereo camera). In this case, a geometry reference defined by the relative spatial position of the two image acquisition units can be used to identify the object. Other features, objects, advantages and effects of the present invention will be apparent from the following description of specific embodiments. To illustrate the embodiments reference is made to the accompanying drawings. Where:

Fig. 1 is a block diagram illustrating an apparatus for bolting objects according to an embodiment of the present invention;

Fig. 2 is a view of a tool unit which can be used in the embodiment;

Fig. 3 is a perspective view of a replaceable marking unit for attaching an identification mark to the tool unit of Fig. 2; 4 is a perspective view of an image pickup unit with a carrier unit according to an embodiment of the present invention;

5 is a perspective view of an image pickup unit with a carrier unit according to another embodiment of the present invention;

6 is a view of an image pickup unit with a carrier unit according to a still further embodiment of the present invention;

7 shows a schematic representation of a screwing situation to illustrate a method for screwing objects according to an exemplary embodiment of the present invention;

Fig. 8 is a schematic diagram schematically illustrating selected points in the situation shown in Fig. 7 in a coordinate system;

9 is a block diagram illustrating the method of bolting articles according to the present invention;

FIG. 10 shows a marking unit with a two-dimensional barcode. The present invention will now be described in detail by way of preferred embodiments with reference to the accompanying drawings. It should be understood that the pictorial representations are purely schematic and not necessarily to scale. Likewise, it should be noted that the drawings and the following description focus on the helpful features for understanding the invention, without thereby The scope of the present invention, which is defined solely by the appended claims, is to be limited.

Fig. 1 is a block diagram schematically illustrating a device for bolting objects in a bolting case. To perform a screw connection to an object 100, a device is used which has a plurality of elements, namely a tool unit 200, an image acquisition unit 300, a data processing unit 400 and an image output unit 500. As shown in FIG. 1, the object 100 has a plurality of reference points 1 10 and a screw 120 on. The reference points 110 are immutable detection points on the object 100, which are predetermined by the construction of the object 100. The reference points 110 can also be the points that describe a particular body of the object. Preferably, the reference points used are also those points which describe an elongate body. A data model stored in a model data storage unit 440 which enables the recognition and spatial association of the reference points 110 on the object 100 from a spatial representation will be described in more detail below. This model can be derived from design data and / or generated during a learning process in which multiple spatial images are taken with the image acquisition unit 300. Preferably, the model is generated from the spatial images acquired with the image acquisition unit 300, since such a model is very similar to the spatial images acquired during later operation. The multiple spatial images are superimposed and aligned to create the model. The screwing point 120 is predetermined by a work plan and linked to the bolting case. The bolting location 120 shown in FIG. 1 is preferably just one of many bolting locations to be machined according to the work schedule. Furthermore, more than three reference points can be defined on the object 100, but for spatial orientation at least three reference points 110 are required, which preferably do not lie on a straight line.

The screwing location 120 can be hidden behind a part of the object 100 both in the model and with the image acquisition unit 300 of acquired spatial images. For automatic recognition of the object 100, it is sufficient if the reference points are visible in the captured image. If the object 100 is identified and its arrangement in the space or in the coordinate system of the image acquisition unit 300 known, then such hidden Verschraubungsorte 120 can be identified. The Verschraubungsorte 120 and the reference points 1 10 can be entered during the learning process and integrated into the model to be created. Here, the work plan for learning the Verschraubungsorte 120 processed. For screwing to the object 100, a screw 150 is provided in the present embodiment. As shown in FIG. 1, a tool unit 200 is used for the screw connection, which has a torque wrench and a nut 250 adapted to the screw or the screw head of the screw 150. The torque wrench includes a lever arm 210, a pin 220 for coupling with the nut 250, and a torque indicator 230. On the lever arm 210, a marking unit 240 is attached with an identification mark. The identification mark is designed to uniquely distinguish the torque wrench 200 from other tools, and will be described in greater detail below with reference to a specific example of the marking unit 240. The torque wrench is understood as a tool unit 200 in the sense of the present invention, the lever arm 210 being understood as a lever unit, the torque indicator 230 as a turning output device, the pin 220 as a tool holder, and the nut 250 as a replaceable tool. The spigot 220 has a square cross-section in a known and conventional manner, which mates with a quadrangular recess in the nut 250, and may have a spring element (not shown in more detail) on one side, which has a lateral inner wall of the recess cooperates in the nut 250 to the nut 250 with the pin 220 captive, but releasably couple. The pin 220 may also be mounted in a ratchet unit (not shown in detail), which allows quasi-continuous screwing without repeated settling and reapplication of the tool. The torque indicator 230 comprises in the present embodiment an optical display 232 and a radio transmitter 234, which enables both a visual reading of the applied torque and a transmission of the torque in the form of measured value data or signals to a correspondingly configured radio receiver. Alternatively, instead of a radio transmitter, it is also possible to provide a cable interface or an attached cable in order to transmit measured value data or signals by cable. It should be noted that the present invention also works with only one of the optical display and the radio transmitter / cable interface / transmission cable. Essentially, in the context of the present invention, it is important to output the torque that is applied to the tool unit during the screwing in perceptible or receivable form.

Instead of the torque and the rotation angle of a screw can be detected. The detection of the rotation angle takes place by detection of the tool unit 200 in its different rotational positions by means of the image recording unit 300 and corresponding evaluation by the data processing unit 400. There are assembly instructions, which in glands a prescribed torque or a certain angle of rotation or a combination of both. For example, there are assembly instructions that initially prescribe a specific torque for screwed connections. When this torque has been reached, the screw connection must still be screwed on by a predetermined angle (eg 45 ° or 90 °). The rotation specification output device is therefore designed such that it can specify both a torque and a rotation angle.

As shown in FIG. 1, the image acquisition unit 300 comprises as essential components a camera unit 310, a control unit 320 and a radio interface 305. The camera unit 310 is designed and aligned to receive at least a section of the object 100 with the reference points 110 and the screwing location 120, and supplies image data of the recorded images to the control unit 320. The control unit 320 is designed to receive image data from the camera unit 310 and may be configured to send control data to the camera unit 310 to adjust acquisition parameters such as focus, aperture, depth of field, or frame. The control unit 320 is further coupled to the radio interface 305, and supplies the image data received from the camera unit 310 such that the radio interface 305 can provide the recorded image data to a remote station. In the present embodiment, the image pickup unit 300 further includes a laser pointer 330 as a laser pointing device, a speaker unit 340 as an acoustic output unit, and a light emitting diode unit 350 as an optical output unit. The laser pointer 330, the speaker unit 340 and the light-emitting diode unit 350 are data-coupled to the control unit 320, the control unit 320 being designed to output control data to the laser pointer 330, the loudspeaker unit 340 and the light-emitting diode unit 350.

As shown in FIG. 1, the data processing unit 400 includes a radio interface 405, a control unit 410, an image data acquisition unit 420, a measurement data acquisition unit 430, a model data storage unit 440, an image analysis unit 450, a timer unit 460, a logging unit 470, a schedule storage unit 480 , a default unit 485, and a feedback generation unit 495.

As shown in FIG. 1, the radio interface 405 is configured to establish and maintain a wireless data communication link with the rotation default output device 230 of the tool unit 200 and the radio interface 305 of the image capture unit 300. The control unit 410 is coupled to the radio interface 405 and is designed to supply image data received from the image acquisition unit 300 via the radio interface 405 to the image data acquisition unit 420 and to supply torque data received from the rotation specification output device 230 via the radio interface 405 to the measurement data acquisition unit 430 and feedback data from the response generation unit 495. The image data acquisition unit 420 is designed to receive image data from the control unit 410 and, if appropriate, store it temporarily and / or permanently and to provide it to the image analysis unit 450. The model data storage unit 440 is designed to store model data which store constructive and geometric characteristics of the object 100 with its fixed reference points (110, FIG. 1), the tool unit 200 or their identification features and to the image analysis unit 450. The image analysis unit 450 is configured to automatically recognize the object 100, the tool unit 200 and the screwing location 120 on the object 100 by comparing the image data supplied from the image data acquisition unit 420 with the model data supplied from the model data storage unit 440 and data indicating the detected bolting location and preferably reproduce the recognized tool unit to provide the log generation unit 470.

The measurement data acquisition unit 430 is designed to detect torque data and rotation angles received from the control unit 410, to temporarily and / or permanently store them, if necessary, and to provide them to the protocol generation unit 470.

The protocol generation unit 470 is configured to generate a protocol memory 275 from the data supplied by the image analysis unit 450 regarding the screwing location and preferably the detected tool unit, the torque data / rotation angle data supplied by the measurement data acquisition unit 430 and timestamps supplied by the timer unit 460 for backup and provide to the review unit 490. The log tuples generated by the log generation unit 470 may further include default data supplied from the default generation unit 485. The prediction generation unit 485 is designed to generate specification data from a work plan taken from the work plan storage unit 480 and to provide it to the protocol generation unit 470 and the verification unit 490. To synchronize with the actual workflow, the default generation unit 485 may also be coupled to the timer unit 460 and the image analysis unit 450. For example, the default generation unit 485 may generate default data based on a work cycle based on a time supplied by the timer unit 460 or may flexibly supply default data adapted to an individually selected work order based on a bolting location provided by the image analysis unit 450.

The checking unit 490 is configured to check the protocol tuples p (not yet saved) to see whether the defaults supplied by the default generating unit 485 have been reached or not, and to provide a corresponding check result to the feedback generating unit 495. The backup of the protocol tuple p in the log memory 475 can be linked to the condition that the specifications supplied by the default generation unit 485 are achieved, at least with regard to the screwing location and the detected torque or detected rotational angle. The backup of the protocol tuple p can also be done by manual acknowledgment be triggered by a user. Furthermore, a backup of a protocol tup p can occur automatically when the screwing process is interrupted, for example when the tool is set down by the screw 150. The feedback generation unit 495 is configured to receive protocol tuples provided by the protocol generation unit 470, and default data provided from the specification generation unit 485, and verification results provided by the verification unit 490, to generate feedback therefrom and to provide it to the control unit 410. The control unit 410 is further configured to provide feedback received by the feedback generation unit 495 via the radio interface 405 of the image acquisition unit 300 and to an image output unit 500 to be described below.

The control unit 320 of the image acquisition unit 300 can be configured to convert, via the radio interface 305, feedback received from the data processing unit 400 into control signals to the laser pointer 330, the loudspeaker unit 340, the light-emitting diode unit 350 and / or the camera unit 310. Thereupon, for example, the feedback may be output in acoustic form via the speaker unit 340, an optical confirmation or warning signal may be output via the light-emitting diode unit 350 and / or the camera unit 310 may change image acquisition parameters. If the feedback generated by the feedback generating unit 495 contains default data, the laser pointer unit 330 can also generate a laser beam directed onto the screwing location 120 to be processed in order to guide a person to the corresponding screwing location 120.

The above-mentioned image output unit 500 is provided in the present embodiment as a tablet PC having a radio interface 510, a display unit 520, and a speaker unit 530. Based on the feedback received via the radio interface 510, as shown here, default values for the screw 150 to be selected as well as the torque prescribed for this purpose (tightening torque) can be output. It is understood that the principle described above allows many variations and options. For example, a green, a red and possibly a yellow LED of the light-emitting diode unit 350 of the image recording unit 300 can be activated in order to convey a work result to a user in color-coded form. For example, the correct positioning of the tool unit 200 by a yellow illumination and the correct tightening torque achieved can be indicated by a green illumination. By flickering with different frequency in yellow color, for example, an approximation of the prescribed torque, by flickering in red color too high a tightening torque can be signaled. In addition, on the screen 520 of the image output unit 500, the work success can be indicated by a text message or a color underlay. An embodiment of the torque wrench (tool unit) 200 is shown in more detail in FIG. 2, and the marking unit 240 for use on the tool unit 200 is shown in FIG. 2 is a plan view of the tool unit 200 (torque wrench), and FIG. 3 is a perspective view of the marking unit 240 in the form of a sleeve.

As shown in FIG. 2, the torque indicator 230 is in the form of a digital display. The digital display of the torque indicator 230 is readily recognizable by the camera unit 310 of the image capture unit 300 (FIG. 1). The marking unit 240 is attached to the lever unit 210 near the tool holder 220. As an alternative to a digital display, the applied torque can also be moved by a pointer instrument or a cantilever that does not deform with the lever unit 210, optionally in conjunction with a scale attached to the lever arm 210, which moves along the cantilever upon deformation of the lever arm 210 ., or any other suitable visual representation. As shown in FIG. 3, the two-shell cuff marking unit 240 is formed with a top 242 and a bottom 243 connected by a series of screws 244 for clamping the marking unit 240 to the lever arm 210.

The marking unit 240 further carries an identification mark in the form of a bar code 246 on the one hand and a hole marking or coding 248 on the other hand. Both types of marking can be easily read by the camera unit 310 of the image recording unit 300.

10 shows an alternative example of an approximately rectangular marking unit 240 having a two-dimensional barcode 247 for coding an identification number for the respective tool and at the corners each having a position mark 248, by means of which the position of the tool in space or in the coordinate system the image acquisition unit is detectable. The two-dimensional code is mirrored around a centerline. Based on the position marks 248, the position of the tool in the three-dimensional coordinate system of the camera can be easily determined.

This marking unit 240 can be printed out with a conventional printer. The marking unit 240 is then glued onto the tool unit 200, in particular the lever arm 210. After attaching the marking unit 240 to the tool unit 200, it is scanned in a learning process, whether the mounting unit is a collar or a glued label, so that the position of the marking unit 240 on the tool unit 200 is uniquely detected. In later operation, the position of the tool unit 200 in the three-dimensional coordinate system of the image recording unit 300 can then be clearly deduced on the basis of the detection of the marking unit 240. For convenient entrainment of the image pickup unit 300 by a user, a support unit may be provided on which the image pickup unit 300 can be carried compactly or distributed on the body. Some concrete embodiments of the imaging unit 300 with carrier unit 600 are shown in FIGS. 4 to 6 shown schematically.

According to the representation in FIG. 4, the carrier unit 600 is designed according to a specific exemplary embodiment in the form of protective goggles. This goggle comprises a mask unit 622 which encloses protective glasses 624 and an elastic band 626 located behind the head, the band 626 being adjustable by means of a closure unit 628. The image pickup unit 300 is laterally attached to a junction between the mask unit 622 and the elastic band 626 in this embodiment. According to the representation in FIG. 5, the carrier unit 600 according to a further concrete exemplary embodiment is in the form of safety glasses with a frame 642 which accommodates two protective glasses 644 and two brackets 646, 646, wherein the image recording unit 300 is distributedly arranged on the carrier unit 600 , Thus, each camera unit 310 is attached to the right and left brackets 646, 646, respectively, and a speaker unit 340, the housing of which also accommodates the control unit 320 and the radio interface 305 (FIG. 1), is attached to the left bracket 646 a light-emitting diode unit 350 with three different-colored LEDs mounted on a nose root portion of the frame 642. A wiring 648 between the camera units 310, the light emitting diode unit 350 and the speaker unit 340 with radio interface 305 and control unit 320 is integrated at the upper edge of the frame 642.

As shown in FIG. 6, a carrier unit 600 according to yet another exemplary embodiment of the present invention is designed as a protective helmet 660, on each of whose sides a camera unit 310 is attached. The camera units 310 may be fixedly mounted, may be attached to a bracket integrated with the helmet 660, or may be integrated with a bracket attachable to the helmet 660. The further device technology of the image recording unit 300 can be distributed in one of the camera units 310, on both camera units 310 or integrated into the protective helmet 660.

FIG. 7 is a spatial schematic representation of a deployment situation within the meaning of the present invention. A user 700 uses the torque wrench 200 to make a screw connection at a screwing point 120 on an object 100, in this case a motor vehicle. The user 700 carries an image pickup unit 300 by means of a helmet-shaped carrier unit 600 (see Fig. 6). Three reference points 1 10, which are structurally predetermined and immutable per se, are specified on the object 100. On the tool unit 200 (torque Key) are three reference points 710 constructive and / or by marking unit (240, Fig. 3) predetermined or identifiable.

As seen in FIG. 7, in the illustrated application, the bolting site 120 is hidden behind structural members of the article 100. Nevertheless, it is possible to determine from the reference points 110 of the object 100 and the reference points 710 of the tool unit 200 at which point on the object 100 the tool unit 200 attaches.

This process will be described below with reference to a schematic diagram shown in FIG. 8. FIG. 8 shows the reference points 110 of the object 100 and the reference points 710 of the tool unit 200 in a virtual space 800. As shown in FIG. 8, the reference points 110 of the article 100 span a first reference triangle 810 defining a first reference plane 815. Likewise, the reference points 710 of the tool unit 200 clamp a second reference triangle 820 defining a second reference plane 825. The reference points 1 10, 710 are detected by the camera units 310 of the image acquisition unit 300. Since the camera units 310 are spatially separated from one another, from the acquired image data, the reference points 110, 710 can be assigned to coordinates X1, X2, X3 of the virtual space 800, whereby the absolute position of the object 100 and the tool unit 820 in virtual space 800 as well as the relative position between the object 100 and the tool unit 200 are completely defined. It is understood that the mathematical operations required for this, including the generation of a mathematical model of the object 100 with its reference points 110, a mathematical model of the tool unit 200 with its reference points 710, the generation of a spatial image from two stereoscopic images and their assignment to coordinates in a virtual space are well-known in the art as are the techniques of image analysis that can be used to compare delivered image data with the previously generated and stored model, and therefore need not be described here, but instead, as pertinent to the art the technique can be referenced. FIGS. 5 and 6 each show a carrier unit 600 with two camera units 310, so that these camera units 310 each form a stereo camera. In Figure 4, the image pickup unit 300 is shown only schematically. This is also a stereo camera. The use of a stereo camera over other known image acquisition units for detecting three-dimensional structures offers the advantage that the three-dimensional images can be generated very quickly. In a prototype of the invention, the imaging unit 300 generates two spatial images per second. It is expedient that at least one spatial image is generated at least every two seconds, preferably at least one, two or at least three images are generated per second. On the other hand, the resolution of a stereo camera is sufficient to be able to unambiguously indicate the screwing locations 120 to the operator. The use of the stereo camera thus makes it possible to provide the image pickup unit 300 on a carrier body 600 arranged on an operator's body, whereby the operator can move freely and occupy an arbitrary position with respect to the object 100, as long as the viewing direction of the image pickup unit 300 on the object 100 is directed. By comparing the acquired spatial image (s) to the spatial model of the object stored in the model data storage unit 440, the spatial image data can be quickly matched with the spatial model. As a result, the relative positions of the image pickup unit 300 relative to the object 100 are uniquely identified, so that the three-dimensional coordinate system of the image pickup unit 300 is fixed and the position of the object 100 in this coordinate system is determined. Pointer elements, such as the laser pointer 330, are preferably mechanically connected to the image acquisition unit 300, so that their position in the coordinate system of the image acquisition unit 300 is also clearly defined.

An operator who carries the carrier unit 600 can thus move freely with respect to the object 100, whereby on the one hand the screwing locations 120 are automatically displayed or predetermined and on the other hand the screwing is completely logged (torque and / or angle of rotation, time stamp).

9 is a block diagram illustrating processes in the implementation of an article screwing method according to another embodiment of the present invention. In FIG. 9, an area 910 symbolizes a physical production level, an area 920 symbolizes a data processing level, and an area 930 symbolizes a tool data level. A first interface level 921 establishes a link between the physical production level 910 and the data processing level 920, and a second interface level 923 establishes a link between the tool level 930 and the data processing level 920. The physical plane 910 may be substantially compared to the article of manufacture 100 image capture unit 300 and the image output device 500 in FIG. 1, the data processing plane 920 may be substantially equated with the data processing unit 400 of FIG. 1, and the tool level 930 may be referred to in FIG Substantially compared to the tool unit 200. The interface plane 921 can be compared here with the radio interfaces 305, 405, 510 and associated radio transmission links in FIG. 1, and the interface plane 923 can essentially be integrated with a radio transmitter integrated in the rotary output device 230 of the tool unit 200 and the radio interface 405 of FIG Data processing unit 400 in Fig. 1 are compared. An arrow 912 in FIG. 9 symbolizes a production sequence, for example, on a vehicle assembly line during vehicle production. Once a vehicle can be picked up at an assembly site, a NOK message with vehicle ID is made in step 914. Of course, bypassing the data processing level 920, the assembly of the vehicle can also be carried out manually (generally represented as step 915) with manual OK setting 916. According to the invention, the assembly, in particular the setting of fittings on the vehicle, but including the data processing level 920. In this case, data from the physical production level 910 are introduced via the interface level 921 in the data processing level 920 and provided in the data processing level 920 data via the interface level 921 in the physical production level 910 made representable. For this purpose, a query 917 takes place, in the form of a repetitive loop, until an OK message 918 takes place from the data processing level 920. After manual OK setting 916 or automatic OK message 918, the vehicle can be returned from the assembly point in the production process 912.

At data processing level 920, multiple processing blocks are defined. These are in particular a start block 940, a data load block 950, a recognition block 960, a navigation block 970 and a monitoring block 980. On the tool level 930, essentially a measurement data generation block 990 is to be considered, in which at the request of the navigation block 970 of the data processing level 920 measurement data on the tool (FIG. Tool unit 200, FIG. 1) and provided to the monitoring block 980 of the data processing level 920. The request and provision of data between the data processing level 920 and the tool level 930 is via the interface level 923. After implementation of the start block 940, the data load level 920 is implemented on the data processing level 920 where data is loaded on the vehicle, the screwing situation, and the tool. Then, the recognition block 960 is implemented, in which a recognition is first started in a step 962, then a recognition is performed in step 964 and, after recognition in step 966, the recognized state is registered. The start of the recognition in step 962 can be compared to an activation of the image recognition unit 300 in FIG. 1, in particular of the camera units 310. The execution of the detection in step 964 corresponds, for example, to the detection of the vehicle (object 100), the screw joint 120 and the tool unit 200 by image analysis in the image analysis unit 450 on the basis of the image data supplied by the camera unit 310 and acquired by the image data acquisition unit 420 in comparison with FIG the model data stored in the model storage unit 440, in particular based on the predetermined and invariable reference points 1 10 and the marking unit 240 (FIG. 1) as well as on the tool unit 200 identifiable reference points 710 (FIG. 7). The registration of the detection in step 966 corresponds to the generation of a data set containing data that identifies the recognized vehicle (the detected object 100), the detected tool unit 200 and the recognized screw 120 (Fig. 1) play. The generated record is then passed to navigation block 970.

In the navigation block 970, the assembler navigates to the screwing location in step 972, for example via the laser pointer device 330 in FIG. 1, the output of default data via the image output unit 500 and optical and / or acoustic signaling via the loudspeaker unit 340 or the light-emitting diode unit 350 can be done in Fig. 1. As soon as it has been recognized that the tool (the tool unit) 200 with the required screw 150 is set at the intended screwing point 120, a detection message is sent to the monitoring block 980 in step 974. Simultaneously, in step 976, a start of the screwing operation is started includes a request for measurement data from the tool unit 200 (FIG. 1).

Upon request by navigation block 970, processing block 990 on tool level 930 generates measurement data corresponding to the applied torque on tool unit 200 (FIG. 1) and also transmits it to the monitoring block.

The monitoring block 980 performs a monitoring of the tightening operation in step 982 based on the measurement data supplied from the tool level 930 (processing block 990), and ends the monitoring in step 984 at the time of Sole achievement. Thereupon, in step 986, a generation of logs is performed, generating log tuples, which generate and store the record registered in step 966 along with the data representing the detected torque. The protocol can be transferred to the interface level 921 in the form of log files and / or video logs 925. Similarly, the monitor block 980 provides an OK message 927 to the interface level 921 when the vehicle is still active, and via the interface level 921, the OK message 927 is translated into the OK message 980 described above.

It should be pointed out that the sequence described above is under the regime of a predetermined clocking and, within the framework of this cycle, screwing operations are executed on the basis of a work plan (see 480, FIG. 1), with a bolting ID being generated for each screwing case , The Schraubfall-ID and the above-mentioned vehicle ID are respectively part of the protocol tuple of the generated protocol. In the example explained above, the article 100 is a motor vehicle. Within the scope of the invention, other articles, such as e.g. Aircraft, medical devices or the like may be used.

In the embodiments described above, an operator or fitter carries the carrier unit 600, on which the camera units 310 are arranged. Within the scope of the invention It is also possible to provide an automatic screwing, in which the tool unit 200 is automatically moved to the respective screw 120 and operated. For this purpose, for example, a robot arm may be provided. The automatic screwing machine and the object to be screwed can be moved relative to each other, wherein in each case at least one spatial image of the object is detected in the different positions. The spatial resolution of the spatial images acquired with the image acquisition unit 300 may not be sufficient to control the tool unit 200 sufficiently precisely to the screwing location 120 based solely on a single spatial image of the article 100. If this is the case, additional spatial images can be recorded for positioning the tool unit 200 during the positioning process, wherein the tool unit 200 and the object 100 are detected here, so that the relative position of the tool unit 200 to the object 100 is determined on the basis of these images. This relative position is determined with a much higher spatial resolution than the absolute position of the object 100 in the three-dimensional coordinate system of the camera. Thus, with a closed control loop, the tool unit 200 can be positioned very precisely fully automatically at the screwing location 120.

Such a closed-loop control for positioning the tool unit 200 can also be used in a "mobile" system with a portable carrier unit 600, wherein the fitter then optical, acoustic or other instructions are given as the tool unit 200 relative to the object 100 to has to position.

It should be understood that the foregoing description may include only exemplary embodiments of the present invention, and the invention is not limited to the specific embodiments described, but is defined solely by the appended claims in their broadest sense. Variations, additions, substitutions, and equivalents which those skilled in the art will make from the skilled artisan in the described embodiments are to be understood as embodiments of the present invention insofar as they come within the scope of the appended claims.

Claims

claims

Device for screwing objects, comprising:

a tool unit (200) configured to screw threaded fasteners (150);

a rotation specification output device (230) configured to output the torque to be applied when screwing with the tool unit (200) and the rotation angle to be performed in the screwing, and

a data processing unit (400) configured to detect torque data or rotation angle data representing the torque applied during screwing or the rotation angle performed during screwing,

an image pickup unit (300) having a stereo camera for picking up a spatial image of a work area with at least a portion of an article to be bolted (100) and providing the captured image data, and the data processing unit (400) comprises an image analysis unit (450) formed is to detect the object (100), the tool unit (200) and a screwing location on the object (100) in the captured image, and a log tuple (p) with data representing the detected bolting location and the detected torque data and / or rotational angle data associated with each other.

2. Apparatus according to claim 1, characterized in that the data processing unit is designed such that the detection of the object (100) based on at least three reference points (1 10) of the detected spatial image.

3. Apparatus according to claim 2, characterized in that the detection of the object (100) takes place on the basis of a body which is part of the object and is represented in the spatial image by a plurality of reference points.

4. Device according to one of claims 1 to 3, characterized in that the protocol tuple (p) further data representing the detected tool unit (200), and / or a time stamp, the time applied to the torque or the Rotation angle has been executed, reproduces, has. 5. Device according to one of claims 1 to 4, characterized in that the tool unit (200) has a lever unit (210) with a tool holder (220) and a replaceable tool (250) which is received in or on the tool holder (220) and the tool (250) is adaptable to the screw connection element (150) to be screwed, wherein the rotation specification output device (230) is preferably arranged in or on the lever unit (210).

6. Device according to one of the preceding claims, characterized in that the rotation specification output device (230) has at least one of the following:

an optical display (232), such as a pointer, a scale, a clock or a digital display, wherein the image analysis unit (450) of the data processing unit (400) is designed to detect the torque or torque indicated by the optical display (232). to detect the rotation angle from the image supplied from the image pickup unit (300);

a data transmission unit (234) configured to transmit measurement data representing a sensed torque, wherein a receiving unit (405) of the data processing unit (400) is designed to be the one of the data transmission unit

(234) and a measurement data acquisition unit (430) of the data processing unit (400) is designed to interpret and to record the received measurement data as the torque data. 7. Device according to one of the preceding claims, characterized in that the tool unit (200), in particular the lever unit (210) and / or the interchangeable tool (250), an identification mark (246, 248), in particular a bar code, a color code, a 2D bar code and / or a hole marking, wherein the data processing unit (400) is designed to recognize the tool unit (200) based on the identification mark (246, 248), the tool unit (200) preferably a replaceable marking unit (240) carrying the identification mark (246, 248).

Device according to one of the preceding claims, characterized in that it further comprises a carrier unit (600) which carries at least the image pickup unit (300) and preferably in the form of a spectacle frame, goggles, a protective helmet, a headband or part of a garment such as an epaulette, a collar or a belt, or fasteners for attachment to glasses, goggles, a hard hat, a headband or a garment such as an epaulette, a collar, a belt.

9. Device according to one of the preceding claims, characterized in that the data processing unit (400) a default generating unit (485) which is formed, defaults with respect to a Verschraubungsorts on the object (100) and / or a Schraubfalls and / or a tool selection, preferably the protocol generation unit (470) is preferably designed to add specifications generated by the default generation unit (485) to the respective protocol tuple (p), and wherein the data processing unit (400) preferably has a verification unit (485). , which is designed to check the achievement of the specifications and to generate a verification result has.

10. The device according to claim 9, further characterized by an acoustic output unit (340), which is coupled to the data processing unit (400), wherein the data processing unit (400) is configured to supply the specifications and / or check results to the acoustic output unit (340 ), and the acoustic output unit is designed to output acoustic indication, warning and / or confirmation signals corresponding to the specifications and / or check results.

1 1. Apparatus according to claim 9 or 10, further characterized by an optical output unit (330, 350), which is coupled to the data processing unit (400), wherein the data processing unit (400) is configured, the defaults and / or verification results to the optical output unit (330, 350), and the optical output unit (330, 350) is designed to output optical indication, marking, warning and / or confirmation signals corresponding to the specifications and / or verification results, the optical output unit (330 , 350) preferably comprises one or more light emitting diodes (350) of one or more colors, a laser pointing device (330) and / or a hologram generating device.

12. Device according to one of claims 9 to 1 1, further characterized by an image output unit (500) which is coupled to the data processing unit (400), wherein the data processing unit (400) is formed, the specifications and / or verification results of the image output unit (500), wherein the image output unit (500) is configured to output instructions, instructions, warnings or confirmations corresponding to the specifications and / or the verification results.

The apparatus according to one of the preceding claims, further characterized by a log backup unit (475) configured to save the log tuple (p) generated by the log generation unit (470), wherein the log generation unit (470) is adapted to create a log tuple (p ) preferably to the log backup unit (475) if at least one of the following conditions is met:

the tool unit (200) is removed from the bolting station (120);

the recorded data concerning the object (100) and the screwing location and the applied torque, preferably also with regard to the tool unit (200), are in accordance with specifications;

- A completed screwdriver case is acknowledged by a user.

Method for screwing objects, with the steps:

Capturing a spatial image of a workspace with a stereo camera, wherein the captured image comprises at least a portion of an article (100) to be bolted; Recognizing the object (100) from at least three reference points (110) of the acquired spatial image, and a screwing location on the object (100) from the one or more captured images;

- Automatic detection of a screw applied by the tool unit (200) torque and / or an executed angle of rotation;

15. The method according to claim 14, characterized in that the detection of the counter state (100) takes place on the basis of reference points which describe a body which is part of the object.

16. The method of claim 14 or 15, characterized in that the detection of the object by comparing the reference points with the corresponding points in a three-dimensional model of the article (100) takes place.

17. The method according to any one of claims 14 to 16, further characterized by the steps:

Recognizing a tool unit (200) for screwing the object (100), generating data representing the recognized tool unit (200), and

Add the data to the respective protocol tuple (p).

18. The method according to any one of claims 14 to 17, further characterized by the steps:

Detecting a time at which the torque and / or rotation angle has been applied, generating a time stamp representing the detected time, and

Add the timestamp to the respective protocol tuple (p).

19. The method according to any one of claims 14 to 18, characterized in that the method is repeated for a plurality of Verschraubungsorten.

20. The method according to any one of claims 14 to 19, characterized in that the detected and contained in the protocol tuple (p) torque is a maximum applied to the Verschraubungsort (120) torque.

21. Method according to one of claims 14 to 20, characterized in that the object (100) is detected on the object (100) on the basis of predetermined, stationary first detection points (110), wherein the first detection points (110) by comparison with a pre stored model of the article (100) are detected, wherein the first detection points (1 10) are preferably the object (100) immanent.

22. The method according to claim 14, characterized in that the screwing location is detected on the tool unit (200) on the basis of predetermined, stationary second detection points (710), the second detection points (710) being compared with a second detection point previously stored model of the tool unit (200) are detected, wherein the second detection points (710) are preferably the tool unit (200) immanent.

A method according to claim 21 or 22, characterized in that the model is derived from design data such as CAD data or generated by previous teaching.

24. The method according to any one of claims 14 to 23, characterized in that the tool unit (200) on the basis of a tool unit (200) pre-attached image coding (240 is detected.

25. The method according to any one of claims 14 to 24, characterized in that the detection of the torque comprises at least one of the steps:

Evaluating the image and detecting a torque indicator (230) on the tool unit (200) in the image;

- Evaluating sensor data, which are generated by the tool unit (200) and preferably transmitted to a data processing unit (400) wireless or wired.

26. The method according to any one of claims 14 to 25, further characterized by the step:

Specifying a Verschraubungsort and / or a Schraubfalls and / or a tool selection, preferably based on a previously stored workflow, wherein the generated protocol tuple (p) preferably also data, which the predetermined screwing and / or the predetermined screwing and / or the predetermined Play tool includes.

27. The method according to claim 26, characterized in that further comprising the step:

Outputting information about the predefined screwing location and / or the prescribed screwdriving case;

wherein the dispensing comprises at least one of the following measures:

- displaying the information on a display unit (520);

Outputting a voice by an acoustic output unit (340, 530);

- Marking the Verschraubungsorts by illumination (350) and / or laser light (330);

- generating a hologram; Outputting an acoustic and / or optical approach signal, which indicates the distance or approach with respect to the screwing location and / or the reaching of the screwing location and / or the achievement of a predetermined torque, by an acoustic output unit (340, 530) or an optical output unit ( 350).

Method according to one of claims 14 to 27, further characterized by the step of avoiding a work progress and / or result to a user, preferably by at least one of the measures indicated in the preceding claim 25.

A method according to any of claims 14 to 28, further characterized by the step of saving the protocol tuple (p), preferably when at least one of the following conditions is met:

- The tool unit (200) is removed from the Verschraubungsstelle (120);

- A completed screwdriver case is acknowledged by a user.