DE102012111345A1 - Mobile handheld device e.g. smart phone for aligning arrangement of optoelectronic sensor, has interface unit to receive range of view image and information about position of light spots in range of view image - Google Patents

Abstract

The mobile handheld device (100) has a display unit (104) that is provided for displaying an instantaneous orientation representing an overlay image that is on a range of view image of a monitored area (18) of sensor (10). A destination indicator (116) is provided for indicating position of sensor based on generated light spot (22). An interface unit (108) is provided to receive the range of view image and information about the position of light spots in the range of view image. A fusion unit (114) is provided for producing the overlay image. Independent claims are included for the following: (1) an optoelectronic sensor for detecting objects in monitored area; (2) an arrangement for aligning optoelectronic sensor; and (3) a method for aligning sensor for detecting objects in monitored area.

Description

The invention relates to a mobile handheld device for aligning a sensor, an optoelectronic sensor for detecting objects in a surveillance area and a method for aligning a sensor for detection by means of a handheld device according to the preambles of claim 1, claim 7 and claim 13.

Many opto-electronic sensors emit a light beam and evaluate the own transmitted light reflected from an object in its surveillance area. Such sensors often require a fairly accurate orientation, in which the light spot which the transmitted light generates on the object is brought into a position suitable for the task of the sensor. For example, in this way a reflection light barrier is aligned with its reflector or a data light barrier on its counterpart, with a light grid of the transmission rod with the receiver rod, with a distance sensor the object to be measured touched or set the monitoring level of a laser scanner.

As long as the transmitted light leaves a visible light spot, the alignment is relatively easy to do with the eye. However, this is often not the case for two reasons: the intensity of the light spot is not necessarily sufficient to recognize it well from a distance, and above all, many sensors use invisible light, for example in the infrared or ultraviolet range.

Therefore, numerous alignment aids are known in the art. In a conventional solution, the sensor will indicate the receive level and, if possible, use a position-sensitive light receiver to indicate in which direction the sensor should be rotated for better alignment. With active remote station, such as in a light grid or a data light barrier, this can give a feedback about the received intensity. Other solutions seek to increase the visibility of the light spot, such as by providing an additional pilot transmitter with visible light in a parallel direction of radiation to the own light emitter or an optical reticle. It is also conceivable to start from a fixed radiation direction and to align the sensor independently of the actual light spot. This can be a target help on the sensor over the sight and grain or the use of a spirit level.

Many of these solutions have to be tailored to the specific type of sensor. This requires effort for the development and production of the sensors. Nevertheless, the user-friendliness remains limited.

From mobile devices is known to represent images in the so-called augmented reality with their own camera. In the process, additional information is superimposed on the live image. Such applications mark, for example, the sun track on arbitrary days in the year for the apartment inspection or planning a photovoltaic system. For the alignment of sensors such applications are not intended and unsuitable.

In addition, known cameras display selectable focus areas during the image search, some of which are measured by a distance sensor. However, this serves at best the orientation of the camera itself and in no way helps to align an external optoelectronic sensor.

It is therefore an object of the invention to improve the alignment of an optoelectronic sensor.

This object is achieved by a mobile hand-held device for aligning a sensor, an opto-electronic sensor for detecting objects in a surveillance area and a method for aligning a sensor for detection using a hand-held device according to claim 1, claim 7 and claim 13. The invention is based on the basic idea of outsourcing evaluations and displays required for alignment from the sensor to a mobile handheld device. To this end, the invention comprises, on the one hand, a mobile handset capable of receiving a field of view image of the sensor and a position of the light spot in the field of view image from a sensor and, based thereon, the alignment in a sub-picture of the field of view and a target display of the position of one of To visualize the light spot generated by the sensor. On the other hand, an associated sensor is provided which provides its field of view image and the position of the light spot within the field of view image for a mobile handset. Under a mobile handset here is a portable computer in the broadest sense, such as a notebook or a tablet and in particular a mobile device or smartphone are understood.

The invention has the advantage that the orientation of a sensor is greatly facilitated and supported by a particularly intuitive, user-friendly universal alignment, which requires no special knowledge of a particular sensor. High ease of use, high computing power and a high resolution display are not provided by the sensor, but by the handheld device. Access to such a handheld device in current hardware is almost always given independent of the sensor. While the sensors can be extremely simple and cost-effective in their performance and visualization, and remain unchanged for many years, the alignment still does not become obsolete, allowing the user to work easily, quickly, and comfortably without the devices being outdated to feel. The user can even make the alignment with their own familiar device, such as their smartphone, and does not have to get involved with third-party technology. It creates an ideal synergy with the advances in mobile consumer electronics, which can be used decoupled from the development of the sensor.

The handset preferably has its own image sensor for receiving an additional field view image of the monitored area, wherein the fusion unit is adapted to generate the overlay image also based on the additional field view image. The handset uses its own camera to record its own field of view. This camera is usually available anyway, powerful and high-resolution, so that the sensor needs to provide no good for the user viewing area image and thus already gets by with a comparatively low-power and therefore inexpensive image sensor. Because the field of vision image of the sensor is only used to adjust the position of the light spot in the additional field view image of the camera of the handset, and possibly even to increase the image quality of the additional field view image such as filling gaps or a plausibility of image areas.

Also, image information of the light spot itself, which may be shown with, comes from the visual field image of the sensor, as long as the image sensor of the handheld device is not sensitive to the wavelength of the light spot. The handset should have approximately the same perspective as the sensor for taking in the extra field of view. Remaining perspective differences, however, the fusion unit preferably expects, so that no exaggerated accuracy requirements are placed on the user here.

The fusing unit is preferably configured to match the viewing area image of the sensor and the additional viewing area image of the own image sensor to determine the position of the light spot in the additional field of view image. In the simplest case, it is assumed that the image areas of the field of view image of the sensor and the additional field view image of the handset's own image sensor correspond to each other so that the target display is simply superimposed at the same position on the additional field of view image which the sensor also determines for its own field of view image Has.

Significantly more robust is a registration of the two viewing area images, ie by operations such as shifting, twisting and rescaling the visual field image of the sensor is fitted in the additional field view image of the image sensor of the handset. This is achieved, for example, by correlating the two field of view images, which can be limited to selected image features or regions of interest (ROI) instead of the entire images. Thereafter, the operation is known which transforms the perspective of the sensor to that of the hand-held device, and this transformation rule is used to display the target display in the correct position in the additional field of view image.

If the image sensor of the handset is sensitive to the wavelength of the light spot, an attempt can be made to locate the light spot directly by image processing. The determined position of the light spot can also be used as a starting point or for plausibility checking during the registration of the two viewing area images.

The handset preferably has a position and / or acceleration sensor, wherein the fusion unit is designed to determine a change in orientation and / or position of the handset and to take into account in the merger of the target display in the overlay image. Such sensors are available for many high-quality mobile devices. It is thus possible, based on an initial position and orientation balance between sensor and handset aufintegrieren the movements of the handset and thus gain knowledge of the current perspective of the handset relative to the sensor. This allows the position of the light spot to be converted to the current position and orientation of the handset.

The handset preferably has a position sensor, wherein the fusion unit is designed to take into account a position of the handset determined by the position sensor when fusing the target display into the overlay image. The detection of the position, for example, by GPS, Galileo or a comparable, possibly also proprietary system for outside or inside. As far as the position sensor detects no orientation, a position sensor can be added. If the position and orientation of the sensor is known, which can also be determined by the position sensor sensing during physical contact between the sensor and the handset, the detected current position is one Another alternative to bring the target display in the overlay image in the correct position.

Image fusion by correlation, determination of the path between the sensor and handset by a position and / or acceleration sensor, and the determination of the current position of the handset by a position sensor are three different possibilities, the perspectives of the sensor and the handset for a correct target display in the To convert overlay image into each other. These possibilities can also be combined with each other for more accurate results or to detect errors

The fusion unit is preferably designed to calculate the overlay image back to the perspective of the sensor. In this case, the higher-quality additional field of view image is displayed by an image sensor of the hand-held device, but as if it were taken from the perspective of the sensor. The perspective differences are determined, for example, with one of the procedures summarized in the last paragraph. However, the perspectives of sensor and handheld device should not be too different for this, because otherwise not enough suitable image information will be available. However, it is at least possible to display a superimposed on the position of the light spot overlay image with the field view image of the sensor, which is improved depending on the perspective of the handset with image information of the image sensor of the handset.

The inventive sensor for detecting objects in a surveillance area preferably generates a light spot with transmitted light outside the visible spectrum or with so little expansion or intensity with its light emitter when striking an object that the light spot is not or hardly recognizable with the human eye. Otherwise you could directly use the light spot for alignment. The sensor captures a field of view image, determines or knows the position of the light spot with respect to its own field of view image, and outputs field of view image and position for use in a handset of the invention. The sensor is, for example, a single-beam system, such as a distance sensor or a light barrier, in particular a data light barrier. However, other sensors, such as laser scanners or light grids, are also conceivable. The sensor requires only a small amount of hardware for the alignment without its own display driver, high-quality display or high-resolution image sensor and virtually no computing capacity, since all more complex evaluations can be made in the handset. Of course, the presence of such hardware equipment of the sensor should not be excluded.

The light spot position determination unit preferably has a memory in order to advance the position of the light spot in advance. The sensor thus knows the orientation of the light transmitter to itself and thus also with respect to its own image sensor for recording the field view image. This own orientation can be easily determined by a tolerance model, which ultimately assumes that the type of construction of the sensor determines its own orientation. If you want to get a closer grip on the resulting tolerances, an initial teach-in during the adjustment of the sensor components in the course of production or in the final test is possible. Here, defined environmental conditions and standardized procedures are given by qualified personnel, so that simple and intuitive procedures are less important. The orientation addressed by the invention, on the other hand, preferably does not relate to the sensor in itself but to its environment in use.

The light spot position determination unit is preferably designed to determine the position of the light spot by means of image processing. In order to fulfill its object detection task, the sensor is inevitably able to detect the light spot even with transmitted light outside the visible spectrum. However, the image sensor does not necessarily have to be used to record the field of view image. If, however, this image sensor is sensitive to the light spot, then the position of the light spot required as the output variable to the handheld device for the sensor's own perspective, that is, with respect to the field of view image of the sensor, can be determined from the field of view image by image processing.

If the sensor is designed to be distance-measuring, the stored, taught or determined positions of the light spot can also have a dependence on the respective object distance. So no single position, but a distance-dependent function is provided.

The sensor preferably has a preprocessing unit to prepare the field of view image for output at the interface. This then requires some internal computational capacity, but helps to reduce the bandwidth needed. Examples of such pre-processing are compression, binarization, restriction to areas of interest, in particular with the light spot or prominent objects, or image features or object edges presumably recognizable by the fusion algorithm of the hand-held device.

In an advantageous embodiment, an arrangement for aligning an optoelectronic sensor according to the invention with a mobile handheld device according to the invention is provided. The interfaces of the sensor and the handset are preferably wireless. In addition, practically all known and future standards are in question, such as Bluetooth, 3G, LTE, WLAN, IR and others. Mobile handsets are usually equipped anyway with a variety of such interfaces, so that for the sensor a suitable selection can be made.

The inventive method can be configured in a similar manner by further features and shows similar advantages. Such further features are exemplary, but not exhaustive, in which subclaims following the independent claims are described.

The invention will be explained below with regard to further advantages and features with reference to the accompanying drawings with reference to embodiments. The figures of the drawing show in:

1 an overview of a sensor and a hand-held device, by means of which the sensor is aligned;

2 an example of one on the handset according to 1 displayed superimposed image of the field of view of the sensor and a target display of the light spot; and

3 and 4 other examples of overlay images displayed on the handset.

1 shows an overview of a sensor 10 and a hand-held device 100 with whose help the sensor 10 is aligned. In the illustrated sensor 10 it is a laser measuring device in the form of a single-beam distance sensor according to the time of flight principle. However, this type of sensor is to be understood purely as an example because the alignment according to the invention does not depend on the specific sensor principle or structure. Instead of the distance sensor so also other sensors come into question, some of which were mentioned a few, in particular distance sensor according to the triangulation principle, light barriers, data light barriers, laser scanners or light grids.

The sensor 10 sends by means of a light source 12 , For example, a laser diode or an LED, via a transmission optics 14 a transmitted light beam 16 into a surveillance area 18 out. In many cases, the sensor uses 10 a wavelength invisible to the human eye, so that when hitting an object 20 resulting light spot 22 with the naked eye is not recognizable. That on the object 20 Remitted light falls on a receiver optics 24 on a light receiver 26 , The offset between the optical axis of the light transmitter appears 12 and the light receiver 26 Due to the presentation, it is relatively large, but hardly plays a role in the real application. In addition, other known arrangement principles of light emitter 12 and light receiver 26 conceivable, such as a Autokollimationsanordnung with a splitter mirror.

The sensor further comprises a control and evaluation unit 28 , which with the light transmitter 12 and the light receiver 26 connected is. One from the light receiver 26 generated received signal is the evaluation unit 28 supplied therefrom in a conventional manner by determining the light transit time between emitting the transmitted light beam 16 and registering the remitted light the distance of the object 20 certainly. Other evaluations are possible, such as the generation of a binary object detection signal. The result of the evaluation represents the evaluation unit 28 at an exit 30 ready.

The described sensor function is based on the fact that the sensor 10 to the desired surveillance area 18 is aligned. For example, the transmitted light beam 16 hit a reflector, not shown, as long as no object in the surveillance area 18 is available.

To aid alignment, the sensor gives 10 by means of the evaluation unit 28 on a preferably wireless interface 32 a view area image of the surveillance area 18 out. This field of view image is of the trained as an image sensor light receiver 26 generated. Alternatively, the sensor has 10 via two separate light receivers, namely the image sensor for recording the field of view image and the light receiver for the object detection.

Further information is provided by the evaluation unit 28 a position of the light spot within the field of view image at the interface 32 out. For this purpose, the evaluation unit has a light spot position determination 34 on. In the simplest case, this is just a memory for one of the type of sensor 10 dependent light spot position, which also for the individual sensor 10 Is accepted. Alternatively, the light spot position is stored as part of the production or final inspection in a teach-in process, or it is determined by image processing from the field of view image. In addition to the position, the spot position determination can also determine and output its shape or size.

The handset 100 , For example, a smartphone, includes two visible buttons as an example visible components 102 and an ad 104 , A processor 106 evaluates the inputs on the control buttons 102 and generates the necessary data for the display 104 , Via a wireless interface 108 can change the field of view image and the light spot position of the sensor 10 be received. Optional are on the handset 100 own sensors provided. This includes a dedicated camera 110 as well as a position, motion and / or position sensor 112 , Elements of the handset 100 that are inside or on the back are indicated by dashed lines.

A fusion unit 114 adds information from the sensor 10 and in some embodiments additionally information of the own sensors 110 . 112 to an overlay image of the field of view of the sensor 10 with the object 20 and a destination display 116 together. This overlay image will appear on the display 104 shown to the user the current orientation based on the position of the light spot 22 of the sensor 10 to visualize.

Depending on the equipment of the handheld device 100 , the quality of the sensor 10 Given visual field image and accuracy of the light spot position, various embodiments are conceivable, which in the following with reference to on the display 104 illustrated exemplary overlay images are explained. It is called an object 20 a reflector is shown in each case, this being only for illustration and the invention is in no way intended to limit this example.

2 shows the display for an embodiment in which the handset 100 only as an indication of the sensor 10 is used. The light spot position determination 34 determines a confidence interval for the light spot position via a tolerance model which estimates the internal tolerances of the optical axes between light emitters 12 and light receiver 26 as well as, if for the light receiver 26 two components are provided, also taken into account between the image sensor and the light receiver. The tolerance model can also be distance-corrected. The sensor 10 therefore knows the light spot position independent of a measurement. The fusion unit overlaps the field of view image of the sensor 10 a target display, here in the form of a circle, which visualizes the position of the light spot and the confidence zone.

3 shows the display for a similar embodiment, but in which the light spot position is more accurately known and therefore highlighted with a more precise target display. Instead of a tolerance model, namely, the position, shape and size of the light spot 22 also be adjusted during the adjustment during production or in a final test with the field of view image. For example, the sensor 10 calibrated so that the light spot at a certain distance, about 20 m, centered in the field of view image. As a further alternative, the light spot is determined by the spot position determination 34 localized by image processing in the field of view image. To do this, the field of view image must be the spectrum of the light spot 22 recognize at all and also have sufficient contrast. The spot position is more accurately known in all these cases than only in a confidence interval predicted by a tolerance model.

4 shows the display for an improved overlay image. This is not directly the field of view image of the sensor 10 displayed, but the qualitatively much higher quality and better resolved viewing area image of your own camera 110 of the handset 100 , The fusion unit 114 determines by correlation a fusion image from the field of view image of the sensor 10 and your own field of vision, so the two pictures fit together. This determines a transformation rule, by which the of the sensor 10 supplied light spot position with respect to its field view image in a light spot position with respect to the field view image of the handset 100 can be converted. 4 shows with a dashed rectangle 118 the border thus found the Sichtbereichsbildes of the sensor 10 within the field of vision of the handset 100 , In order for the correlation to succeed, the field of view images must have sufficiently overlapping image areas and, if possible, an at least reasonably consistent viewing angle.

Based on the transformation rule, the fusion unit hides 114 the destination display 116 in the correct position in the own viewing area image of the hand-held device 114 one. In addition, image information can be taken from the field of view of the sensor 10 be taken in the overlay image, for example, the light spot 22 myself, because the camera 110 of the handset 100 whose wavelength may not be sensitive at all.

To prepare the correlation and the bandwidth requirements of the transmission at the interfaces 32 . 108 To reduce already preprocessing can be made on the sensor side. For this example, the field of view image of the sensor 10 from the evaluation unit 28 compressed, binarized, an edge or other image feature detection performed or areas of interest defined. The correlation is then limited to this information essential for the determination of the transformation rule.

The correlation of the two images represents one possibility, the different perspectives of sensor 10 and handset 100 to compensate. Alternatively or additionally, further information of the handset 100 be used to obtain an even more robust system.

This will be for an initial referencing handset 100 and sensor 10 placed in a defined position to each other, for example, the handset 100 to the sensor 10 created. This position is defined as zero position or learned in the handset. A subsequent movement of the handset 100 is tracked via integration of the respective change in position and / or acceleration with the position and / or acceleration sensor 110 be determined. The handset remains in this way, the current position with respect to the sensor 10 and the changed perspective can be taken into account when creating the overlay image. As an alternative to such a relative position determination becomes an absolute position sensor 110 used, based for example on systems such as GPS or Galileo. Based on such a position compared to the initial position is also the relative position and orientation of the handset 100 opposite the sensor 10 determined, and the changed perspective can be incorporated into the merger of the overlay image.

It is pleasant for the user, if the overlay image is constantly updated, so a kind of live film is displayed as possible without latency. It is also conceivable, however, a snapshot operation without computationally intensive real-time processing. This also has advantages, because it reduces the need for computing power, prevents jerking and blurring, gives the user the opportunity to place the handset in front of him and have both hands free for a reorientation of the sensor and prevents bandwidth bottlenecks at the interfaces 32 . 108 ,

By decoupling of display 104 and sensor 10 is also possible in principle, an inaccessible sensor 10 align by using the handset actuators of the sensor 10 or a bracket to be controlled to change its orientation.

Claims (15)

Mobile handset ( 100 ) for aligning a sensor ( 10 ) with an advertisement ( 104 ) for displaying an overlay image representing the current orientation, which is displayed on a viewing area image of a surveillance area ( 18 ) of the sensor ( 10 ) and a destination display ( 116 ) a position of one of the sensor ( 10 ) generated light spot ( 22 ), characterized in that the handset ( 100 ) an interface ( 108 ) from which to the handset ( 100 ) external sensor ( 10 ) the field of view image as well as information about the position of the light spot ( 22 ) in the field of view image, and a fusion unit ( 114 ) for generating the overlay image. Handset ( 100 ) according to claim 1, which has its own image sensor ( 110 ) for receiving an additional field of view image of the surveillance area ( 18 ) and wherein the fusion unit ( 114 ) is adapted to generate the overlay image also based on the additional view area image. Handset ( 100 ) according to claim 2, wherein the fusion unit ( 114 ) is adapted to the field of vision image of the sensor ( 10 ) and the additional viewing area image of the own image sensor ( 110 ) to match the position of the light spot ( 22 ) in the additional field of view image. Handset ( 100 ) according to one of the preceding claims, which is a position and / or acceleration sensor ( 112 ) and wherein the fusion unit ( 114 ) is adapted to an orientation and / or position change of the handset ( 100 ) and when merging the target display ( 116 ) into the overlay image. Handset ( 100 ) according to one of the preceding claims, comprising a position sensor ( 112 ) and wherein the fusion unit ( 114 ) is adapted to receive one from the position sensor ( 112 ) certain position of the handset ( 100 ) when merging the target display ( 116 ) into the overlay image. Handset ( 100 ) according to one of the preceding claims, wherein the fusion unit ( 114 ) is adapted to the overlay image on the perspective of the sensor ( 10 ) to recalculate. Optoelectronic sensor ( 10 ) for capturing objects ( 20 ) in a surveillance area ( 18 ), with a light transmitter ( 12 ) for emitting a light signal ( 16 ), which when striking an object ( 20 ) a light spot ( 22 ), with an image sensor ( 26 ) for capturing a field of view image of at least part of the surveillance area ( 18 ) and with a light spot position determination unit ( 34 ) for determining a position of the light spot ( 22 ) in the field of view image, characterized in that the sensor ( 10 ) an interface ( 32 ) for outputting the field of view image and the position of the light spot ( 22 ) having. Sensor ( 10 ) according to claim 7, wherein the light spot position determination unit ( 34 ) has a memory for storing therein the position of the light spot ( 22 ) in advance. Sensor ( 10 ) according to claim 6 or 7, wherein the light spot position determination unit ( 34 ) is adapted to the position of the light spot ( 22 ) by image processing. Sensor ( 10 ) according to one of claims 7 to 9, which comprises a preprocessing unit ( 28 ) to display the viewport image for output at the interface ( 32 ). Arrangement for aligning an optoelectronic sensor ( 10 ) according to one of claims 7 to 10 with a mobile hand-held device ( 100 ) according to one of claims 1 to 6. Arrangement according to claim 11, wherein the interface ( 32 ) of the sensor ( 10 ) and the interface ( 108 ) of the handheld device ( 100 ) wireless interfaces ( 32 . 108 ) are. Method for aligning a sensor ( 10 ) for capturing objects ( 20 ) in a surveillance area ( 18 ), in which on a handset ( 100 ) an overlay image representing the current orientation is displayed which comprises at least part of the surveillance area ( 18 ) and a destination display ( 116 ) a position of one of the sensor ( 10 ) generated light spot ( 22 ), characterized in that the sensor ( 10 ) a viewing area image of the surveillance area ( 18 ), the position of the light spot ( 22 ) in the field of view image and the field of view image and the position to the handset ( 100 ) and that the handset ( 100 ) generates and displays the sub-picture based on the field of view image and the position. The method of claim 13, wherein the handset ( 100 ) an additional field of view image of the surveillance area ( 18 ), the field of view image of the sensor ( 10 ) and match the additional field of view image to the position of the light spot ( 22 ) in the additional field of view image, and as an overlay image the additional field of view image superimposed with the target display ( 116 ). A method according to claim 13 or 14, wherein the handset ( 100 ) at a first time in a known position and orientation to the sensor ( 10 ) and based on data from a position, acceleration and / or position sensor ( 112 ) the position and orientation with respect to the sensor ( 10 ) determined at a second time by the movement of the handset ( 100 ) between the first time and the second time caused perspective changes in the merger of the target display ( 116 ) into the overlay image.

Images