DE102022107919A1 - System for automatically determining the score of several darts stuck in a dartboard at the same time - Google Patents

Abstract

2.1 In the sport of darts, the scores of several darts (2) thrown at a round target (1) are determined. A disk (1) consists of several segments (3) arranged circularly and depending on the angle, to which variable values (4) are assigned and which are separated by wires (5). In particular, for the steel darts variant, the automatic determination of the score is usually only possible using non-contact sensors such as cameras (6) or light sensors. However, previous systems for this do not have any robustness against displacements and rotations of the disc relative to the sensors (cameras) and their wires (5) due to mechanical deformation and production inaccuracies before and during game operation and require their own calibration.2.2 The invention solves this problem through use at least one camera (6) and a calculation unit (7), which algorithmically compares the scores with the images recorded with the at least one camera from different viewing angles (8) using suitable rule-based image processing algorithms (9) and/or using artificial intelligence methods (10) can be calculated through training and subsequent application of deep learning (e.g. neural networks). Calibration of the system is therefore no longer necessary.2.3 This means that several arrows (2) stuck in a real steel darts board at the same time can be assigned to the segments (3) by visually separating the arrowheads (11) from the wires (5). Edges of the segments are possible using the calculation methods mentioned and thus an automated determination of the scores of steel darts.

Description

1. Technical field of the invention

With reference to the drawings 1 and 2 , the invention relates to a system and a method for the automatic, non-contact determination of the score (sometimes also called “score”) of several darts 20 (also called darts or arrows) stuck simultaneously in a flat dartboard 10 (also referred to as a dartboard, board or target). designated) with tips 21 preferably made of metal. Such a disc usually does not have any contact surfaces with pressure sensors for direct electronic determination of the score (as in so-called electronic darts, or E-darts for short), but due to the self-calibration capability of the invention described here, its position in the plane of the disc and can also be slightly moved against its original positioning at any time perpendicular to the plane. The invention includes at least one processor (also referred to as a computing unit) on which software is executed which, using suitable classic rule-based algorithms, but also artificial intelligence algorithms, is still able to produce a unique result even when the disk is shifted and rotated Assign the arrowheads to the respective segments (sometimes called fields) of the dartboard. Geometric errors in the wires 16 separating the segments and fields resulting from the manufacture and/or use of the dartboard 10 can also be compensated for using the algorithms of the invention, so that even in this case a correct calculation of the points of the arrows in the dartboard is still possible is.

In particular, the system and the method of the invention can therefore also be transferred to other sports in a similar manner in which a pointed or round projectile (e.g. an arrow or a ball) hits a target without electronic contacts (e.g. a target) and then gets stuck there remains. Examples of this are archery or other shooting sports. The impact points of bullets and their assignment to hit or point segments could be determined as well as the assignment of arrows shot with a bow, which are usually stuck in a round disc with concentric ring segments.

In the precision sport of darts, the scores of several darts 20 thrown at a round, flat target 10 are determined. In classic darts, these darts usually have metallic tips 21 (which is why, like the darts category, they are called “steel darts”) and the discs 10 that hold them are usually made of sisal fibers. In particular, such discs generally do not have any contact surfaces connected to electronics with, for example, mechanical pressure sensors and hole grids for the arrowheads, as in e-darts, in order to be able to determine the score using arrows with plastic tips directly via these contacts. Instead, in classic darts it has usually been necessary to determine the scores by looking at them yourself and either writing them down manually or typing them into a device to determine the score and/or remaining points.

A disc of 10 in classic darts (as is usually the case with e-darts) is made according to this 1 in its center consists of two segments: the “Bull's Eye” segment 11, which is usually marked in red, as an inner circle, which is surrounded by the “Single Bull” segment 12, which is usually marked in green, as the outer ring segment. This single bull segment is adjoined by 20 segments with values between “1” and “20”, which radiate outwards in a radial and angle-dependent manner. Each of these 20 areas is interrupted by an inner narrow ring, the triple field 14, which triples the value of an arrow in the respective segment, and a double field 15, which doubles the value of an arrow in the respective segment. All remaining fields 13 of the segment provide simple values per arrow. The resulting ring segment fields are arranged concentrically around the central “Bull's Eye” segment. The fields 14 and 15 alternately have the colors green and red, while the fields 13 alternately have the colors white and black. Outside the double fields, the value or number of points per arrow is “0”. All segments or fields are each made of wires 16 (in 1 shown as black contours) to separate them from each other. In this context, the resulting network made of wire to delimit the segments and fields from one another is also referred to as the “spider” of a dart board. Im in 1 In the case shown, the assigned number for the radial value range would be “1”, for example. Consequently, fields 13 would mean one point each, segment 15 would mean two times one, i.e. two points and segment 14 would mean three points. Thus, with a radial value range of a segment of value “20”, a maximum of “20” times three, i.e. “60” points can be achieved for a triple field in a specific segment field on the target.

A single steel dart or arrow 20 consists, as in 2 shown, usually consisting of four essential components: the arrowhead 21 made of metal, which remains stuck in the board; the “barrel” 22, which acts as a handle on which the arrows to be thrown are grasped with the fingers; dem “Shaft” 23 as an extension and connection to which the “Flights” 24 are attached to stabilize the trajectory of the dart. In steel darts, the score (unlike in e-darts) is only determined after all darts have been thrown. The score is determined for all arrows stuck in the board at the same time, and the arrows must remain in the board until the total score is announced or determined, until they are pulled out by the darts player concerned. This means that the scores of arrows that have fallen out of the board until the score is determined or the arrows are pulled out do not count.

In e-darts, however, the score is determined individually after each throw into the target (field on the dartboard with a score other than “0”). Any arrows that may be knocked out of the dartboard by a subsequent arrow remain in their scoring. Likewise, in e-darts, arrows that hit an arrow that was already stuck in the dart and thereby cause the arrow to touch the contact surface again are counted, even if the new arrow does not touch the board itself or gets stuck in the board. In the eyes of many players, steel darts provide a cleaner score determination than e-darts because they do not allow such “manipulations”. The points have to be earned “harder” with steel darts, especially since arrows that were previously stuck in the board can be knocked out of the board by subsequent ones and are therefore not included in the scoring after several arrows have been thrown. Arrows can also bounce off the spider's wire and then don't get stuck in the board.

However, the simultaneous determination of the scores of several arrows stuck in a board at the same time without electronic connections and pressure sensors on the board (as with steel darts) can usually only be done with the help of non-contact sensors such as light sensors or cameras.

For one or more steel darts players, there is a so-called caller who determines the scores by looking at the target and then verbally announces them, as well as at least one scribe, to determine the scores achieved from, for example, three darts each in professional tournaments or amateur tournaments , who repeatedly subtracts these scores from the total number of points to be achieved, preferably 501 points.

The aim of the invention is not to make the callers and/or writers superfluous in such tournaments. Instead, it is preferably about offering, for example, a professional player or an ambitious/interested hobby player the possibility of automatically recording their throwing performance (e.g. average number of points per three darts thrown, also called “3-dart average”) during a training session, without that he has to interrupt his throwing series by manually writing down the scores. A good steel darts player will still have to calculate in advance for each throw which fields he has to target next in order to achieve a suitable remaining score for the following darts. Although the invention described here can optionally relieve him of this task, the remaining points should preferably only be displayed updated after, for example, three darts thrown, as a scribe would otherwise do. The system could therefore also be used to control scores/results in addition to manually determining and writing down scores, for example in tournaments. In case of doubt, the previous remaining scores could be reconstructed using the system.

2. State of the art and shortcomings of previous systems and procedures

However, previous systems and methods from the field of steel darts for the problems described above do not have any robustness against minor shifts in the position of the disc in the millimeter range or in the low centimeter range due to purely two-dimensional position determination in the plane. With reference to drawing 3 This applies to the arrows 20 both in the of the spider due to mechanical deformation as well as inaccuracies in the production of the discs cannot be compensated for with previous systems or only with great difficulty, so that incorrect results can easily occur in the determination of individual points, especially if arrowheads are close to the wires of the spider and thus stuck to the edges of the field.

The invention DE 42 07 497 A1 determines the point of impact on the target optically by arranging a light source (transmitter) on one side and a sensor (receiver) on the opposite side in two mutually perpendicular directions outside the target. As soon as an arrowhead interrupts the light beam, an impulse is triggered. The light source and sensor are moved synchronously with each other so that scanning can be carried out in the X and Y directions to determine the XY coordinate of the arrowhead. You then know the point of impact on the plane. But you don't know the exact positions of the darts disc in relation to the measuring device (light sources and sensors). If the wires of the spider are slightly distorted, i.e. have minimal deviations in the small millimeter range (and even less) compared to their optimal shape, it is no longer possible to correctly assign the arrowheads to a segment of the disc.

There are other similar inventions that also use multiple light sources (transmitters) and sensors (receivers) to measure and determine the impact positions of the arrows EP 0 752 089 B1 and EP 0 182 397 A1. The problems of the latter invention are in turn caused by the invention WO 98/31979 explained. However, also used WO 98/31979 again a stationary light source with a band-shaped luminous surface surrounding the target, together with a sensor device consisting of several angle sensors that can determine the point of impact of an arrow by means of triangulation. Here, too, it is not discussed how slight shifts of the disk and thus its segments and fields against the light source and its sensors should be dealt with in order to still be able to determine a correct score.

Overall, systems made up of light source sensor elements for determining the position of arrowheads in discs that are slightly rotated and shifted against the structure and in which the arrows, i.e. whose tips are usually stuck at an angle, appear to be complicated and prone to errors.

More modern today are systems consisting of at least one digital camera (CCD or CMOS) as sensors for taking photographic images to determine the positions of the arrows. An example of this is invention US 2006/0066053 A1 , which uses at least two digital cameras to determine the location of the arrows. However, the board used is not a traditional disc with color patterns (black, white, green, red), but a colorless sisal fiber board onto which game images (a kind of pattern) are projected using a monitor (that can also be the pattern of a dart board) which is aimed at and in relation to which the arrow position is determined. This means that this disc has no physical wires and is therefore unsuitable for real steel darts with a real, physical segmentation. Robustness against rotations around the Z-axis like a real disk does not have to be guaranteed or treated by the software due to the pattern projection. The description of the invention also does not address how the software should be able to determine the score of a concentric segment or field on a real dart board.

At the invention WO 98/31979 the invention is based US 7,175,182 B2 , which uses a number of digital imaging elements such as CCD or CMOS sensors instead of angle sensors. However, according to the description, the algorithm for determining the score (“Image Processor”) from the recorded sensor images essentially only compares two images with each other (position of single arrow in the old image versus position of single arrow in the new image) and therefore does not provide a solution for the actual problem of determining several arrow positions at the same time, and whether or how the “image processor” is robust against shifts of the target. It is also claimed that it should be possible to capture a score with a single image-capturing sensor (i.e. a single camera). This will not always be the case due to the possibility of the arrowheads being obscured for three arrows stuck in the board at the same time in a single camera image.

3. Solving the problems of the current state of the art using the invention

It is therefore the main task of the invention to solve the problem described based on the current state of the art in determining the scores of several steel darts stuck simultaneously in a board made of sisal fibers, which is primarily due to the lack of robustness against slight shifts and rotations of the disc, using appropriate software in conjunction with a system of digital cameras and a computing unit with memory, to be solved algorithmically.

To solve this main task, the positions and their assignment to corresponding score fields for several arrows stuck simultaneously in a traditional steel darts board are determined using the images recorded with at least one digital camera from different angles and fields of view using algorithms for image processing via a computing unit with memory determined. These algorithms usually consist of several calculation steps that are carried out one after the other and/or repeatedly in possibly different combinations and orders. On the one hand, this can be done on a rule-based basis using classic methods, preferably for segmenting image areas based on edges, which are given here by the tips of the arrows and the boundaries of the score fields in the form of wires defined by the spider, and then, for example, by calculating the Overlap or intersection of several previously segmented areas. On the other hand, artificial intelligence methods can also be used Find solutions that preferably include previously trained neural networks assigned to deep learning and that are also able to segment geometric shapes and color areas from each other or to recognize patterns in order to determine the scores of arrowheads on the board.

The sequence of an individual work cycle of the system with simultaneous application of the evaluation procedure is as follows:

For example, after three darts have been thrown, there will be zero, one, two or three darts stuck in the dartboard. As soon as this is the case, the player triggers the digital camera system, consisting of at least one camera, to capture images from different angles and fields of view. This triggering can be done, for example, by pressing a button on the system/device itself near the dart board or by a contact surface attached to the floor near the launch line, e.g. by a footstep.

The cameras of the system are preferably mounted at approximately equal distances from one another to the side and below/above the pane. For example, with three cameras, the angle in the X-Y plane between the individual cameras would be 120 degrees, or with four cameras, the angle between the cameras would be 90 degrees.

It is advantageous to place the cameras diagonally at the bottom next to the board and not to place a camera in the middle below the board, since if the arrows fall down or jump out, they will preferably fall centrally forward and thereby damage this camera. A corresponding translucent physical protection of the cameras is still provided for the system described. With this protection, care must be taken to ensure that it does not lead to any significant distortion of the recorded camera images. In addition, placing a camera centrally above near the target is dangerous when aiming at the top double field of the target, as the camera can easily be hit directly by an arrow, for example from a less trained player, due to a “slip”. In principle, for example, three cameras do not have to be arranged symmetrically in height around the board. However, it is important to roughly adhere to the 120 degree angle between the cameras in the X-Y plane in this case.

Furthermore, the cameras should preferably be placed in the Z direction at the same distance just in front of the board (a few centimeters), but so far in front of the board that there is still good visual coverage (angle of view and field of view, or FOV for short). ) of the disk in the surface (X-Y plane) is given on the two-dimensional images. Not every angle of the pane has to be completely captured by every camera, but this should preferably be the case (this is easily possible by positioning the cameras appropriately in relation to the pane). However, every spot on the pane must be recorded by at least one camera in the overall camera system so that each score can be calculated.

In order to record a large number of images with a single camera, a camera attached to a circular support (also referred to as a rail) can also be provided for the invention. This ring rail with camera preferably has a larger diameter than the dartboard and is attached slightly in front of the target in the Z direction (e.g. on the ceiling of a room or on rods perpendicular to the target above and/or to the side and/or below the target. The camera is aligned towards the center with a view towards the center of the disk. A motor ensures that the camera rotates along this rail around the disk. After a definable distance, which is preferably constant, the camera stops, taking at least one image at a time to record. The routes are preferably chosen so that the camera stops and records a number of times divisible by 360 degrees during a full revolution before it reaches its starting point again. An example would be six stops, with an angle of 60 degrees from the camera , based on the ring, would be painted over. The advantage of a camera attached to a circular rail would be that only one camera would be required to generate a large number of views and thus images for evaluation around the pane. This would result in significant cost savings. The camera can also be placed to the side in a safe area between the arrow throwing phases, so that hitting the camera with an arrow can be practically ruled out. However, it is important to ensure that the camera is securely attached to the ring and that the distances covered are preferably equidistant. It is also conceivable to reset the starting point to a definable starting point before a measurement cycle, e.g. after each series of thrown arrows, in order to compensate for mechanical deviations of 360 degrees after the camera rotates around the ring.

The cameras described above are so-called mono cameras that record images with color values for each image point (also called pixel). Your resolution should preferably be at least Full HD, meaning 1920 by 1080 pixels. The format or the aspect ratios of the X and Y axes of the cameras can also be square, for example, so that, for example, a resolution of 1920 by 1920 pixels is possible. However, so-called stereo camera systems can also be used, which allow a three-dimensional evaluation of the arrow positions. However, this may preferably require a different positioning of the camera system. The description below is therefore limited to cases with mono cameras.

The 1 to n (individual) images (n a natural number) recorded by one or more cameras from preferably n different viewing angles of the pane are temporarily stored by a storage unit. This storage unit can not only be an internal storage medium of the system, but also, for example, an external storage medium such as a USB stick or an SD card in order to permanently save a larger number of images, for example for analysis or manual To enable subsequent correction of the scores determined, if appropriate. Otherwise, the internal memory is chosen to be at least large enough to be able to store at least the above up to n images at the same time for the subsequent image processing algorithms, but at an absolute minimum one image in full resolution that can be further processed. In addition, additional internal memory (ROM, RAM) is provided in an amount appropriate for the application in order to store the code, the intermediate results of the individual calculation steps, the determined results (e.g. remaining points, etc.) for display on the display, and so on ( to be able to temporarily store.

The buffered image data is passed on from the memory to a computing unit with a processor for calculating the scores using the algorithms executed on the processor, for example, or is requested by the computing unit. This computing unit can, for example, consist of one or more CPUs or GPUs or combinations thereof or similar components suitable for calculating results using algorithms.

A variety of possible algorithms with combinations and variants of individual calculation steps are conceivable for determining the scores. Therefore, only the calculation steps that are essential for determining the score of several arrows stuck in a board are described here. The algorithms can differ from each other in the order in which the individual calculation steps described are applied. Individual calculation steps can also be omitted or replaced and/or supplemented by other steps not explicitly mentioned or described here.

First, a rule-based algorithm (which here means: an algorithm that can, but does not necessarily have to, do without explicit training) will be explained, which allows the arrowheads to be clearly assigned to their respective fields.

The n number of given images are loaded, initially at least one of the images. This image is two-dimensional and has an X and a Y coordinate. Due to the slant of the camera position to the disk and the arrows in it, this X-Y coordinate system of the images has little or nothing in common with the X-Y coordinates of the disk. However, if the camera has a suitably large field of view, at least a large part of the pane can be recognized for each recorded image. As described above, all the images together just need to be able to depict the entire disk. Preferably, each area of the disk should be captured by at least two camera images from very different angles and fields of view in order to be able to resolve any obscurations of arrows and to clearly assign the corresponding arrows to the appropriate segment fields.

Referring to drawing 2 The parts of a dart 20 that are irrelevant for the evaluation (i.e. barrel 22, shaft 23 and flight 24) are separated from the tips 21 that are exclusively relevant for the score, for example using pixel-oriented, edge-based, region-oriented or model-based segmentation (see e.g. book by author: Bernd Jähne; title: Digital image processing and image acquisition; publisher: Springer Vieweg, edition: 7th edition, place of publication: Berlin, Heidelberg, year of publication: 2012, pages 541 to 554) and/or methods of Fourier transformations (see also Book by Bernd Jähne, ibid., Chapter 4, pages 136 ff.) and/or similar methods and combinations thereof based on this.

The next step of the rule-based algorithm checks whether one or more arrowheads in one of the images are within or near one of the ring segments in the middle of the board. Whether an arrow lies within a segment can be determined, for example, by whether there is an overlap between a segment clearly defined by the edges (such as the single bull) and the tip of the arrow in one or more images (from multiple viewing angles). For this purpose, a ring-shaped element such as the single bull or the bull's eye is preferably approximated by a polygon (polygon) with the highest possible number of corners as edges. Afterwards, using a polygon clipping algorithm as an intermediate step, the cut or overlap of the tip of the dart, which is also approximated as a polygon, can be determined (that is: is there a cut/overlap or not). Since the polygons can be non-convex under certain circumstances (such as all segment fields running radially outwards or the single bull), a suitable polygon clipping algorithm is, for example, that of Greiner and Hormann (authors: G. Greiner and K. Hormann ; Title of the article: “Efficient clipping of arbitrary polygons.”; Journal: ACM Transactions on Graphics; Year of publication: 1998, Issue number: Vol. 17, No. 2; Pages 71 to 83), which describes the cutting surfaces of at least two such non-convex Can calculate polygons. It is preferred to first try to identify the arrowhead and the fields closest to this arrowhead (this also includes the field in which the tip is actually located) on the target by scanning the image for the arrowheads and to limit the image areas around the arrowheads accordingly.

If, for example, the result in the previous step was that the arrow cannot be reliably assigned to one of the inner ring segments, you move from the image area that contains the center of the dartboard radially outwards in the angular range in which the arrow is approximately located. You then assign the arrow(s) to one of the segments running radially outwards, again preferably by segmenting the edges of the wires, forming polygons and then polygon clipping the arrowhead and segment or field on the board. The values between “1” and “20” assigned to each of the segments are determined at the edge of the dartboard using image recognition. A neural network from the field of artificial intelligence can also be used for this purpose. If you have delimited the edges/wires of the possible radial segments from each other, and if an arrowhead is close to the edges of a segment, for example, it makes sense to take two or more segments or fields into account in the calculation steps with polygon clipping as possible score fields.

At the same time, it is also important to count the number of transitions when scanning the wires radially outwards along the radial lines and/or to determine the colors of the fields. For this purpose, the memory unit can also provide a fixed memory area that stores the colors of the fields (black, white, green, red) in combination with the values of the respective segment and can make these available to the algorithm upon request. In this way, it can be determined whether a found polygon structure of a field within a segment is a single, double or triple field with single, double or triple points. This information can also be taken into account during polygon clipping.

By appropriately combining the cuts of arrowheads, preferably obtained by means of segmentation and polygon clipping, with the edges of the individual score fields of the target from a large number of images and thus views of the dartboard, it is possible to calculate the scores of several arrows stuck in the board at the same time in a classic rule-based manner determine.

If previous calculation steps are used appropriately, it may also be possible to further narrow down an individual arrowhead to an almost point-like or small-scale ring-shaped area where an arrowhead is stuck in the board. This approximate point formed in this way can, for example, be approximated by a small polygon or even simplified as the geometric center of this mini-polygon by a point with (X,Y) coordinates as a floating point number. Afterwards, for example, a sub-step of the polygon clipping algorithm from Greiner & Hormann can easily determine which point of the arrowhead lies within the edges of which score segment field (i.e. which polygon was previously determined by the segmentation) and return the corresponding score value of this arrow.

If several arrowheads overlap in an image from the camera system, either a segmentation of the overlapping arrowheads can be carried out or one can preferably look at an image that was taken with a camera from a different angle and thus try to clearly resolve the situation.

It may be necessary to combine different variants and combinations of individual image evaluations, depending on the viewing angle, field of view and number of cameras as well as evaluation algorithms for segmentation and polygon clipping algorithms. For example, from the perspective of one camera, no overlap of arrowhead with a certain segment field can be returned as a result from a certain angle, but from the remaining images with other angles it can. Here it depends on the viewing angle of the individual cameras, for which view an arrow must have an intersection with a segment field and for which it does not, in order to be able to be correctly assigned to a specific segment in the end. Probabilities can also be used in the algorithms, for example to indicate the certainty of the assignment of an arrowhead to a specific field (and its score) in an individual image and to express this using probabilities. The probabilities determined for a field in different individual images can then, for example, be combined and, for example, the field with the highest probability A correct assignment combined from several individual images and angles is then selected and used as the field with the correct score.

It should be noted that the arrowheads do not always have to be the color “silver” and shades of it. For example, they can also be black. However, as long as the edges, for example, continue to have the color “silver” or shades thereof, this should not be a problem for the algorithms, since, for example, black fields are always surrounded by silver edges, and the silver edges are intersected by the black tip of the arrow. This means that if a black dot overlaps an edge of a field, this is usually an arrowhead, as long as the segmentation of the remaining components of a dart (barrel, shaft, flight) has been carried out properly. This allows the score of each arrow to be determined again.

In order to check the validity of the described algorithm and its variants, it is necessary to fine-tune the influencing variables and calculation steps described above in conjunction with checking the correctness of the respective results in reality.

After the rule-based algorithm just described, the (possibly better) alternative is an algorithm based on artificial intelligence, which is able to calculate the scores, preferably using a neural network and especially preferably a so-called “convolutional neural network” (CNN). several arrows at the same time can be determined automatically after training with a sufficient amount of correct data.

A CNN consists of the components and properties described below (based on the book by author: A. Geron; title: Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools and Techniques for Building Intelligent Systems; Publisher: O' Reilly Verlag, edition: 1st edition, place of publication: Sebastopol (USA), year of publication: 2017, pages 355 to 380):

Starting from an input layer with several channels for the colors red, green and blue of an RGB camera image, a CNN further consists of a combination of several convolutional layers. The number of these convolutional layers can be very large, for example a hundred or more. Each of these convolutional layers consists, for example, of several so-called feature maps of the same size. Within each such feature map, all so-called neurons have the same parameters (these are “weights” and “bias term”), but different feature maps can have different parameters. A so-called receptive field of a neuron extends over the feature maps of all previous layers. A neuron of a feature map in a particular row and column in a convolutional layer is linked to a neuron in the previous convolutional layer across all feature maps in that previous convolutional layer.

The feature maps of the individual convolutional layers are created using specific filters, such as region proposal according to Girshick et al. (Authors: R. Girshick, J. Donahue, T. Darrell, J. Malik; article title: “Rich feature hierarchies for accurate object detection and semantic segmentation.”; in journal: 2014 IEEE Conference on Computer Vision and Pattern Recognition, Year of publication 2014, issue number: 2014-06, pages 580 to 587), created as square regions of the same resolution per page (X and Y). A so-called pooling layer is connected to some, several or all convolutional layers, which reduces the size of the input image in order, among other things, to reduce the number of parameters (to prevent so-called overfitting). However, the image reduction also causes the CNN to tolerate small shifts in the image (location invariance). This property is of crucial importance, for example, to compensate for slight shifts (translations) of the disk in the X-Y plane, but also perpendicular to it along the Z axis, in relation to the camera position. The same applies to rotations or tilting of the disc around the axes as well as displacements caused, for example, by mechanical deformation of the wires during play or faulty locations in the wires of the spider of a board that arise during production or during transport of the board.

Between the individual convolutional layers and pooling layers there are so-called activation functions such as the so-called ReLU function or the tangent hyperbolic function, which connect the inputs and weights with each other. Finally, one or more so-called fully connected layers follow, before the output layer follows.

During training, the output layer contains, for example, information about which arrows are within which segment field and thus which arrow or arrowhead corresponds to which score on the board. Also the score of the up to three arrows per picture and recording (that's three arrows thrown each; at the end of a leg, i.e. a dart game from "501" points down to "0" points, it is all arrows until the remaining number of points is reached “0”) can be saved in the output layer. But this information can also can be obtained from the up to three points of the arrows per shot.

The data used to train a CNN consists of previously recorded camera images from different angles, each of which contains the correct scores as a label. This means that these scores were previously read correctly by a person on the board and assigned to the images as a label. This makes it possible, for example, to map situations with three arrowheads that are close together, but which belong to more than one segment field and therefore have different points, and to pass them on to the system for training. Arrowheads with non-silver colors can also be included in such images for training the CNN, or even three arrowheads in the board, each with a different color for the individual tips. Spider wires with colors other than silver can also be included as images in the training data set.

The features for the CNN preferably include: the edges of the segments in the form of wires; the arrowheads, whose colors can vary; the colors of the segments or fields (preferably black, white, red, green); the numbers on the edge of the disc, which indicate the value of the segments running radially outwards. CNNs are able to detect multiple features at any point in the input, including the features described above that are contained in one or more images from the camera system after, for example, three arrows thrown.

During training (preferably of the “supervised learning” type), a so-called loss function is calculated from, for example, scores or the like, using so-called forward propagation on the training data set. With a variant of a so-called gradient descent method such as batch or stochastic gradient descent (or similar), learnable parameters such as kernels and weights are updated according to the loss value using so-called backpropagation.

After training, the output, i.e. the scores of the individual arrows or arrowheads in the board, can be calculated using the trained CNN. After training with the training set, for example, a so-called cross-validation set is first tested before the CNN and its weights are ready for use in real operation (i.e. real darts game).

Any optimized or trained variants, preferably of a CNN, but also of other neural networks or machine learning methods, which ensure the robustness of the score determination against shifts and rotations of the disk as well as errors that arise on the disk, are also possible and can be used within the scope of this Invention find application. This means that the invention explicitly includes these.

After determining the score of a so-called shot (preferably three darts thrown), the total number of points is deducted from the previously given remaining number of points and only the new remaining number of points is shown to the players on a corresponding display. This means that two displays are preferably necessary, i.e. one for each player. The display displays can additionally include, for example: the overall 3-dart average based on the entire game; the score achieved with the last recording; Games won, sets won, legs won for the respective player.

In addition to updating the remaining score, after the score has been determined from the last recording, an acoustic voice output of the total score achieved can be made via a loudspeaker belonging to the system, as is normally done by the caller in steel darts games. This is a standard feature in e-dart games today, but is probably not yet available for a steel darts application, or is rarely available, or has not yet been possible in a meaningful way.

As soon as a player's remaining score in the "Double Out" variant, for example, is "170" or less (except for so-called bogey remaining scores such as "159", which in the Double-Out variant are not set to "0" with three arrows ), it may be necessary to determine the remaining points after just one or two arrows. Depending on the remaining number of points, the system determines from a list in the memory how many arrows at least still have to be thrown in order to set the remaining number of points to “0”. Alternatively, the player can also start the score calculation manually. If the remaining score is zero, the relevant player is awarded the leg in question as having won and the leg counter in the memory is increased by one. As is usual in steel darts, depending on the mode, the player is awarded a set (German set) if he wins a certain number of legs and the match win is assigned to the player if he wins a defined number of sets. As soon as this is the case, a singles game is over.

The results (scores or 3-dart average, information about the overall winner, sets won, legs won, and the like) can be saved, for example, after each game or when the game is abandoned, for example on an internal one Hard drive of the system or externally on a USB pen for data storage. If there is enough memory, the images of the individual recordings can also be exported for analysis purposes and/or to validate the correctness of the results determined.

With the double-in variant, at the start of each leg, each player must first check whether each arrow thrown is in a double field or not. For this purpose, the measurement can either be triggered manually by the player after each arrow or, preferably, the camera system continuously monitors whether and when an arrowhead has moved somewhere into the camera's field of view or touched the board, i.e. is in the board, and then releases it measurement. A similar procedure is also conceivable for measuring the arrows towards the end of a leg in “double-out” mode: this means that every time a new arrow touches the target and/or appears in the image area of one of the cameras, camera images are recorded and evaluated in terms of points achieved. If the number of points is not enough or the player has fallen out (more points than allowed) or the player has not hit a double field at the end, the remaining number of points will be corrected according to the rules of Steeldarts and the player in question will continue with the next shot .

As soon as the score is set to "0" by hitting a double field and the system has recognized this by evaluating the camera images, i.e. calculated it, the leg is assigned to the player in question as having been won and the counter for the legs won is incremented by one.

4. Examples of embodiments with drawings

• 1: eine allgemeine Dartscheibe, die in mehrere ringförmige und radiale Segmente und Felder mit unterschiedlichen Punktzahlen unterteilt ist,
• 2: einen Dartpfeil aus mehreren Bestandteilen,
• 3: ein Koordinatensystem für eine Dartscheibe mit darin steckenden Pfeilen,
• 4: ein Kamerasystem, bestehend aus in der dargestellten Ausführungsform vier Monokameras, die ihre Bilddaten an einen Bildspeicher weiterreichen können und/oder die von einer Recheneinheit dazu aufgefordert werden können, Bilder aufzuzeichnen,
• 5: eine Schrägansicht, die beispielhaft die Abstände und Positionierungen der Kameras neben bzw. vor der Dartscheibe zeigen,
• 6.a und 6.b: eine einzelne Kamera, die entlang einer kreisförmigen Schiene in Richtung Scheibenmitte orientiert ist und die Bilder aus vielen unterschiedlichen Blickwinkeln aufnehmen kann, zum einen in 6.a in einer Frontansicht zur Scheibe, zum anderen in 6.b in einer Schrägansicht,
• 7: die technische Systemarchitektur für die Hardware des Systems, die im Wesentlichen als Bestandteile die mindestens eine Kamera, mindestens eine Speichereinheit für Bilddaten, Zwischen- und Endergebnisse der Berechnungen, sowie mindestens eine Recheneinheit und Displays zur Anzeige der (Zwischen- )Resultate umfassen,
• 8: eine beispielhafte Darstellung für ein mit einer Kamera aus einem typischen Blickwinkel aufgenommenes Einzelbild, auf dem in diesem Fall die gesamte Scheibe mit Pfeilen bzw. deren Spitzen sichtbar ist, so wie es der Software des Systems zur Berechnung der Punktzahlen bevorzugt zur Verfügung gestellt wird,
• 9.a, 9.b, 9.c, 9.d: beispielhafte Darstellungen für wesentliche Auswertungsschritte des Verfahrens mit bevorzugt regelbasierten Algorithmen, dargestellt anhand der gegebenen Bilddaten und möglicher Verarbeitungsschritte der Algorithmen,
• 10: ein beispielhaftes Ablaufdiagramm des Verfahrens mit den möglichen Schritten eines regelbasierten Algorithmus zur Ermittlung der Punktzahlen mehrerer Pfeile,
• 11: eine beispielhafte Darstellung für ein auf künstlicher Intelligenz basierendes Verfahren, hier eines Neuronales Netzes (genauer eines Convolutional Neural Networks), zur Ermittlung der Punktzahlen mehrerer Pfeile gleichzeitig.

The invention is further explained below by preferred exemplary embodiments and their drawings. The individual drawings represent:

• 1 : a general dartboard divided into several annular and radial segments and fields with different scores,
• 2 : a dart made of several components,
• 3 : a coordinate system for a dart board with arrows stuck in it,
• 4 : a camera system, consisting in the illustrated embodiment of four mono cameras, which can pass on their image data to an image memory and / or which can be requested by a computing unit to record images,
• 5 : an oblique view that shows an example of the distances and positioning of the cameras next to or in front of the dartboard,
• 6 .a and 6 .b: a single camera that is oriented along a circular rail towards the center of the pane and can take images from many different angles, on the one hand 6 .a in a front view of the window, on the other in 6 .b in an oblique view,
• 7 : the technical system architecture for the hardware of the system, which essentially includes as components the at least one camera, at least one storage unit for image data, intermediate and final results of the calculations, as well as at least one computing unit and displays for displaying the (intermediate) results,
• 8th : an exemplary representation of an individual image taken with a camera from a typical viewing angle, on which in this case the entire target with arrows or their tips is visible, as is preferably provided to the software of the system for calculating the scores,
• 9 .a, 9 .b, 9 .c, 9 .d: exemplary representations of essential evaluation steps of the method with preferably rule-based algorithms, shown using the given image data and possible processing steps of the algorithms,
• 10 : an exemplary flowchart of the method with the possible steps of a rule-based algorithm for determining the scores of several arrows,
• 11 : an exemplary representation of a method based on artificial intelligence, here a neural network (more precisely a convolutional neural network), for determining the scores of several arrows at the same time.

The drawing 1 The prior art shows a general dart board 10 as used as a target in steel darts. The representation is limited to 1 on the essential properties of a disc without presenting the details. In the middle, a dartboard consists of a segmented double field, the so-called bull's eye 11, with the ring-shaped "single bull" segment 12 adjoining the edge. Both are separated from the segments surrounding them by ring-shaped wires 16. Wires as boundaries between the segments and fields are in 1 represented by black lines 16. Radial lines, i.e. wires, radiate outwards from the outer edge of the single bull segment. This makes a total of 20 Segments are defined whose values are between “1” and “20”. In the 1 As an example, the segment is shown with a simple value of “1” (as “Value=1”). It is divided into two fields with a single value 13 and one field each with a triple value 14 and a double value 15. The subdivision of the segments running radially outwards is achieved by ring-shaped wires such as 17, so that for each radial segment the The four fields just described serve as further subdivisions for the scores or values. The environment outside the double fields 15 is not shown, but it is also part of a dart board (usually a circular environment around the double field area of a target). The value of this area is “0”, meaning arrows in this area have a score of “0”.

drawing 2 The prior art shows an example of the preferred structure of a steel dart 20. After a throw at the dartboard, usually only the front part 21 of the arrow 20 remains stuck in the board. This is made of metal and is called the arrowhead (tip for short) 21. This arrowhead is usually attached to the so-called barrel 22, where the arrows are gripped for throwing. Behind them are the Shaft 23 and the Flight 24. Only the arrowheads 21 are crucial for calculating the scores of arrows stuck in a dartboard. Nevertheless, the other parts 22, 23 and 24 must be segmented when scoring points and must not, for example, obscure the view of the arrowheads.

drawing 3 The prior art shows a possible coordinate system 30 for a dart board 10, the origin of which lies in the segment and is oriented from there in the direction of the player towards the throw line. In addition, three arrows 20 stuck in the board are shown. Parts of their front tips are no longer visible because these parts are stuck in the board.

drawing 4 shows a possible embodiment for a system of cameras 40, consisting of several (in the example four) individual cameras (e.g. CMOS or CCD mono cameras) 41, which are preferably ring-shaped equidistant from the center of the disk 10 with arrows 20 stuck therein, for example on an annular rod 44 are mounted. A combination of storage unit and computing unit 42 can record or receive, store and further process the image data from the individual cameras via data lines 43 or via remote connection (not shown). However, if necessary, in one embodiment of the invention, the storage and computing unit 42 can also give commands via the data lines 43 to the cameras to record one or more images and then receive them again via them. The individual cameras 41 can be mounted symmetrically around the pane in various ways and preferably at equal distances in a slightly positive z direction to the coordinate origin in order to achieve a better view of the entire pane if possible through each of the individual cameras (see also the following 5 ), and thus improve the evaluation of the images by the algorithms. The distances between the individual cameras 41 and the disk 10 should not be too large, especially in the Z direction, in order to still have a sufficiently large resolution for the arrowheads. However, it would also be conceivable to place the cameras far in front of the target in the Z direction (e.g. near or behind the player's throw line on the floor and/or the ceiling). In this embodiment, the system should preferably include at least two cameras, but optimally three or four, in order to be able to view or photograph the disc from several angles for a more precise evaluation with regard to the scores.

In addition to 4 shows the drawing 5 an oblique view of the Center point) of the dartboard 10 to be illustrated more clearly.

In 6 .a is an alternative, preferred embodiment to the system of several individual cameras 4 and 5 shown. A single digital camera 41 is sufficient here, which is mounted on a carrier 61 (or rail 61) and which is preferably circular and is attached slightly in front of the pane 10 in the positive Z direction (see also the following drawing 6 .b) . Likewise, this carrier 61 for the camera 41 is located outside the disk in the XY direction, so that there is a good field of view of each arrow 20 stuck in the disk. The data lines 43 can be in the situation of 6 .a can also be replaced by remote connections for transmitting the image data and commands, so that the camera 41 can move more freely around the disk 10 along the carrier 61. Storage and computing unit 42 can, among other things, also pass on commands to the camera as to which angle or length along the carrier it should stop again and record an image. The camera 41 can rotate clockwise or counterclockwise around the disk. In 6 .a this direction of rotation is clockwise 62. In addition, a stop of the movement is defined here ned points 63 are provided along the carrier. These stops preferably take place at equidistant intervals along the beam. These stop positions are marked by the cameras 63 with dashed edges, but it should be noted that these camera elements 63 are each a single camera 41, which rotates around the preferably circular carrier ring 61 in a clockwise direction 62 and only stops briefly at these positions 63 in order to record at least one image each time. The variety of viewing angles available using only a single camera make this embodiment of the invention particularly advantageous as it saves costs for multiple cameras while at the same time offering more flexibility than a system with permanently mounted cameras. Since the algorithms ensure sufficient robustness against displacements and rotations of the camera in relation to the disk, mechanical inaccuracies when mounting and moving the camera on the carrier are entirely acceptable and manageable.

drawing 6 .b illustrates with an oblique view of the system 6 .a again primarily the displacement or the distance 51 of the preferably ring-shaped carrier 61 with the camera 41 mounted thereon in relation to the Z axis of the be approximately the same. The dashed camera 64 should again represent a possible stop position of one camera 41 in the Position at 41, correspond).

drawing 7 shows a general representation of the technical architecture and hardware of the system. The overall system 70 consists of at least one digital camera 71, preferably CMOS or CCD cameras, which can exchange data with a storage unit 73 via lines 72 or via a remote connection. This at least one storage unit is in turn connected, for example via a data line 74, to the at least one computing unit 75, which can also give commands directly to the cameras in order to have new images recorded by them. This computing unit can, for example (if necessary) consist of combinations of CPUs and GPUs. The algorithms are essentially executed on the computing unit 75, that is, the determination of the scores takes place in the computing unit 75 using the camera images read from the storage unit 73 as input data. For example, another memory 77 for the total points, remaining points, statistics, etc. can be written and read via another data line 76. For the sake of simplicity, this memory can also be integrated in the memory unit 73 in a special embodiment of the invention, so that the memory 77 would then be omitted. Finally, the results can be forwarded via a further data line 78 to at least one display (display) 79 and via a further line 87 to a loudspeaker 88 for acoustic speech output (e.g. announcement of the points achieved or determined), so that, for example, current remaining points or Have the scores updated at any time on display 79.

The drawing 8th represents the typical appearance of a two-dimensional individual image 80 recorded with one of the cameras, from a viewing angle to the side of and slightly in front of the pane in the Z direction. In this case, all parts of the target are visible, as well as some arrows with tips, as provided to the software of the system for calculating the scores. The numbers between “1” and “20” indicate in 8th the values 81 of the segments that radiate outwards from the center, ie 1 to 20, are mounted on the edge of the disk 8th no numbers to reference parts of the figure! The positioning of the individual cameras can be chosen so that the entire pane, including the values 81 on the edge of the pane, fall into the image area. The values 81 are usually attached to a metallic ring 82, which can later be used in the algorithms for the invention to search for the values in the image area up to this metallic ring. This means that there is no need to search for the value numbers outside of this ring, thereby speeding up the evaluation process. As can be seen at point 83 of image 80, parts of a dart can obscure the view of the value digits. However, depending on whether the algorithms scan the digits and their segments in a clockwise or counterclockwise direction, it is easy to reconstruct which number the segment in question is. If you search clockwise, “18” would always be followed by “4”. After that would come “13”. Otherwise the number would be easily visible from a different angle or from a different camera image. So it is an advantage to record and analyze the arrows and the target from multiple angles.

In 9 .a shows how to extract areas 91, 92 and 93 from the camera image from a two-dimensional camera image 80 using the previously described digital image processing algorithms for segmentation. For this purpose, for example, the center of the target is first scanned for arrows. In case 91 shown is the one previously segmented from the rest of the arrow Arrowhead in or at least near the so-called “Bull's Eye” segment of the target. The immediate neighboring fields that border this bull's eye, which was initially roughly assumed to be the target segment of the arrow, are then included in the reduced section 91 of the overall image around the bull's eye. This means that the “Single Bull” segment will also be included in the further analysis for this arrow. The image section should preferably be rectangular, and can especially be square with X and Y axes. The other two arrows or their tips do not appear to be near the center of the target. For example, the segments with values from “1” to “20” are searched radially outwards. Finally, after crossing several wires towards the edge of the disc, you find the other two arrows in fields with double value. Here, too, an image section is formed: an image section 92 around the double "14" field and an image section 93 around the double "17" field. The adjacent segments and fields should be included in the image sections in order to to be able to clearly assign the arrowhead to a field in the following steps.

The drawing 9 .b shows how the image section 91 is created from a rule-based algorithm 9 .a can preferably be further processed. In this case, this is done using a polygon clipping algorithm that can handle the cuts or overlap of non-convex polygons. In 9 .b two approximately ring-shaped polygons have been created for the edges of the bull's eye and the single bull field by corresponding detection of the edges of the wires. Both polygons each consist of at least three vertices 94 (corners) and three edges 95 (lines that connect these vertices). Likewise, the tip of the arrow can be extracted and also approximated as a polygon with vertices 96 and edges 97. In the illustration, the recognized arrow consists, in simplified form, of a polygon with four corners. From this viewing angle, the absolute tip of the arrowhead now forms a cut 98 or a cut surface 98 or a cut polygon 98 or an overlap 98 (shown here as a checkerboard pattern) with the “Bull's Eye” segment 99. Image excerpts from camera images below obtained from different angles may produce different results. However, if from several different angles of camera images, for example for a defined number or quota of images, the result of the overlap calculation returns the same field as a result (possibly with a probability), then this field can be viewed as the most likely hit field and its score is saved as a result.

Then on 9 .b shows in the same way 9 .c the situation for the image section 92 9 .a. Here too, the wires of the fields (only two polygons of fields shown here; in reality, polygons can also be formed by the wires of other adjacent fields, which are included in the further analysis, ie section calculation) form the corners 94 and edges 95 of the polygons. The arrowhead is again a polygon with corners 96 and edges 97. The tip finally forms a cut 98 (checkerboard pattern) with the outermost field, i.e. a double field. However, we already know from previous steps that the single value here is “14”, so it is the double “14” field 99.

It would also be conceivable to search for or approximate the arrowhead of the arrow polygon and then simply check in which field polygon this point (the tip) is located. This can also be determined with a partial calculation step of the polygon clipping algorithm by Greiner and Hormann (turn number calculation or ray casting), as shown in the drawing 9 .d again illustrated using the image section 92. The field polygons are again given by vertices 94 and edges 95. The arrowhead is approximated by suitable image processing as point 85, preferably given by a single (X,Y) floating point value tuple. This means that it can now be determined by means of a number of turns calculation or ray casting whether the point (ie the tip) lies within a polygon and thus a point field and also in which field 99, so that the point number can then be determined. This would make determining the point value for an arrowhead even quicker and easier than for an arrowhead given simply by a larger polygon. For example, if the point of the tip is close to a field polygon edge, you can slightly reduce the probability that the point is in the assumed field, and thus obtain an even more realistic statement about the certainty of the number of points determined.

drawing 10 shows an exemplary flowchart of the method with the possible, preferred sub-steps of a rule-based algorithm 100 for determining the scores of several arrows stuck in a dartboard at the same time. The starting point is a dartboard with up to three darts 101 stuck in it, after three darts have been thrown. In step 102, a series of two-dimensional images from one or more digital CCD/CMOS cameras from different viewing angles are first recorded on the pane. This gives you as an intermediate result 103 a number of 1 to n individual images from different viewing angles. In step 104, the regions/areas of the arrows in the are searched for each of the individual images Disc. To do this, in step 104 you preferably segment the tip of an arrow from the remaining components of an arrow, look for the fields adjacent to the estimated locations of each arrowhead in the board and save an image section with the arrowhead and relevant adjacent fields (including the real field) for each arrow , in which the arrow is located). This means that the interim result is 105 up to three image sections for each of the 1 to n individual images. In the next step 106, for each of these up to three image sections and for each of the 1 to n individual images, the wires (ie segments or fields) of the spider on the disk are broken down into polygons by image processing algorithms, for example for segmentation, so that each relevant field is one Polygon forms. Afterwards, in step 106, the tip of the arrow is still searched for in each image section and a polygon of its part in the arrow is formed at least for this tip, but preferably the frontmost point should be searched for or found as the (X,Y) position of the tip which represents the contact of the arrowhead with the disk. As an intermediate result 107, per image section and arrow you get several polygons of the fields (ie their wire boundaries) on the disk as well as a polygon or a point of the arrowhead (this point can also be approximated as a very small polygon, e.g. square, or just e.g as a tuple of two floating point numbers for the (X,Y) position in the image). For example, if all three arrows or their tips are spatially close to each other, then one image section may be sufficient, and in this image section you look for up to three arrowheads and form up to three polygons or points from them. In the subsequent step 108, a polygon clipping algorithm is preferably repeatedly used to calculate the section between all polygons of the relevant fields of the disk on the image section with the respective polygon of the arrowhead. If the arrowhead in step 108 from intermediate result 107 is already present as a point (ie the point of impact of the arrowhead on the disk), then by determining the number of turns for all polygons in the fields around the point, the field in which the point lies and thus the arrowhead can be determined plugged. Alternatively, a so-called ray casting process can be used to find the polygon in which the point lies or the arrow is located. As an intermediate result 109 you now have fields in which there are intersections or overlaps between the arrowhead and a field polygon or the direct result as to whether an arrow point is in a field polygon. When using cut surfaces, for example, it must still be checked for which field in 1 to n evaluated image sections (from different camera viewing angles) an overlap with the polygon of the arrowhead occurs most often or most likely. This is, for example, the field for which the score must now be determined. For arrowheads approximated as a point, after step 108 you immediately have the polygon in which the tip is located (should provide the same, clear result for all viewing angles). Finally, in step 110, the value of the field determined in the previous steps must be determined for each section or arrow. To do this, for example (if it is not one of the bull squares in the middle), a scan from the inside to the outside can be initiated in the middle of the board on the full frame of the disc from step 102, with the value of the digit on the edge of the board can be determined again through image processing (a neural network or machine learning can also be used for this), as well as from the number of crossings of perpendicular wires, whether it is a single, double or triple field of the corresponding field segment. If an arrowhead does not lie in a polygon, then it is preferably considered not to be in the points range, ie the score in this case would be “0”. Overall, even if there are slight deviations in the wires on the disk compared to exact, uniform processing, or if the disk is slightly shifted or rotated relative to the sensors (ie the camera or cameras), the arrowheads can still be accurately assigned to the respective fields of the disk and thus the points possible.

drawing 11 shows a neural network, here specifically a convolutional neural network (CNN) 111, as a preferred embodiment of a method or algorithm from the field of artificial intelligence. Starting from an input layer 112 with several channels for the colors red, green and Blue of an RGB camera image, it further consists of several convolutional layers, 113 and 114 (only two marked here; however, the convolutional layers include all the rectangles shown similar to 113 and 114 in the corresponding sub-steps of the process shown). The number of these convolutional layers can be very large, for example a hundred or more. Each of these convolutional layers consists, for example, of several so-called feature maps of the same size. Within each such feature map, all so-called neurons have the same parameters (these are “weights” and “bias term”), but different feature maps can have different parameters. A so-called receptive field of a neuron extends over the feature maps of all previous layers. This property is intended to be exemplified by the tubular elements 115 and 116 for many such “tubes”. A neuron of a feature map in a particular row and column in a convolutional layer (like 114) is linked to a neuron in the previous convolutional layer (like 113) across all feature maps in that previous convolutio nal Layer 113. As mentioned previously in the description, some or all convolutional layers are followed by a pooling layer (not shown here). Between the individual convolutional layers and pooling layers there are so-called activation functions such as the so-called ReLU function or the tangent hyperbolic function, which connect the inputs and weights with each other. Finally, one or more so-called fully connected layers 117 and 118 follow, before finally the output layer 119 follows. During training, the output layer 119 contains, for example, the information about which arrows are within which segment field and thus which arrow or arrowhead corresponds to which score on the board. The data used to train a CNN essentially consists of previously recorded camera images from different angles, each of which contains the correct scores as a label. The features for the CNN preferably include: the edges of the segments or fields in the form of wires; the arrowheads, whose colors can vary; the colors of the segments or fields (preferably black, white, red, green); the numbers on the edge of the disc, which indicate the value of the segments running radially outwards. The advantage of a CNN 111 or artificial intelligence is, on the one hand, an expected faster calculation of the scores because the neural network has been trained beforehand and the weights matrix, which is essential for the calculation, has been optimized for the application of scoring scores through the training of the neural network. However, it should be noted that a very large number of individual images will be necessary for training such a neural network from different perspectives of the target in order to enable correct determination of the score of arrows even in complex situations (order of magnitude -1 million individual images). Likewise, this training data set must contain a sufficient amount of all angles and complex situations such as three arrows stuck close together in order to cover all potentially possible cases.

The embodiment shown and just described represents the invention only as an example. However, the invention is not limited to this embodiment only. Further embodiments and refinements of the invention are conceivable, which are not detailed here. In addition, all new features presented in the description and/or drawings are essential to the invention, even if they are not explicitly claimed in the patent claims.

5. Advantages of the invention

Based on the description with drawings and preferred embodiments of the invention, the innovations of the invention result in a number of advantages for the user, i.e. a steel darts player, based on the previous state of the art, which are briefly summarized again below. Firstly, the invention is suitable for determining the scores of more than one arrows stuck in a steel darts board, even if the arrowheads that determine the score would be obscured from a camera's perspective. This is solved by the large number of views of at least one camera and the recording of images from different angles of the pane as well as subsequent analysis of these images using software or algorithms and a suitable combination of their individual results (which field is the arrow most likely in?). Secondly, with the current state of technology, it is probably not possible to react appropriately to shifts and rotations of the disk in relation to the measurement technology (here these are the cameras or at least one rotating camera with multiple viewing angles of the disk). nor to assign the X-Y position of the arrowhead exactly to the correct field, especially if arrowheads are close to the wires of the target or are hidden. The invention solves this problem primarily via software using the described algorithms and combinations thereof as well as extensions thereof that are not further detailed here. This means that material errors in the disc (e.g. slight deviations of the spider from the geometrically exact, optimal shape) are no longer a problem, as these slight shifts can be robustly absorbed using the calculation methods. Thirdly, the invention described is also self-calibrating, since in particular the algorithms based on artificial intelligence can be trained on corresponding steel darts and different arrows with metallic tips. But the rule-based algorithms can also be designed on the system in a trained, optimized form based on real data in such a way that calibration is no longer necessary.

For the steel darts player, the invention described makes it possible for the first time to have an easy-to-set-up, self-calibrating, stable system that is robust against external influences such as shifts, with an automated evaluation process of his throwing results plus result statistics, as has so far only been possible for e-darts. This was the case with machines with soft tip darts. The invention could also be transferred to the soft-tip darts area if the scores were to be determined here too without contact in relation to the disc.

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 4207497 A1 [0011]
• EP 0752089 B1 [0012]
• WO 9831979 [0012, 0015]
• US 20060066053 A1 [0014]
• US 7175182 B2 [0015]

Claims (10)

System for automatically determining the score of several arrows (20) stuck in a dartboard (10) at the same time based on their assignment to defined segments or fields (11, 12, 13, 14, 15) on the dartboard, consisting of - at least one camera ( 41), - at least one storage unit (73 and/or 77), - at least one computing unit (75), - at least one display (79), - at least one loudspeaker (88), characterized in that a) the at least one camera ( 41) records one or more individual images (80) of the dart board (10) and the darts (20) stuck therein from different viewing angles (63) and fields of view, b) the individual images (80) recorded in method step a) in the at least one storage unit ( 73) are stored, c) from the images recorded and stored in method steps a) and b) by the at least one computing unit (75) using the at least one algorithm (100, 111) the positions of one or more arrows in relation to the segments and fields (11, 12, 13, 14, 15) of the dartboard (10) can be calculated, d) a further storage unit (77) temporarily stores the scores determined in method step c), remaining scores and other statistical values, e) the at least a display (79) at least partially visually represents the results buffered in method step d), f) the at least one loudspeaker (88) additionally acoustically communicates the score buffered in method step d) after a number of darts have been thrown through a voice output, and g) the so far The necessary procedural step of calibrating and recalibrating the at least one camera (41) with respect to the dartboard (10) is no longer necessary, which would otherwise occur in previous systems due to minor shifts and/or rotations of the dartboard (10) against the sensor system (here the camera 41 ) and would have been necessary due to errors in the wires (16) separating the segments and fields resulting from the manufacture and/or use of the dartboard (10). Procedure according to Claim 1 , characterized in that in method step c) a rule-based algorithm (100) runs on the at least one computing unit (75), consisting of the following individual steps and algorithms in variable order and frequency of execution: a) segmentation and selection of the areas with the individual arrowheads into the under Claim 1 , process steps a) and b), recorded and saved individual images, as well as storing the reduced image areas (91, 92, 93) created in this way, b) segmentation of the arrowhead and the wires of the boundary in the respective image areas (91, 92, 93 ) contained segments or fields of the dartboard, and subsequent formation of polygons from the determined corners and edges, c) calculation of the contact point of the arrowhead with the dartboard as a circle with a minimum radius (ie approximated as point 85), d) calculation of the overlap or the intersection (98) of the polygons of the wires with the polygon or point of the arrowhead, e) determining the most likely assignment of an arrowhead to a point segment or point field by combining the results of the same image areas in images (80) from different viewing angles (63 ), f) Determining the simple value of the segment or field based on the numbers on the edge of the dart board and their multiplicity, g) Adding up the scores for each individual arrow. Procedure according to Claim 1 , characterized in that in method step c) an algorithm from the field of artificial intelligence is executed, consisting of the following essential individual steps in variable order and frequency of execution: a. Definition of a structure for the application of deep learning methods, especially a neural network, consisting of a large number of layers of neurons, b. Training a neural network defined according to the previous method step a) of this claim with a large number of images (80) of the dart board (10) with arrows (20) previously recorded from different perspectives, their scores being used as correct data for training the neural network in addition to the images are made available, c. Application of the neural network trained in the previous method step b) of this claim as an algorithm for automatically determining the score of several arrows stuck in a dart board at the same time. System after Claim 1 , characterized in that the dart board (10) is a steel dart board made of preferably sisal fibers for holding arrows with hard, in particular metallic, tips. System after Claim 1 , characterized in that the at least one camera (41) is each a digital camera, preferably a CCD or CMOS camera. System after Claim 1 , characterized in that the more than one camera (41) is each attached to a fixed position laterally in front of the dart board, with the viewing angle (63) and the field of view of the respective camera being aligned in the direction of the dart board. System after Claim 1 , characterized in that a single camera (41), oriented in the direction of the dartboard in terms of viewing angle and field of view, is attached to a circular carrier (61) and can rotate around it in a direction of rotation (62), this camera being at definable points (63 ), ie viewing angles in the direction of the disk, can stop along the annular support in order to record at least one individual image of the disk (10) with arrows (20) at these points. Procedure according to Claim 2 , method step a), b), c) and f), characterized in that digital image processing algorithms, preferably segmentation and/or Fourier transformation algorithms, are used to calculate the image areas and polygons or points. Procedure according to Claim 2 , method step d) and e), characterized in that polygon clipping algorithms are preferably used to calculate the overlap or intersection of two or more polygons or to determine whether a point lies in or outside a polygon, as well as for determining of the field in which an arrow is located, stochastic algorithms can be used. Procedure according to Claim 3 , method step a), b) and c), characterized in that the neural network is preferably a so-called “convolutional neural network” (111).

Images