WO2014132080A2

WO2014132080A2 - Method and system for overlaying images or data

Info

Publication number: WO2014132080A2
Application number: PCT/GB2014/050603
Authority: WO
Inventors: Steven E. Williams; Nicholas W. F. LINTON
Original assignee: Kings College London
Current assignee: Kings College London
Priority date: 2013-03-01
Filing date: 2014-02-28
Publication date: 2014-09-04
Anticipated expiration: 2015-09-01
Also published as: WO2014132080A3; GB201303659D0

Abstract

A method and system that generates a dot-based representation of at least one of the images or data sets of multiple such images or data sets that are to be overlaid on top of each other is described. The dot based representation relates the number of dots per resolution unit of the representation to the underlying image values or data set that they represent, in that the more dots which are shown the higher the number (or vice versa). The primary advantage of the dot based representation technique is that its meaning is easily understood by a viewer, and it provides a representation wherein the underlying image can be readily seen through the dots, and hence overlaying two or more data sets on top of each other is more readily facilitated. Moreover, the generation of the dots per resolution unit is deterministic, in that the number of dots is determined based on the underlying image or data set values.

Description

Method and System for Overlaying Images or Data

Technical Field The present invention relates to a method and system for overlaying images or other image type data. In particular, specific embodiments relate to overlaying two dimensional or three dimensional image data or related image type or other data in registration, wherein at least one of the data sets has been converted into a dot based representation. In this way, a first image or data set can be viewed in an unmodified state, and the other(s) as dot based representation(s) overlaid on the first image without significantly obscuring the first image.

Background of the Invention and Prior Art

Displaying one image representing a first data set overlaid onto another image representing a second data set can be useful in many fields. For example, in the medical imaging field images or data sets of parts of the human body can be obtained from many different modalities and sources, whilst relating to the same region of the body. It is often useful to be able to view the different images obtained from different imaging modalities of the same anatomical part in registration, so as to try and glean information therefrom to improve diagnosis or treatment. Figure 1, for example, adapted from Gupta S, Desjardins B, Baman T, et al. Delayed-enhanced MR scar imaging and intraprocedural registration into an electroanatomical mapping system in post-infarction patients. J Am Coll Cardiol Img 2012; 5: 207-210, illustrates various imaging modalities of a patient's heart, with the left most figure representing an electro-anatomical voltage map of the heart obtained by point sampling the voltages across the organ's surface and interpolating between the points, whereas the central figure shows scar tissue (shown as the lighter colour) obtained via magnetic resonance (MR) imaging. The right most figure illustrates a combination of the left most and central figures, where the figures have been overlaid in registration. However, as will be seen, in this case, simply overlaying one visual data set on top of another visual data set in registration with each other means that some of the underlying data set (in this case the electrical point sampling image) is obscured by the scar image, and vice versa. This issue might be solved by making at least one of the images translucent, but then further problems arise for users in determining the proper colour of a part of an image. This problem particularly arises in systems where the displayed colour is used to convey information, such as in the left most image of Figure 1.

The issue of overlaying images one on top of the other and processing at least one of the images to allow the other image (or both images) to be more clearly seen has also been encountered in other fields outside medical imaging. For example, in WO2005/025219 a videoconferencing system is described wherein video images representing the local image and the remote image are overlaid on top of each other. In this case, one of the images has been processed prior to being overlaid on top of the other, the processing involving, in some embodiments, increasing the image opacity and smoothing the image to produce a "washed- out" image, or in other embodiments using Laplacian edge detection operators to produce edge based images. Whichever operation is involved, the purpose appears to be to perceptually de-emphasise one of the images with respect to the other, such that when overlaid on top of each other the processed image draws less attention than the non-processed image. The object of the system is to allow multiple video images to be simultaneously displayed on small screens, such as mobile phones.

However, neither of the above solutions permit two images or image type data sets to be overlaid one on top of the other in registration with each other in a manner which allows the viewer to reliably discern both images, and in particularly in a manner which allows the user to discern properties inherent in the images, such as the values of measurements represented by the images.

Summary of the Invention

The present invention addresses the above problem by providing a method and system that generates a dot-based representation of at least one of the images or data sets of multiple such images or data sets that are to be overlaid on top of each other. The dot based representation relates the number of dots per resolution unit of the representation to the underlying image values or data set that they represent. Thus, the greater the numbers of dots shown the higher the underlying image value or data set number (or vice versa). The primary advantage of the dot based representation technique is that its meaning is easily understood by a viewer, and it provides a representation wherein the underlying image can be readily seen through the dots, and hence overlaying two or more data sets on top of each other is more readily facilitated. Moreover, the generation of the dots per resolution unit is deterministic, in that the number of dots is determined based on the underlying image or data set values.

In particular, in some embodiments the location of the dots represents the location of features in the image that they represent, and, the screen density of the dots is related to the signal intensity of the data. Therefore dot mapping overcomes the challenge of manual segmentation by providing a direct mechanism to display the location, intensity and extent of various types of data without manual operator input or the use of automatic segmentation algorithms. In particular because only a fraction of the available screen space is used to display the dots, it is possible to 'see through' the dots to the underlying (or overlying) surfaces. Therefore these surfaces can be colour-coded in the traditional fashion in order to represent a second data field. In this way dot mapping overcomes the traditional limitation of only being able to display one data set at any one time. In view of the above, from a first aspect there is provided a method for displaying a data set, the method comprising: receiving at least one data set relating to spatially distributed features or characteristics of an object or entity; generating a dot-based representation of the at least one data set, the dot-based representation deterministically relating the number of dots per area or volume to the underlying values of the at least one data set; and displaying the generated dot-based representation.

In one embodiment, the display of the dot based representation is made in the context of at least one other data set. Therefore, in such an embodiment the method further comprises receiving one or more further data sets relating to spatially distributed features or characteristics of the object or entity, wherein the displaying comprises displaying the dot- based representation substantially in spatial registration with one or more further data sets.

In one embodiment the data set(s) is/are preferably any of a point-based data set, or pixel or voxel-based data sets.

In one embodiment the at least one data set comprises point based data, wherein the generating comprises: forming polygons or polyhedrons relating to said points; determining respective values for the polygons or polyhedrons in dependence on the point values; calculating a number of dots per polygon or polyhedron based on the determined respective values; and distributing the calculated numbers of dots across the polygons or polyhedrons. Preferably, the polygons or polyhedrons are simplices

In another embodiment the at least one data set comprises pixel or voxel based data, wherein the generating comprises: calculating a number of dots per pixel or voxel based on the respective pixel or voxel values; and distributing the calculated number of dots across the pixels or voxels.

In some embodiments the distributing includes randomly distributing the calculated number of dots.

In some embodiments the number of dots is adapted in dependence on a zoom factor and the resultant size of the generated dot-based representation on a display. More particularly the number of dots is optionally adapted to maintain substantially constant dot density on the display. In an alternative embodiment, the inter-dot relationship is modified to maintain appropriate relative position.

Another embodiment further includes tracking the position of a viewer's head, and adapting the display to provide a pseudo-3 -dimensional display in dependence on the tracked viewer's head position.

Another aspect of the invention provides a system for displaying multiple data sets, the system comprising: a processor; a display; and a computer readable medium, the computer readable medium storing one or more computer program(s) so arranged such that when executed they cause the processor to: i) receive at least one data set relating to spatially distributed features or characteristics of an object or entity; and ii) generate a dot-based representation of the at least one data set, the dot-based representation deterministically relating the number of dots per area or volume to the underlying values of the at least one data set; and cause the display to display the generated dot-based representation.

In one embodiment the processor is further arranged to receive one or more further data sets relating to spatially distributed features or characteristics of the object or entity, and to control the display to display the dot-based representation substantially in spatial registration with one or more further data sets. From a further aspect there is also provided a computer program or suite of computer programs so arranged such that when executed it/they cause a computer to operate in accordance with the first aspect above. A yet further aspect provides a computer readable storage medium storing one or more such computer programs.

Description of the Drawings

Further features and advantages of the present invention will become apparent from the following description of embodiments thereof, presented by way of example only, and by reference to the accompanying drawings, wherein like reference numerals refer to like parts, and wherein: -

Figure 1 is a set of figures illustrating an image overlay technique of the prior art;

Figure 2 illustrates a diagram of an embodiment of the invention;

Figure 3 illustrates an example plot of connected simplexes that can be formed for point data on a 2 dimensional surface;

Figure 4 is a diagram illustrating how different simplexes can be given a different dot density, depending on the values of their vertexes;

Figure 5 illustrates how a tetrahedral simplex can be formed from three dimensional data, and a dot density found therefor;

Figure 6 illustrates how different dot densities may be found and applied to individual pixels in an imaging system, where the pixel size from the imaging system is larger than the display resolution;

Figure 7 illustrates how different dot densities may be found and applied to individual voxels in an imaging system, where the voxel size from the imaging system is larger than the display resolution;

Figure 8 is a flow diagram of the overall process of an embodiment according to the invention;

Figure 9 is a flow diagram of a part of the process for generating a dot representation for point based data;

Figure 10 is a flow diagram of another part of the process for generating a dot representation for point based data; Figure 11 is a flow diagram of a process for generating a dot representation for pixel or voxel based data;

Figure 12 is an example of two overlaid images, one of which has been converted to the dot representation, wherein the dot representation is a 2D representation on a dot plane;

Figure 13 is a second example of two overlaid images, one of which has been converted to the dot representation, wherein the dot representation is a 3D representation;

Figure 14 is a graph of a correction factor to be applied to dot generation depending on the zoom factor applied to the overall image;

Figures 15 and 16 are diagrams illustrating how a virtual three dimensional view can be obtained using head tracking and change of image displayed.

Description of the Embodiments

An embodiment of the invention will now be described. Figure 2 illustrates an example computer system 20 that may form the platform for embodiments of the invention, The computer system 20 comprises a central processing unit (CPU) 201, a video interface 202 arranged to drive a display 22, and an input interface 203 arranged to receive control inputs from a user via an input device 24 such as a keyboard, mouse, or other controller. A network interface 205 is also provided to provide connectivity to a network 25 such as an intranet or the Internet, and which provides a route for data input and output to and from the system.

The computer system 20 is also provided with a computer readable storage medium 204 such as a hard disk drive (FIDD), flash drive, solid state drive, or any other form of general purpose data storage, upon which is stored data and various control programs arranged to control the computer system 20 to operate in accordance with embodiments of the present invention. For example, an overall control program 2048 is provided, which is arranged to provide overall control of the system to perform embodiments of the invention, for example including receiving user inputs as to which images or data should be processed, and calling other functions or launching other programs to perform specific data processing tasks. There is also provided a dot generation program 2050 which is arranged to generate dot representations of images under the control of the control program 2048. An overlay image generation program 2052 is also provided, which generates output overlay images of dot representations and other images, again under the control of the control program 2048. Finally, a mesh generation program 2042 is provided, that acts to generate meshes of simplexes formed from point data, that are required to allow the present embodiment to process point based data sets.

In addition to the above, the computer readable medium 204 also stores thereon respective image or other data sets 2044 and 2046, representing the image data or other data that is to be overlaid in registration with each other to generate output images.

Various other components and systems would of course be known to the person skilled in the art to permit the computer system 20 to operate, but such are beyond the scope of the present embodiments.

Turning to Figure 8, the operation of embodiments of the invention will now be described. Figure 8 is a flow diagram illustrating the overall operation of the control program 2048 to provide embodiments of the invention. In particular, at s.8.2 the control program operates to receive image data, or other data sets, such as point based data, to be processed and overlaid on top of each other. The received data is stored on the computer readable medium 204 as data sets 2044 and 2046. Next, the control program 2048 controls the dot generation program 2050 to convert at least one of the images or data sets into a dot based representation, in a manner to be described. This is performed at s.8.4, as shown. Finally, once the dot based representation has been generated for at least one of the images or data sets, at s.8.6 the control program 2048 uses the overlay image generation program 2052 to generate an output image illustrating the dot based representation overlaid onto the other image in registration therewith. Examples of the output images that can be obtained are shown in Figures 12 and 13.

Figures 9 and 10 illustrate in more detail the steps involved in generating a dot-based image representation for point based data. For example, the point based data might be medical data from an electro-anatomic mapping system, and in some embodiments may represent electrogram voltage at each point, or local activation time (for example where the point based data relates to characteristics of the heart). More particularly, at s.9.2 the dot generation program 2050 receives the point based data set. For example, this could be an image which has been generated by interpolating between values obtained for measurement points across the region the image represents, or could be a set of data point values with attendant location data as to where each value represents in the data set. For example, where the data points are electrical activation times in the heart of a subject, then the data points may represent and encode the actual times and the positions in the heart. Furthermore, the data points may represent interpolation of data between measured samples in such a way as to provide a high resolution representation of a 2D or 3D object.

The next step is that it becomes necessary to connect the point based data into areas over which the dots might be spread. In this respect, embodiments of the invention rely on the density of dots across a two dimensional area or three-dimensional region to convey information, and hence associated 2D areas or 3D regions should be generated related to the point based data. This is performed by the processing loop shown in steps 9.4 to 9.10 in Figure 9. Here, for each point in the received data set (s.9.4) the two closest other points (in the case of a 2D arrangement over a plane surface) or the three closest points (in the case of a 3D arrangement) are determined (s.9.6) and a triangular simplex (in the case of 2D or tetrahedral simplex (in the case of 3D) is formed from the determined points. Once a simplex has been formed, the next point in the data set is processed in the same way (s.9.10) until all the points have been processed. In this way triangular simplexes are formed as shown in Figure 3 for the case of a 2D plane representation, or tetrahedral simplexes formed as shown in Figure 5 for a 3D representation. The purpose of forming the simplexes is to provide an area or region over which dots can be distributed with a particular density to convey information relating to the value of the points that form the vertexes of each simplex. Therefore, the next step in the dot generation program is to determine a value for each simplex that can then be converted into a dot density for distribution over the simplex. Figure 10 illustrates the process performed to generate the dot distribution.

More particularly, a processing loop is started at s.10.2 to process each generated simplex. For each simplex a value for the simplex is determined at s.10.4 from the values of the points that form the vertexes of the simplex. In this respect, it will be recalled that each simplex is formed from the point based data, where each point has a value representing measured data. For example, the point based data may be electrical activation time or electrogram voltage, where the data represents heart data. That is, each point is a measurement point with a value and location, and hence a value may be generated for a simplex by applying a mathematical function to the values of the points at the vertexes of the simplex. For example, the value could be simply the mean value of the values of the points at the vertexes.

Having determined a value for each simplex from the vertex values, it then becomes necessary to calculate the number of dots to be drawn on the simplex based on the value. In this respect, the underlying data will usually be within a known range of values from a maximum to a minimum, for example from roughly 0 to 10 mV in the case of heart electrical activation voltages. The determined value can therefore be mapped against the range of values represented by the set of points, to determine where in the range the determined value lies. The density dot coverage of the simplex can then be found for example by linearly interpolating between the ranges to the value, to determine how much of the simplex should be covered with dots. For example, if the known range is 0 to 10 mV, and the determined value is 5.9, then in one embodiment the percentage of the simplex that should be covered with dots might be set at 59%. The number of discrete dots required can then be calculated based on the size of the simplex.

In other embodiments a different function might be used to relate the number of dots to the determined value. For example, it might be undesirable to completely cover the simplex with dots in the case of a high value, as then the simplex would be substantially opaque, and one of the main advantages of embodiments of the invention would be lost. Therefore, in other embodiments a maximum dot density might be set, such as for example, 30%, wherein only 30%) of the area or region of the simplex might be occupied with dots as a maximum. In this case, interpolation between zero dots and this maximum value can be performed in dependence on the determined value. More generally, a fraction of the simplex that should be covered by dots can be found from the following:

Fraction of simplex = (Determined value / (max data value - min data value)) * density factor where the determined value is the value is determined at s.10.4, the maximum and minimum data values are those maximum and minimum possible values that can be obtained i.e. 0 to 10 mV in the example above, and the density factor is maximum dot density that might be set e.g. 30%. Having found the fraction, the precise number of dots can then be found from the display size of the simplex and the display resolution, and the calculated fraction.

In other embodiments a further different function might be used to relate the number of dots to the determined value. For example, rather than a percentage-based approach given above, an absolute maximum, and if necessary minimum, number of dots might be set for each simplex. Once these levels are set the number of dots for each simplex may be calculated by scaling and offsetting the original data set range to these maximum and minimum numbers of dots. The number of dots per simplex then becomes directly identifiable from the new, scaled data range. This approach might be further extended to include additional maximum and minimum 'threshold' levels on the original data set. In this way the system may be set up to always draw the maximum number of dots at each and every simplex for which the original data set value is above the maximum threshold, and correspondingly to draw no dots at any simplex for which the data set value is below the minimum threshold.

Having determined the number of dots to be drawn on a simplex, at s.10.8 the dot generation program 2050 acts to distribute the dots at positions across the simplex, and then processing proceeds to the next simplex, at s.10.10. In this way, a number of dots and the dot distribution is calculated for each simplex. In terms of how the distribution is performed, in one embodiment the distribution is a randomised distribution, in that the calculated number of dots are scattered at random across the surface or region of the simplex. However, in other embodiments other distributions can be used, for example by distributing the dots so that they are substantially equally spaced across the surface or region. Other distribution functions may also be employed.

Once the dot number and dot distribution has been found, at s.10.12 the overlay image generation program 2052 acts to generate an output image with the dot based registration of the a first image or data set overlaid in registration with a second image showing different aspects of the same article.

For example, in the medical imaging field, Figure 12 shows an example image showing cardiac scar/voltage data (dots) and local activation time data (colours) on a single shell, whereas Figure 13 shows scar location within the myocardial wall of the left ventricle of a human heart. In Figure 13, the underlying image is the interior (endocardial) surface of the ventricle, whereas the dot representation has been generated in 3D to show scar location actually within the ventricular wall. In this regard, the ventricular wall is modelled as a tetrahedral simplex mesh described above. With the above therefore, an imaging technique is provided that allows a dot based representation of a set of data, which might be point based data to be overlaid onto a second image in registration therewith, to allow a viewer to see both image sets simultaneously.

The above embodiment is concerned with point based data, and generating 2D or 3D simplexes to form the areas or regions over which dots may be distributed to represent the data. In a second embodiment described next the input data is already image data, either in pixel form in the case of a 2D image, or voxel form in the case of a 3D image (such as from an MR scan). In this case, the pixel or voxel from the imaging system that produced it will have a certain resolution or size, depending on the imaging system. For example, a 1.5T MRI image may have a spatial resolution of 1.5 to 2 mm i.e. each voxel is of size 1.5 to 2 x 1.5 to 2 mm x 1.5 to 2 mm. The native resolution of the imaging system that produced an input image therefore provides a natural area or region over which dots may be distributed to provide a dot based representation. Figure 11 therefore shows the procedure performed by the dot generation program when dealing with pixel or voxel based input data. At s.11.2 the pixel or voxel based data is received, and at s. l 1.2 a processing loop is commenced to process each pixel or voxel. In this respect, for each pixel or voxel in the image the required number of dots is calculated based on the actual pixel or voxel value. In this respect, the same considerations as described above in respect of the first embodiment also apply, and in this respect s. l 1.4 is the same as s.10.6, but with different input values (i.e. based on the pixel/voxel value, rather than determined value for a simplex). Once the number of dots has been calculated, at s. l 1.6 the dots are randomised across the pixel/voxel, and then at s. l 1.8 the next voxel or pixel in the image is selected. The result is that a dot based representation is obtained that for each pixel or voxel replaces the pixel or voxel value with a number of dots randomly distributed across the area or region of the pixel. Of course, for display purposes in this embodiment the display resolution is assumed to be greater than the resolution of the input image data, such that each pixel or voxel in the input image would occupy many pixels in the display. Once the dot based representation has been produced the overlay image generation program 2052 then acts to generate an overlay image of the dot based representation with the second input image, in registration therewith. This image may include representations of all of the original image pixels, or some of the image pixels where some of the pixels are selected for display based on an algorithm, for example a clipping plane.

Figures 6 and 7 show respective pixel and voxel based dot representations generated by the second embodiment. Figure 6 shows a pixel based representation where an array of pixels 60 has had dot representations generated for each pixel area. In this example, pixel 602 has had a first dot density calculated for it, where as pixel 604 has had a higher dot density calculated. Other pixels have also had their respective dot densities found and displayed.

Figure 7 shows how dots can be found and distributed throughout voxel representations. In this Figure the dots are represented by different shapes in each voxel, but that is for representation purpose only to show the different dots in each voxel. In a real-life embodiment that dots would typically be of identical shape in each voxel. From Figure 7 it can be seen that different dot densities can be found for different voxels, depending on the initial voxel values. Thus, embodiments of the invention provide for the generation of dot based representations from both point based and pixel or voxel based input data sets. The dot based representations use the display density of dots across a 2D area or in a 3D region to communicate information to the viewer as to the value of the original data sets or images at those points. As such, multiple spatially distributed data sets such as image data, or point based data may be displayed to and easily understood by the user. In this respect, the cognitive burden on the user is reduced.

Various modifications may be made to the above described embodiments to produce further embodiments. For example, when viewing the rendering of dot based representation produced by the image generation program 2052 it is beneficial to adjust the number of dots in a simplex depending on the relative screen size of the image. In this respect, if the rendered image has been zoomed out on the screen so that the whole image appears smaller, then fewer dots will be needed per simplex. In contrast, if the image has been zoomed in such that each simplex appears larger, then more dots may be required. A zoom factor may therefore be applied to the calculated number of dots per simplex, or per pixel or per voxel, as appropriate, an example of which is shown in Figure 14.

In this respect, when the dot based representation is being rendered, then the number of dots per simplex will have been calculated as described in the previous embodiments. This calculated number is then used as an input to a function such as that shown in Figure 14 that keeps the number of dots per simplex (or pixel or voxel) at the same display density irrespective of the zoom factor. If such a function is not provided then in the case of zooming in towards a pixel or simplex the number of dots displayed per unit area (ie dot density) would appear to decrease as the dots became further apart, and information would be lost to the viewer. Conversely, in a zoom out situation the distribution of dots would become closer together and eventually become opaque. By adjusting the displayed dot density so that it remains substantially constant as viewed irrespective of zoom factor the advantages of the invention can be maintained.

Further in this regard, by substantially constant we mean that the displayed dot density is maintained within a range of a few percent irrespective of zoom. In the example zoom adaptation function of Figure 14 the dot density is maintained between approximately 5.5% to 7% across the range of zoom factors i.e. the zoom adaptation function extends over a range of approximately 1.5%. Other zoom adaptation functions may be used, which give slightly larger or smaller ranges across the range of zoom factors, bearing in mind the purpose of trying to maintain the perceived dot density by the user as substantially constant. For example, a dot density zoom range of no more than 5%, or more preferably no more than 3%, or even more preferably no more than 1.5% or even 1% should be sufficient to achieve the purpose.

The above dot density zoom ranges and the zoom adaptation function may be applied to the initial dot density calculated in accordance with the previous embodiments. For example, if the dot density is initially calculated in accordance with the previous embodiments for unity zoom (xl) to be 25% dot density, then using the zoom adaptation function the dot density may be kept in the range of 27.5% to 22.5% (i.e. 5% range) across the zoom range, or more preferably in the range of 26.5% to 23.5% (i.e. 3% range), or even more preferably 25.5 % to 24.5% (i.e. 1%) range), by way of example. With such a zoom adaptation function the perceived dot density and hence the information conveyed thereby can be understood by the user irrespective of the actual zoom factor applied to the display. In another modification, in order to allow perception of the location of dots within a model an algorithm has been developed to adjust the rendering of the model based on the viewer's head position. By allowing the viewer to 'look around' the object they are able to perceive depth, and hence position of the dots both around and within the myocardial wall. The principles of this technique are shown in Figures 15 and 16. In particular, Figure 15 is a schematic diagram of the change in view frustum with head position movement from P_e to P_e, (A), and the resultant alteration in the appearance of the cube drawn on the screen (B). Figure 16 is a graphic representation of the same giving the illusion of depth.

Head tracking algorithms with attendant changes of view to give the illusion of depth and a 3D image are well known, and will not be described further here, suffice to say that in a preferred embodiment the Viola Jones algorithm for facial recognition is used to perform the head tracking with a standard desktop webcam. The advantage of providing such a pseudo- three-dimensional representation is that it can further allow the viewer to more easily appreciate the dot based representation.

In a further embodiment it will be appreciated that the dot based representation may be displayed by itself i.e. without being shown with a second image or other data set (whether in registration therewith or otherwise). In such a case the overlay image generation program simply displays the dot based representation on the display.

Moreover, in further embodiments plural dot based representations relating to a plurality of input images or data sets may be generated in the manners described above, and displayed simultaneously to the viewer. Such display may be together with conventional images or data sets in registration therewith (as in the main embodiments described above), or without such conventional images. The plural dot based representations may be displayed in registration with each other, with or without other, conventional, images or data sets. Finally, in the above describe embodiments we have referred to simplices being formed, such as the triangular simplices for the 2D planar representation, or the tetrahedral simplices for the 3D representation. In other embodiments where point based data is being processed then other polygonal connected shapes may be formed by the points to provide the processing units of embodiments of the invention, and embodiments of the invention are not limited to the triangular or tetrahedral shapes described.

In addition, it should be emphasised that the dot based data representation overlaid on to other two or three dimensional data may be used to represent data pairs of several different related data types. For example, whilst in the medical fields there are several different location-related physiological measurements that may be displayed together, embodiments of the invention are not related to such physiological data, and other pairs of data may be displayed using such a representation. By way of example: i) In the field of neurology - one set of data might be white matter hyperintensities or amyloid deposition within the brain, with the dot-based representation representing regions of cortical atrophy determined from MRI imaging; ii) Map data and seismic data - one set of data might be geographic map data, with the dot-based representation representing seismic data gathered from the corresponding area; iii) Combined meterological data - one set of data might be atmospheric pressure data, with the dot-based representation representing a corresponding meterological dataset such as local precipitation or temperature across the same region.

Other pairs of data sets that might be conveniently displayed using such a dot based representation will be apparent to the person skilled in the art, and the present invention is not intended to be limited in any way to the specific examples given herein.

Further embodiments, whether by way of addition, deletion or modification of features, will be apparent to the intended reader being a person skilled in the art, any and all of which are intended to be encompassed by the appended claims.

References

1. http ://www. ensitevelocity. com/

2. http://www.biosensewebster.com/products/navigation/carto3.aspx Arujuna A, Karim R, et al. Acute pulmonary vein isolation is achieved by a combination of reversible and irreversible atrial injury after catheter ablation: evidence from magnetic resonance imaging. Circ Arrhythm Electrophysiol 2012; 5 : 691-700

Knowles BR, Caulfield D, Cooklin M. 3-D visulaization of acute RF ablation lesions using MRI for the simultaneous determination of the patterns of necrosis and edema. IEEE Trans Biomed Eng. 20101; 57: 1467-1475

Schuleri KH, Centola M, Evers KS. Cardiovascular magnetic resonance characterization of peri-infarct zone remodelling following myocardial infarction. J Cardiovasc Magn Reson 2012; Apr 17: 14-24

Ghoshhajra BB, Maurovich-Horvat P, Techasith T, et al. Infarct detection with a comprehensive cardiac CT protocol. J Cardiovasc Comp Tom 2012; 6: 14-23

Dr Rob MacLeod, Associated Director, Nora Eccles Harrison Cardiovascular Research and Training Institute, Utah. Personal communication.

Codreanu A, Odille F, Aliot E, et al. Electroanatomic characterization of post-infarct scars comparison with 3-dimensional myocardial scar reconstruction based on magnetic resonance imaging. J Am Coll Cardiol 2008; 52: 839-842

Perm EC, Silva GV, Sarmento-Leite R, et al. Assessing myocardial viability and infarct transmurality with left ventricular electromechanical mapping in patients with stable coronary artery disease: validation by delayed-enhancement magnetic resonance imaging. Circulation 2002; 106: 957-961

Crawford T, Cowger J, Desjardins B, et al. Determinants of postinfarction ventricular tachycardia. Circ Arrhythm Electrophysiol 2010; 3 : 624-631

Desjardins B, Crawford T, Good E, et al. Infarct architecture and characteristics on delayed enhanced magnetic resonance imaging and electroanatomic mapping in patients with postinfarction ventricular arrhythmia. Heart Rhythm 2009; 6: 644-651 Gupta S, Desjardins B, Baman T, et al. Delayed-enhanced MR scar imaging and intraprocedural registration into an electroanatomical mapping system in postinfarction patients. J Am Coll Cardiol Img 2012; 5: 207-210

Spears DA, Suszko AM, Dalvi R, et al. Relationship of bipolar and unipolar electrogram voltage to scar transmurality and composition derived by magnetic resonance imaging in patients with nonischemic cardiomyopathy undergoing VT ablation. Heart Rhythm 2012; Jul 27. Epub ahead of print. 14. Spragg DD, Khurram I, Zimmerman SL, et al. Initial experience with magnetic resonance imaging of atrial scar and co-registration with electroanatomical voltage mapping during atrial fibrillation: success and limitations. Heart Rhythm 2012; Aug 30: Epub ahead of print.

15. Oakes RS, Badger TJ, Kholmovski EG, et al. Detection and quantification of left atrial structural remodelling with delayed-enhancement magnetic resonance imaging in patients with atrial fibrillation. Circulation 2009; 119: 1758-1767

16. Suk T, Edwards C, Hart H, et al. Myocardial scar detected by contrast-enhanced cardiac magnetic resonance imaging is associated with ventricular tachycardia in hypertrophic cardiomyopathy patients. Heart Lung Circ 2008; 17: 370:374

17. Spears DA, Suszko AM, Dalvi R, Crean A, Ivanov J, Nanthakumar K, Downar E, Chauhan VS. Relationship of bipolar and unipolar electrogram voltage to scar transmurality and composition derived by magnetic resonance imaging in patients with nonischemic cardiomyopathy undergoing VT ablation. Heart Rhythm. 2012 Jul 27. [Epub ahead of print] PubMed PMID: 22846338.

18. Santangeli P, Pieroni M, Dello Russo A, Casella M, Pelargonio G, Di Biase L, Macchione A, Burkhardt JD, Bellocci F, Santarelli P, Tondo C, Natale A. Correlation between signal-averaged ECG and the histologic evaluation of the myocardial substrate in right ventricular outflow tract arrhythmias. Circ Arrhythm Electrophysiol. 2012 Jun l;5(3):475-83. Epub 2012 Mar 15. PubMed PMID: 22423142.

19. Bilchick KC. Integration of CMR scar imaging and electroanatomic mapping: the future of VT ablation? JACC Cardiovasc Imaging. 2012 Feb;5(2):211-3. PubMed PMID:22340830.

0. D'Silva A, Wright M. Advances in imaging for atrial fibrillation ablation. Radiol Res Pract. 2011;2011 :714864. Epub 2011 Feb 16. PubMed PMID: 22091384; PubMed

Central PMCID: PMC3200077.

1. Oduneye SO, Biswas L, Ghate S, Ramanan V, Barry J, Laish-FarKash A, Kadmon E, Zeidan Shwiri T, Crystal E, Wright GA. The feasibility of endocardial propagation mapping using magnetic resonance guidance in a Swine model, and comparison with standard electroanatomic mapping. IEEE Trans Med Imaging. 2012 Apr;31(4):977-83.

Epub 2011 Nov 4. PubMed PMID: 22067265.

2. Govil A, Calkins H, Spragg DD. Fusion of imaging technologies: how, when, and for whom? J Interv Card Electrophysiol. 2011 Dec;32(3): 195-203. Epub 2011 Oct 1. Review. PubMed PMID: 21964620. Bala R, Ren JF, Hutchinson MD, Desjardins B, Tschabrunn C, Gerstenfeld EP, Deo R, Dixit S, Garcia FC, Cooper J, Lin D, Riley MP, Tzou WS, Verdino R, Epstein AE, Callans DJ, Marchlinski FE. Assessing epicardial substrate using intracardiac echocardiography during VT ablation. Circ Arrhythm Electrophysiol. 201 1 Oct;4(5):667-73. Epub 2011 Aug 31. PubMed PMID: 21880675; PubMed Central

PMCID: PMC3220997.

Andreu D, Berruezo A, Ortiz-Perez JT, Silva E, Mont L, Bonis R, de Caralt TM, Perea RJ, Fernandez-Armenta J, Zeljko H, Brugada J. Integration of 3D electroanatomic maps and magnetic resonance scar characterization into the navigation system to guide ventricular tachycardia ablation. Circ Arrhythm

Electrophysiol. 2011 Oct;4(5):674-83. Epub 2011 Aug 31. PubMed PMID: 21880674. Daoud EG. Electroanatomic mapping and cardiac magnetic resonance imaging for right ventricular disease: complimentary modalities. J Cardiovasc Electrophysiol. 2011 Dec;22(12): 1367-8. doi: 10.1111/j .1540-8167.2011.02143.x. Epub 2011 Aug 1. PubMed PMID: 21806703.

Estner HL, Zviman MM, Herzka D, Miller F, Castro V, Nazarian S, Ashikaga H, Dori Y, Berger RD, Calkins H, Lardo AC, Halperin HR. The critical isthmus sites of ischemic ventricular tachycardia are in zones of tissue heterogeneity, visualized by magnetic resonance imaging. Heart Rhythm. 2011 Dec;8(12): 1942-9. Epub 2011 Jul 26. PubMed PMID: 21798226.

Santangeli P, Hamilton-Craig C, Dello Russo A, Pieroni M, Casella M, Pelargonio G, Di Biase L, Smaldone C, Bartoletti S, Narducci ML, Tondo C, Bellocci F, Natale A. Imaging of scar in patients with ventricular arrhythmias of right ventricular origin: cardiac magnetic resonance versus electroanatomic mapping. J Cardiovasc Electrophysiol. 2011 Dec;22(12): 1359-66. doi: 10.1111/j .1540-8167.2011.02127.x.

Epub 2011 Jul 7. PubMed PMID: 21736658.

Haqqani HM, Tschabrunn CM, Tzou WS, Dixit S, Cooper JM, Riley MP, Lin D, Hutchinson MD, Garcia FC, Bala R, Verdino RJ, Callans DJ, Gerstenfeld EP, Zado ES, Marchlinski FE. Isolated septal substrate for ventricular tachycardia in nonischemic dilated cardiomyopathy: incidence, characterization, and implications.

Heart Rhythm. 2011 Aug;8(8): l 169-76. Epub 2011 Mar 8. PubMed PMID: 21392586.

Nakahara S, Vaseghi M, Ramirez RJ, Fonseca CG, Lai CK, Finn JP, Mahajan A, Boyle NG, Shivkumar K. Characterization of myocardial scars: electrophysiological imaging correlates in a porcine infarct model. Heart Rhythm. 2011 Jul;8(7): 1060-7. Epub 2011 Feb 23. PubMed PMID: 21354335.

Pandozi C, Dottori S, Lavalle C, Ficili S, Galeazzi M, Russo M, Pandozi A, Camastra G, Ansalone G, Santini M. Integration of MR images with electroanatomical maps: feasibility and utility in guiding left ventricular substrate mapping. J Interv Card

Electrophysiol. 2010 Dec;29(3): 157-66. Epub 2010 Oct 20. PubMed PMID: 20960225.

Badger TJ, Daccarett M, Akoum NW, Adjei-Poku YA, Burgon NS, Haslam TS, Kalvaitis S, Kuppahally S, Vergara G, McMullen L, Anderson PA, Kholmovski E, MacLeod RS, Marrouche NF. Evaluation of left atrial lesions after initial and repeat atrial fibrillation ablation: lessons learned from delayed-enhancement MRI in repeat ablation procedures. Circ Arrhythm Electrophysiol. 2010 Jun;3(3):249-59. Epub 2010 Mar 24. PubMed PMID: 20335558; PubMed Central PMCID: PMC3128796.

Taclas JE, Nezafat R, Wylie JV, Josephson ME, Hsing J, Manning WJ, Peters DC. Relationship between intended sites of RF ablation and post-procedural scar in AF patients, using late gadolinium enhancement cardiovascular magnetic resonance. Heart Rhythm. 2010 Apr;7(4):489-96. Epub 2009 Dec 13. PubMed PMID: 20122877; PubMed Central PMCID: PMC2843771.

Desjardins B, Crawford T, Good E, Oral H, Chugh A, Pelosi F, Morady F, Bogun F. Infarct architecture and characteristics on delayed enhanced magnetic resonance imaging and electroanatomic mapping in patients with postinfarction ventricular arrhythmia. Heart Rhythm. 2009 May;6(5):644-51. Epub 2009 Feb 14. PubMed PMID: 19389653; PubMed Central PMCID: PMC2735883.

Bogun FM, Desjardins B, Good E, Gupta S, Crawford T, Oral H, Ebinger M, Pelosi F, Chugh A, Jongnarangsin K, Morady F. Delayed-enhanced magnetic resonance imaging in nonischemic cardiomyopathy: utility for identifying the ventricular arrhythmia substrate. J Am Coll Cardiol. 2009 Mar 31;53(13): 1138-45. PubMed PMID: 19324259; PubMed Central PMCID: PMC2747602.

Knackstedt C, Schauerte P, Kirchhof P. Electro-anatomic mapping systems in arrhythmias. Europace. 2008 Nov; 10 Suppl 3 :iii28-34. Review. PubMed PMID:

18955396.

Codreanu A, Odille F, Aliot E, Marie PY, Magnin-Poull I, Andronache M, Mandry D, Djaballah W, Regent D, Felblinger J, de Chillou C. Electroanatomic characterization of post-infarct scars comparison with 3-dimensional myocardial scar reconstruction based on magnetic resonance imaging. J Am Coll Cardiol. 2008 Sep 2;52(10):839-42. PubMed PMID: 18755347.

Massumi A, Rasekh A, Saeed M, Flam S, Cheong B, Mojibian H, Razavi M. Organized incessant atrial arrhythmias in the setting of severe, isolated biatrial scarring. Pacing Clin Electrophysiol. 2008 Jun;31(6): 666-75. PubMed PMID: 18507538.

Casella M, Pieroni M, Dello Russo A, Pennestri F, Meduri A, Natale L, Bellocci F, Crea F. Characterization of the electroanatomic substrate in a case of noncompaction left ventricle. J Cardiovasc Med (Hagerstown). 2008 Jun;9(6):636-8. PubMed PMID: 18475136.

Bello D, Kipper S, Valderrabano M, Shivkumar K. Catheter ablation of ventricular tachycardia guided by contrast-enhanced cardiac computed tomography. Heart Rhythm. 2004 Oct; l(4):490-2. PubMed PMID: 15851204.

Reddy VY, Malchano ZJ, Holmvang G, Schmidt EJ, d'Avila A, Houghtaling C, Chan RC, Ruskin JN. Integration of cardiac magnetic resonance imaging with three- dimensional electroanatomic mapping to guide left ventricular catheter manipulation: feasibility in a porcine model of healed myocardial infarction. J Am Coll Cardiol. 2004 Dec 7;44(11):2202-13. PubMed PMID: 15582319.

Claims

1. A method for displaying a data set, the method comprising:

receiving at least one data set relating to spatially distributed features or characteristics of an object or entity;

generating a dot-based representation of the at least one data set, the dot-based representation deterministically relating the number of dots per unit area or volume to the underlying values of the at least one data set; and

displaying the generated dot-based representation.

2. A method according to claim 1, and further comprising:

receiving one or more further data sets relating to spatially distributed features or characteristics of the object or entity;

wherein the displaying comprises displaying the dot-based representation substantially in spatial registration with the one or more further data sets.

3. A method according to any of claims 1 or 2, wherein the data set(s) is any of a point- based data set, or pixel or voxel-based data set.

4. A method according to any of the preceding claims, wherein the at least one data set comprises point based data, wherein the generating comprises:

forming polygons from said points;

determining respective values for the polygons in dependence on the point values forming the polygons;

calculating a number of dots per polygon based on the determined respective values; and

distributing the calculated numbers of dots across the polygons.

5. A method according to claim 4, wherein the polygons are polyhedrons.

6. A method according to any of claims 1 to 3, wherein the at least one data set comprises pixel or voxel based data, wherein the generating comprises:

calculating a number of dots per pixel or voxel based on the respective pixel or voxel values; distributing the calculated number of dots across the pixels or voxels.

7. A method according to claims 4, 5 or 6, wherein the distributing includes randomly distributing the calculated number of dots.

8. A method according to any of the preceding claims, wherein the number of dots is adapted in dependence on a zoom factor and the resultant size of the generated dot-based representation on a display.

9. A method according to claim 8, wherein the number of dots is adapted to maintain substantially constant dot density on the display.

10. A method according to any of the preceding claim, and further comprising tracking the position of a viewers head, and adapting the display to provide a pseudo-three- dimensional display in dependence on the tracked viewer's head position.

11. A method according to any of the preceding claims, wherein the receiving includes receiving a plurality of data sets relating to spatially distributed features or characteristics of the object;

generating a plurality of respective dot-based representations of the plurality of data sets, the dot-based representations deterministically relating the number of dots per unit area or volume to the underlying values of the plurality of data sets; and

displaying the generated dot-based representations.

12. A system for displaying multiple data sets, the system comprising:

a processor;

a display; and

a computer readable medium, the computer readable medium storing one or more computer program(s) so arranged such that when executed it/they cause the processor to:

i) receive at least one data set relating to spatially distributed features or characteristics of an object or entity; and

ii) generate a dot-based representation of the at least one data set, the dot- based representation deterministically relating the number of dots per unit area or volume to the underlying values of the at least one data set; and cause the display to display the generated dot-based representation.

13. A system according to claim 12, wherein the processor is further arranged to receive one or more further data sets relating to spatially distributed features or characteristics of the object or entity, and to control the display to display the dot-based representation substantially in spatial registration with one or more further data sets.

14. A system according to claims 12 or 13, wherein the one or more computer programs are further arranged such that when executed it/they cause the system to perform the method of any of claims 3 to 11.

15. A computer program or suite of computer programs so arranged such that when executed it/they cause a computer to operate in accordance with any of claims 1 to 11.

16. A computer readable storage medium storing one or more computer programs according to claim 15.