WO2013080101A2

WO2013080101A2 - Image server and method

Info

Publication number: WO2013080101A2
Application number: PCT/IB2012/056655
Authority: WO
Inventors: Fabian Wenzel; Thomas Netsch; Sebastian Peter Michael Dries; Franciscus Theodorus Gerardus Maria VOGELS; Vishnu Vardhan MAKKAPATI
Original assignee: Philips Intellectual Property and Standards GmbH; Koninklijke Philips Electronics NV
Current assignee: Philips Intellectual Property and Standards GmbH; Koninklijke Philips NV
Priority date: 2011-12-01
Filing date: 2012-11-23
Publication date: 2013-06-06
Anticipated expiration: 2014-06-01
Also published as: WO2013080101A3

Abstract

Image server 110 for transmitting, to an image client 150, a plurality of two- dimensional [2D] views 122 of three-or higher dimensional [3D] image data for enabling a user operating the image client to navigate through the 3D image data by viewing different ones of the plurality of 2D views, the image server comprising an input 120 for obtaining the plurality of 2D views, a transmitter 130 for transmitting the plurality of 2D views in a transmission order 142 to the image client, and a processor 140 for (i) analyzing contents of the plurality of 2D views for obtaining a respective plurality of view properties, and (ii) establishing the transmission order in dependence on the plurality of view properties for transmitting the plurality of 2D views,based on a decreasing degree of information content of the 2D views.

Description

Image server and method

FIELD OF THE INVENTION

The invention relates to an image server and a method of transmitting, to an image client, a plurality of two-dimensional [2D] views of three- or higher dimensional [3D] image data. The invention further relates to a workstation, imaging apparatus and system comprising the image server set forth, and a computer program product comprising instructions for causing a processor system to perform the method set forth.

In the technical field of imaging, devices are known that allow users to navigate through image data having at least three dimensions, such as the three spatial dimensions of volumetric image data or those formed by a stack of two-dimensional images. For enabling said navigation, such devices may display different, e.g., consecutive, 2D views of said 3D image data in response to navigation commands provided by the user. Such devices may, for example, allow the user to navigate through a 3D reconstruction of a body structure by displaying different 2D cross-sections of said 3D reconstruction.

The 3D image data may be located, at a given point in time, on an image server, and not on the aforementioned device. For nevertheless enabling the user to navigate through the 3D image data, the device, henceforth referred to as image client, may be arranged for obtaining 2D views of the 3D image data from the image server, for example, using a network connection that is present between the image client and the image server. The image server and image client together thus form a client/server system, in which the image server is arranged for transmitting the 2D views to the image client.

BACKGROUND OF THE INVENTION

The image server may transmit the 2D views consecutively, i.e., as a sequence of 2D views. The ordering of the 2D views in this sequence is henceforth referred to as the transmission order, and specifies which of the 2D views is transmitted first, second, etc. The image client may be arranged for allowing the user to navigate through the already received 2D views, i.e., before having received all of the 2D views from the image server. In such a case, the user may, soon after the image server has started transmitting, view the 2D views that are positioned first in transmission order, whereas the 2D views that are positioned at the end of the transmission order may only be viewed after the image server has finished transmitting.

A paper titled "Matrix Cubic Splines for Progressive 3D Imaging" by Defez et al, Journal of Mathematical Imaging and Vision, no. 17, pp. 41-53, 2002, discloses a so- termed progressive transmission of a 3D digital object, with the 3D digital object being stored in a stack of parallel 2D slices. It is noted that in the progressive transmission of 93 2D slices Ni, the 2D slices were transmitted using a binary subdivision of the subscripts, as indicated by 46; 23, 69; 12, 35, 58, 81; 6, 18, 29, 41, 52, 64, 75, 87, etc. Thus, a transmission order is employed that is based on a binary subdivision of the range of slices.

A problem of the approach of Defez et al. is that the order of transmitting the 2D slices is insufficiently suitable for enabling a user to intuitively navigate through already received 2D slices before all of the 2D slices are received.

SUMMARY OF THE INVENTION

It would be advantageous to have an improved image server or method of transmitting, in a transmission order, a plurality of 2D views to an image client.

To better address this concern, a first aspect of the invention provides an image server for transmitting, to an image client, a plurality of two-dimensional [2D] views of three- or higher dimensional [3D] image data for enabling a user operating the image client to navigate through the 3D image data by viewing different ones of the plurality of 2D views, the image server comprising an input for obtaining the plurality of 2D views, a transmitter for transmitting the plurality of 2D views in a transmission order to the image client, and a processor for (i) analyzing contents of the plurality of 2D views for obtaining a respective plurality of view properties, and (ii) establishing the transmission order in dependence on the plurality of view properties for transmitting the plurality of 2D views based on a decreasing degree of information content of the 2D views.

In a further aspect of the invention, a workstation and an imaging apparatus are provided comprising the image server set forth. In a further aspect of the invention, a system is provided comprising the image server and the image client set forth.

In a further aspect of the invention, a method is provided for transmitting, to an image client, a plurality of two-dimensional [2D] views of three- or higher dimensional [3D] image data for enabling a user operating the image client to navigate through the 3D image data by viewing different ones of the plurality of 2D views, the method comprising obtaining the plurality of 2D views, analyzing contents of the plurality of 2D views for obtaining a respective plurality of view properties, and establishing, in dependence on the plurality of view properties, a transmission order for transmitting the plurality of 2D views to the image client based on a decreasing degree of information content of the 2D views.

In a further aspect of the invention, a computer program product is provided comprising instructions for causing a processor system to perform the method set forth.

The plurality of 2D views enables a user to navigate through the 3D image data on the image client. For that purpose, the image client may show different ones of the plurality of 2D views, e.g., for simulating movement of a virtual camera through the 3D image data. For transmitting the plurality of 2D views to the image client, said 2D views are first obtained from the 3D image data. The 3D image data may be constituted by the plurality of 2D views, e.g., the 3D image data may be formed by a stack of 2D axial slices obtained by Magnetic Resonance Imaging (MRI), Computed Tomography (CT) or similar imaging technique. In this case, the plurality of 2D views is obtained inherently together with the 3D image data. Alternatively, or additionally, obtaining the plurality of 2D views may involve performing computations on the 3D image data. For example, a so-termed multi-planar reconstruction may be applied to volumetric 3D image data for obtaining a plurality of 2D slices that intersect the 3D image data along a plurality of intersection planes. Consequently, each of the plurality of 2D views shows at least a portion of the 3D image data.

The contents of the plurality of 2D views is analyzed in order to generate a respective plurality of view properties, i.e., a view property for each of the plurality of 2D views. Each of the plurality of view properties is an objective measure relating to the contents of the plurality of 2D views. The plurality of view properties is used in determining the transmission order in which the plurality of 2D views is to be transmitted to the image client. In particular, the plurality of view properties is used to establish a transmission order that provides a reduction of the degree of information content over time when sequentially transmitting the plurality of 2D views to the image client. Here, the term information content refers to the information that is provided to the user when the user views the respective 2D view, i.e., the information that is available from the respective 2D view for the user.

The present invention is partially based on the recognition that, when the transmission of the plurality of 2D views to the image client is constrained by, e.g., the available bandwidth of the communication channel, etc., the user generally prefers to first receive those of the plurality of 2D views that are of most relevance to the user. A reason for this is that, otherwise, waiting for all of the plurality of 2D views to be received may be burdensome. The relevance is typically associated with the information content of each one of the plurality of 2D views. Therefore, when one of the plurality of 2D views provides little or no information to the user, said 2D view is most likely also of little or no relevance to the user. In contrast, when one of the plurality of 2D views provides a high amount of information to the user, said 2D view is most likely of high relevance to the user.

The measures of the present invention have the effect that the contents of the plurality of 2D views is analyzed in order to establish a plurality of view properties as objective measures of the information that the user may gather from the plurality of 2D views. By establishing the transmission order in dependence on the plurality of view properties, those of the plurality of 2D views that have a high degree of information content, i.e., deemed to be of relevance to the user, are transmitted before those that have a low or medium degree of information content, i.e., deemed to be of less relevance to the user.

Advantageously, the user may not need to wait for all of the plurality of 2D views to be transmitted to the image client as those of the plurality of 2D views that are of relevance to the user are received first. Advantageously, the user may more quickly inspect the 3D image data, i.e., without having to wait for all of the plurality of 2D views to be received.

Optionally, the processor is arranged for generating each of the plurality of view properties as being indicative of the information content of a respective one of the plurality of 2D views. Each of the plurality of view properties is thus indicative of the information that the user may gather from viewing the respective one of the plurality of 2D views. Advantageously, the decreasing degree of information content may be easily established by sorting the plurality of view properties according to value.

Optionally, the processor is arranged for generating each of the plurality of view properties as being indicative of an entropy of the respective one of the plurality of 2D views. The entropy is a measure of information content as is known from the technical field of information theory. This optional aspect of the present invention is based on the recognition that the entropy of one of the plurality of 2D views is well correlated with the information that the user may gather from said 2D view. Thus, by determining the entropy of the respective one of the plurality of 2D views, an objective measure is obtained that correlates well with the information that the user may gather from said 2D view.

Optionally, the processor is arranged for performing predictive encoding of the plurality of 2D views, and performing a rate distortion analysis of the predictive encoding for generating each of the plurality of view properties as being indicative of a rate distortion incurred by an omission of the respective one of the plurality of 2D views in the predictive encoding. The rate distortion incurred by the omission of one of the plurality of 2D views in the predictive encoding of the plurality of 2D views is a measure of entropy that correlates well with the information that the user may gather from said 2D view.

Optionally, the processor is arranged for iteratively establishing the transmission order by establishing, after having established the position of a previous one of the plurality of 2D views in the transmission order, the position of a next one of the plurality of 2D views in the transmission order in dependence on the rate distortion incurred by a joint omission of said previous one and said next one of the plurality of 2D views in the predictive encoding.

This optional aspect of the present invention is based on the recognition that information provided to the user by a previously transmitted one of the plurality of 2D views affects the information obtained by the user from the remaining ones of the plurality of 2D views. For example, the additional information provided by one of the plurality of 2D views that is immediately adjacent to said previously transmitted one may be negligible in case both 2D views are highly similar. However, if both 2D views differ drastically, the additional information provided by said adjacent 2D view may be significant. This recognition is taken into account by determining the rate distortion incurred by the joint omission of said previously transmitted 2D view and a potential next one of the plurality of 2D views.

Optionally, the 3D image data comprises a region of interest, and the processor is arranged for detecting the region of interest in the plurality of 2D views for emphasizing the region of interest in the rate distortion analysis. A region of interest may be of particular relevance to the user. By detecting the region of interest in the plurality of 2D views and emphasizing the region of interest in the rate distortion analysis, those of the plurality of 2D views that comprise the region of interest are preferred in the transmission order. Advantageously, the user may more quickly inspect the region of interest, i.e., without having to wait for all of the plurality of 2D views to be received.

Optionally, the 3D image data comprises a region of interest, and the processor is arranged for detecting the region of interest in the plurality of 2D views for generating each of the plurality of view properties as being indicative of the visibility of the region of interest in the respective one of the plurality of 2D views. By detecting the region of interest in the plurality of 2D views and generating the view properties in dependence thereon, those of the plurality of 2D views that comprise the region of interest are preferred in the transmission order. Advantageously, the user may more quickly inspect the region of interest, i.e., without having to wait for all of the plurality of 2D views to be received. Optionally, the image client is arranged for performing a view interpolation of received 2D views for enabling the user to navigate through the 3D image data before receiving all of the plurality of 2D views, and the processor is arranged for simulating the view interpolation for obtaining a plurality of interpolated 2D views, establishing a respective plurality of interpolation errors by comparing the plurality of interpolated 2D views to respective ones of the plurality of 2D views, and establishing the transmission order in further dependence on the plurality of interpolation errors for transmitting the plurality of 2D views further based on a decreasing interpolation error.

The interpolation error is indicative of how closely an interpolated 2D view resembles the respective one of the plurality of 2D views. If the interpolation error is low, the interpolated 2D view resembles the respective one of the plurality of 2D views and thus provides same or similar information to the user. In contrast, if the interpolation error is high, the interpolated 2D view does not resemble the respective one of the plurality of 2D views and thus provides different or no information to the user. By establishing the transmission order based on the plurality of interpolation errors, the plurality of 2D views can be transmitted such that those of the plurality of 2D views which have not been transmitted yet, and of which the respective interpolated 2D view provides a high interpolation error, are transmitted before those of which the respective interpolated 2D view provides a low interpolation error. Advantageously, the user is provided faster with a faithful reproduction of the 3D image data.

Optionally, the processor is arranged for establishing, after having established the position of a previous one of the plurality of 2D views in the transmission order, the position of a next one of the plurality of 2D views in the transmission order in further dependence on a distance between said previous one and said next one of the plurality of 2D views within the 3D image data. The distance between two of the plurality of 2D views within the 3D image data is typically indicative of whether or not both 2D views provide the same or similar information to the user. A large distance may be indicative of, e.g., both 2D views showing different information to the user. This may or may not be preferred by the user, i.e., the user may prefer the next one of the plurality of 2D views to provide different information, or rather similar information. By establishing the transmission order on the basis of said distance, the aforementioned preference of the user can be taken into account.

Optionally, the transmitter is arranged for establishing a latency of transmitting the plurality of 2D views to the image client, and the processor is arranged for establishing the transmission order in further dependence on the latency. The latency determines how long the transmission of the plurality of 2D views to the image client takes. The latency may be determined by, e.g., the currently available bandwidth of the

communication channel, the amount of data that is actually transmitted per 2D view, the load, responsiveness, CPU power etc. of the image server and/or image client. When the latency is relatively low, the user may not mind first receiving 2D views that provide less information, as receiving all of the plurality of 2D views takes little time. In contrast, when the latency is relatively high, the user may especially prefer to first receive those of the plurality of 2D views which are most relevant to the user. By establishing the transmission order on the basis of the latency, the above can be taken into account.

Optionally, the transmitter is further arranged for (i) transmitting the plurality of 2D views to the image client with a degree of parallelism, and (ii) establishing the degree of parallelism in dependence on the latency. When the latency is relatively low, it may be advantageous to transmit two or more of the plurality of 2D views in parallel. The

transmitting thus occurs with a degree of parallelism that is based on the latency.

It will be appreciated by those skilled in the art that two or more of the above- mentioned embodiments, implementations, and/or aspects of the invention may be combined in any way deemed useful.

Modifications and variations of the image client, the workstation, the imaging apparatus, the system, the method, and/or the computer program product, which correspond to the described modifications and variations of the image server, can be carried out by a person skilled in the art on the basis of the present description.

A person skilled in the art will appreciate that the method may be applied to multi-dimensional image data, e.g. three-dimensional (3D) or four-dimensional (4D) image data. A dimension of the multi-dimensional image data may relate to time. For example, 3D image data may comprise a time domain series of 2D images. The image data may be medical image data, acquired by various acquisition modalities such as, but not limited to, standard X-ray Imaging, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US), Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), and Nuclear Medicine (NM).

The invention is defined in the independent claims. Advantageous embodiments are defined in the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter. In the drawings,

Fig. 1 shows a system comprising an image server according to the present invention and an image client;

Fig. 2 shows a method according to the present invention;

Fig. 3 shows a computer readable medium comprising a computer program product according to the present invention;

Fig. 4 shows 3D image data comprised of a plurality of 2D views;

Fig. 5a shows a transmission order of the plurality of 2D views;

Fig. 5b shows a transmission order according to the present invention;

Fig. 6a shows a plurality of view properties in accordance with the transmission order of Fig. 5a;

Fig. 6b shows the plurality of view properties in accordance with the transmission order of Fig. 5b.

DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 shows a system 100 comprising an image server 110 and an image client 150. The image server 110 is shown to be connected to the image client 150 via a communications channel 132 for transmitting, to the image client 150, a plurality of two- dimensional [2D] views 122 of three- or higher dimensional [3D] image data for enabling a user operating the image client 150 to navigate through the 3D image data by viewing consecutive ones of the plurality of 2D views. For showing said 2D views to the user, the image client 150 may comprise a display 160. The image server 110 comprises an input 120 for obtaining the plurality of 2D views 122, and a transmitter 130 for transmitting the plurality of 2D views 122 in a transmission order 142 to the image client 150. The input 120 is shown to be connected to the transmitter 130 for providing the plurality of 2D views 122 thereto. Moreover, the transmitter is shown to be connected to the communications channel 132 for transmitting the plurality of 2D views 122 to the image client 150.

The image server 110 further comprises a processor 140. The processor 140 is arranged for analyzing contents of the plurality of 2D views 122 for obtaining a respective plurality of view properties. The processor 140 is shown to be connected to the input 120 for obtaining the plurality of 2D views 122. Moreover, the processor 140 is arranged for establishing the transmission order 142 in dependence on the plurality of view properties for transmitting the plurality of 2D views 122 based on a decreasing degree of information content of the 2D views. The processor 140 is shown to be connected to the transmitter 130 for providing the transmission order 142 to the transmitter 130. The transmitter 130 thus obtains the plurality of 2D views 122 from the input 120 and the transmission order 142 from the processor 140, enabling the transmitter 130 to transmit the plurality of 2D views 122 in the transmission order 142 via the communication channel 132 to the image client 150.

Fig. 2 shows a method 200 of transmitting, to an image client, a plurality of 2D views of 3D image data for enabling a user operating the image client to navigate through the 3D image data by viewing different ones of the plurality of 2D views. The method 200 comprises a first step 210, titled "OBTAINING 2D VIEWS", comprising obtaining the plurality of 2D views. The method 200 further comprises a second step 220, titled

"OBTAINING VIEW PROPERTIES", comprising analyzing contents of the plurality of 2D views for obtaining a respective plurality of view properties. The method 200 further comprises a third step 230, titled "ESTABLISHING TRANSMISSION ORDER", comprising establishing, in dependence on the plurality of view properties, a transmission order for transmitting the plurality of 2D views to the image client based on a decreasing degree of information content of the 2D views. Although not shown in Fig. 2, the method 200 may comprise a fourth step, titled "TRANSMITTING 2D VIEWS", comprising transmitting the plurality of 2D views in the transmission order to the image client. The method 200 may correspond to an operation of the image server 110, and will be further explained with reference to said operation. It will be appreciated, however, that the method 200 may be performed independently of said image server 110, e.g., using another image server or device.

Fig. 3 shows a computer readable medium 240 comprising a computer program product 242 for causing a processor system to perform the method according to the present invention. The computer program product 242 may be comprised on the computer readable medium 240 as a series of machine readable marks, e.g., a series of elements having different electrical, e.g., magnetic, mechanical or optical properties.

Fig. 4 shows 3D image data 300, the 3D image data 300 being constituted by a plurality of 2D views 301-305. A first one 301 of the plurality of 2D views is shown only as an otherwise transparent outline in order to show a second one 302 of said 2D views. The second one 302 of said 2D views is shown to comprise a region of interest 310. Moreover, a third one to fifth one 303-305 of said 2D views are shown stacked behind the first one 301 and second one 302 of said 2D views. The plurality of 2D views 301-305 may correspond to slices through volumetric image data, with a stack formed by the slices constituting said 3D image data 300. The image client 150 may be arranged for enabling a user to navigate through the stack of slices, with the navigating involving the displaying of different ones of the slices when the user moves forward or backward through said stack of slices.

Fig. 5a shows a possible transmission order 320 of the plurality of 2D views

301-305 from the image server 110 to the image client 150. Here, the transmission order 320 is schematically indicated by the relative horizontal position of each of the plurality of 2D views 301-305, with a left-hand position indicating an earlier transmission and a right-hand position indicating a later transmission. Therefore, when transmitting the plurality of 2D views 301-305 in the transmission order 320 of Fig. 5a, the first one 301 of the plurality of 2D views 301-305 will be transmitted first, the second one 302 of said 2D views 301-305 will be transmitted next, etc. Accordingly, the image client 150 will receive the first one 301 of the plurality of 2D views 301-305 first, the second one 302 of the said 2D views next, etc. It is noted that the transmission order 320 of Fig. 5a may correspond to a position of each of the plurality of 2D views 301-305 in the 3D image data, e.g., a position within a stack.

Fig. 5b shows a transmission order 330 of the plurality of 2D views 301-305 according to the present invention. For obtaining said transmission order 330, the processor 140 may be arranged for generating for each of the plurality of 2D views 301-305 a view property that is indicative of the information content of the respective one of the plurality of 2D views 301-305. Thus, the plurality of view properties is, for each of the plurality of 2D views 301-305, indicative of the information content provided by said 2D view to the user.

Fig. 6a shows an example of the plurality of view properties 321 in accordance with the transmission order 320 of Fig. 5a. Here, along the horizontal axis, each of the plurality of 2D views 301-305 is set out in accordance with said transmission order. The vertical axis indicates a view property value 322. It is visible that the view property corresponding to the third one 303 of the plurality of 2D views 301-305 has the highest value, followed by the view properties corresponding to the second one 302 and the fourth one 304 of the plurality of 2D views 301-305, lastly followed by the view properties corresponding to the first one 301 and the fifth one 305 of the plurality of 2D views 301-305. Fig. 6b shows the plurality of view properties 331 in accordance with the transmission order 330 of Fig. 5b. The transmission order 330 may be obtained by sorting the plurality of 2D views 301-305 in accordance with a decreasing view property value 322. As a result, Fig. 6b shows the transmission order 330 providing a decrease in value 322 of the plurality of view properties 331 along the horizontal axis, i.e., along the transmission order 330. Each of the plurality of view properties may be indicative of an entropy of the respective one of the plurality of 2D views 301-305. For obtaining the entropy of the respective one of the plurality of 2D views 301-305, the processor 140 may be arranged for applying data compression to the respective 2D view. The data compression may comprise lossless data compression. Lossless data compression refers to a manner of encoding said 2D view in which the 2D view can be exactly reconstructed from the compressed data. General examples of lossless data compression are, e.g., Run-length encoding (RLE) or Lempel-Ziv- Welch (LZW) encoding. Examples of lossless data compression that may be used specifically for compressing 2D views are, e.g., PNG (Portable Network Graphics) or TIFF (Tagged Image File Format). Of course, other suitable data compressions may also be used. The data reduction obtained by applying data compression to each of the plurality of 2D views 301- 305 is indicative of the entropy of said 2D view, in that a high data reduction is indicative of low entropy of said 2D view, whereas a low data reduction is indicative of high entropy.

Alternatively, or additionally, the data compression may comprise lossy data compression. Lossy data compression refers to a manner of encoding the respective one of the plurality of 2D views 301-305 in which the 2D view cannot be exactly reconstructed from the compressed data. When used for compressing 2D views, lossy data compression typically achieves a higher data reduction by exploiting certain perceptual attributes of the Human Visual System (HVS), e.g. a lower sensitivity to high frequency detail present in hues or saturation than to luminance. Examples of lossy data compression that may be used for compressing 2D views are, e.g., JPEG (Joint Photographic Experts Group). Also here, the data reduction obtained is indicative of the entropy of said 2D view.

The plurality of view properties may be constituted by the data reduction obtained for each of the plurality of 2D views 301-305. Establishing the transmission order 330 may involve sorting the plurality of view properties according to the data reduction, and sorting the plurality of 2D views 301-305 in accordance with the sorting of the plurality of view properties. The order of the plurality of 2D views 301-305 obtained thereby may constitute the transmission order 330, wherein those of the plurality of 2D views 301-305 that obtain the highest data reduction are transmitted last, i.e., are positioned last in the transmission order, whereas those of the plurality of 2D views 301-305 that obtain the lowest data reduction are transmitted first, i.e., are positioned first in the transmission order.

In the example of Figs. 4, 5a and 5b, the 3D image data 300 comprises a region of interest, e.g., an object located therein, and the plurality of views 301-305 constitute intersections of the 3D image data 300 along a plurality of intersection planes. As a result, a second one 302 of the plurality of 2D views 301-305 in Fig. 5a shows the region of interest 309 being intersected partially, i.e., yielding a small cross-section of the region of interest 309. A similar intersection is shown in a fourth one 304 of said 2D views, i.e., showing a small cross-section of the region of interest 311. A third one 303 of said 2D views shows an intersection through the center of the region of interest 310, yielding a large cross-section of the region of interest 310. Moreover, a first one 301 and a fifth one 305 of said 2D views do not intersect, and consequently do not show the region of interest 309-311 at all.

The large cross-section of the region of interest 310 may constitute a high degree of information content for the user, i.e., may be of relevance to the user. At the same time, the presence of the large cross-section of the region of interest 310 in the third one 303 of said 2D views may cause the data compression, when compressing the third one 303 of said 2D views, to achieve only a low data reduction. A reason for this may be that the large cross-section of the region of interest 310 contributes to the third one 303 of said 2D views having a high entropy, inherently resulting in less data reduction. It is noted that, in particular, uncorrected structures, e.g., uncorrected edges, textures or other details, yield a high entropy, whereas homogenous areas or correlated structures, e.g., repetitive structures, yield a low entropy. Furthermore, the small cross-sections of the region of interest 309, 311 within the second one 302 and fourth one 304 of said 2D views may result in these 2D views having a medium entropy, whereas the lack of a region of interest in the first one 301 and fifth one 305 of said 2D views may result in said views having a low entropy.

Consequently, when applying data compression to each respective one of the plurality of 2D views 301-305, a high data compression may be obtained for the first one 301 and the fifth one 305 of the plurality of 2D views, a low data compression may be obtained for the third one 303 of said 2D views, and a medium data compression may be obtained for the second one 302 and the fourth one 304 of said 2D views. Therefore, when ranking the plurality of 2D views 301-305 from lowest data reduction to highest data reduction, the transmission order 330 as depicted in Fig. 5b may be obtained.

It is noted that each of the plurality of view properties may be indicative of the entropy of the respective one of the plurality of 2D views 301-305 without explicitly applying data compression to each of the plurality of 2D views 301-305. For example, the presence of mid- and/or high-frequency detail in each of said 2D views 301-305 may be indicative of its entropy. Thus, instead of applying data compression, a high-frequency or band-pass filter may be applied to each of the plurality of 2D views 301-305. Moreover, the processor 140 may be arranged for applying data compression to the plurality of 2D views 301-305 by performing predictive encoding of said 2D views 301-305. Here, the term predictive encoding refers to data compression which, in order to obtain a higher data reduction, attempts to predict at least portions of one of the plurality of 2D views 301-305 from others of said 2D views. As a result, similarities between two or more of the plurality of 2D views 301-305 are exploited for obtaining a higher data reduction. It is noted that predictive encoding of 2D views is known, e.g., from the technical field of video compression. In particular, encoding standards such as Moving Picture Experts Group (MPEG) 1, 2 and 4 comprise measures for the predictive encoding of video frames. For example, MPEG 1 involves the usage of so-termed P-frames and B-frames which only store differences between so-termed anchor frames in order to obtain a higher data reduction.

The processor 140 may be further arranged for performing a rate distortion analysis of the predictive encoding for generating each of the plurality of view properties as being indicative of a rate distortion incurred by an omission of the respective one of the plurality of 2D views 301-305 in the predictive encoding. It is noted that rate distortion analysis is known from the technical fields of information theory and video compression. By applying the rate distortion analysis, a 2D view is selected amongst the plurality of 2D views 301-305 as being first in the transmission order 330 so that the overall error incurred due to omitting the remaining ones of the plurality of 2D views 301-305 is minimized. The above selection process may then be repeated until only one 2D view remains amongst the plurality of 2D views 301-305, which is then selected as being last in the transmission order 330.

For example, a region of interest 309-311, e.g., an object such as an organ, may be segmented in each of the plurality of 2D views 301-305. Hence, the surface of the region of interest 309-311 may be segmented in all the 2D views and may thus be available, e.g., in 2D or 3D coordinates. The surface of the region of interest 309-311 may then be reconstructed using contours of those of the plurality of 2D views 301-305 that are already selected, i.e., included in the transmission order 330. The reconstruction may employ a so- termed B-spline interpolation. The reconstruction may then be compared to the actual surface of the region of interest 309-311 to determine an error. Accordingly, a 2D view i may then be selected amongst the remaining ones of the plurality of 2D views 301-305, i.e., those not yet included in the transmission order 330, such that the error in the reconstruction which is incurred by omitting said 2D view in the transmission is higher than in the case of omitting any of the other 2D views. After selection and inclusion of the 2D view i in the transmission order 330, a further 2D view j may then be selected such that the error incurred by jointly omitting 2D views i and j in the transmission is higher than by jointly omitting 2D view i and any other of the remaining 2D views. Yet a further 2D view k may then be selected such that said error incurred by jointly omitting 2D views i,j and k in the transmission is higher than by jointly omitting 2D views i and j and any other 2D view. The above process may continue accordingly until all of the 2D views have been included in the transmission order 330.

Alternatively, the selection may not take into account the joint omission of 2D views. For example, the 2D view i may be selected amongst the remaining ones of the plurality of 2D views 301-305 such that the error in the reconstruction which is incurred by omitting said 2D view in the transmission is higher than by omitting any of the other 2D views. A further 2D view j may then be selected amongst the remaining ones of the plurality of 2D views 301-305, i.e., excluding 2D view i, such that the error incurred by omitting 2D view j is smaller than by omitting any other of the remaining 2D views, etc.

Alternatively, when assuming that the region of interest 309-311 comprises a smooth surface, the errors incurred due to not transmitting 2D view m and its adjacent 2D view m+1 will be almost comparable. Assuming that a 2D view n has been included in the transmission order 330, it may be decided to omit p successive 2D views adjacent to 2D view n in the 3D image data 300 to arrive at a 2D view n+p to be included next in the transmission order 330. This may be done by calculating a slope of a rate distortion error curve by varying p from 1 onwards. The slope of said error curve is expected to decrease till a certain value of p and is then expected to increase again. The 2D view n+p where the slope increases may then be selected as being included next in the transmission order 330. The above selection process may be repeated from n+p onwards. The selection process may be iterated after every sweep through the remaining 2D views, with every time, i.e., in every iteration, excluding those 2D views that have already been included in the transmission order 330.

The processor 140 may be arranged for detecting the region of interest 309-

311 in the plurality of 2D views 301 -305. Detecting the region of interest 309-311 may comprise performing an image segmentation of the region of interest 309-311 in each of the plurality of 2D views 301-305. Image segmentation is known from the technical fields of image processing and image analysis. As a result, it may be detected that the first one 301 and the last one 305 of said 2D views 301-305 do not comprise the region of interest, whereas the second one 302 and the fourth one 304 comprise a small cross-section of the region of interest 309, 311 , and the third one 303 comprises a large cross-section of the region of interest 310. As a result, presence, and possibly location, of the region of interest 309-311 in each of the plurality of 2D views 301-305 may be known to the processor 140. The processor 140 may be further arranged for emphasizing the region of interest 309-311 in the rate distortion analysis. Emphasizing the region of interest 309-311 in the rate distortion analysis may comprise targeting or biasing the rate distortion analysis to the region of interest 309-311 in each of the plurality of 2D views 301-305. Each of the plurality of view properties may thus be indicative of the rate distortion in the predictive encoding of the region of interest incurred by an omission of the respective one of the plurality of 2D views 301-305 in said predictive encoding.

Alternatively, or additionally, the processor 140 may be arranged for detecting the region of interest 309-311 in the plurality of 2D views 301-305 in order to generate each of the plurality of view properties as being indicative of the visibility of the region of interest 309-311 in the respective one of the plurality of 2D views 301-305. The visibility of the region of interest 309-311 may be at least partially determined by the size of the region of interest 309-311 in said 2D views 301-305. Detecting the region of interest 309-311 in said 2D views 301-305 may thus comprise, in addition to determining the presence of the region of interest 309-311 in said 2D views 301-305, determining the size of the region of interest 309-311. As a result, when ranking the plurality of 2D views 301-305 in accordance with the plurality of view properties, the same or a similar transmission order as depicted in Fig. 5b may be obtained in which the third one 303 of said 2D views 301-305 is transmitted first owing to the third one 303 of said 2D views 301-305 comprising a large cross-section of the region of interest 310, whereas the fifth one 305 of said 2D views 301-305 is transmitted last owing to the fifth one 305 of said 2D views 301-305 not comprising the region of interest at all.

The image client 150 may be arranged for performing a view interpolation of received 2D views for enabling the user to navigate through the 3D image data before receiving all of the plurality of 2D views 301-305. The view interpolation results in a plurality of interpolated 2D views which may correspond to those of the plurality of 2D views that have not been received yet by the image client 150. Moreover, the processor 140 may be arranged for simulating the view interpolation for obtaining the plurality of interpolated 2D views. Simulating the view interpolation may comprise performing the same view interpolation as that of the image client 150. Simulating the view interpolation may also comprise performing the view interpolation with, e.g., a lesser degree of accuracy or quality.

The processor 140 may be further arranged for establishing a plurality of interpolation errors by comparing the plurality of interpolated 2D views to respective ones of the plurality of 2D views. For example, the processor 140 may establish the plurality of interpolation errors by determining a difference between the plurality of interpolated 2D views and the respective ones of the plurality of 2D views, i.e., those that have not yet been received by the image client 150. Each of the plurality of interpolation errors is thus indicative of how well each of the plurality of interpolated 2D views matches the respective one of the plurality of 2D views 301-305. The processor 140 may be further arranged for establishing the transmission order in further dependence on the plurality of interpolation errors. For example, the processor 140 may be arranged for ranking those of the plurality of 2D views 301-305 that have not yet been transmitted in accordance with a decreasing sum of entropy and interpolation error, or with a decreasing sum of visibility of the region of interest and interpolation error, etc. Here, a high interpolation error may result in transmitting the respective one of the plurality of 2D views 301-305 early so as to prevent the user from viewing interpolated 2D views that do not correspond well to the respective one of the plurality of 2D views 301-305. Accordingly, the processor 140 may establish a transmission order 330 for early transmitting those of the plurality of 2D views 301-305 which provide high information content to the user and of which an interpolated version yields a high interpolation error, i.e., does not correspond well to the respective original 2D view.

The processor 140 may be arranged for establishing, after having established the position of a previous one of the plurality of 2D views 301-305 in the transmission order 330, the position of a next one of the plurality of 2D views 320, 330 in the transmission order in further dependence on a distance between said previous one and said next one of the plurality of 2D views 301-305 within the 3D image data. The distance may be a geometrical distance. For example, when the 3D image data is volumetric image data and the plurality of 2D views are intersections of the volumetric image data along a plurality of intersection planes, the distance may be a geometrical distance between the intersection planes of the previously transmitted 2D view and a potentially next transmitted 2D view. The distance may also relate to the number of intermediate 2D views present in the 3D image data 300 between the previously transmitted 2D view and the potentially next transmitted 2D view. The processor 140 may be arranged for ranking those of the plurality of 2D views 301-305 that have not been transmitted yet in accordance with a decreasing sum of entropy and distance, or a decreasing sum of visibility of the region of interest and distance, etc. Here, a large distance between both 2D views may be a reason for transmitting the potential next one of the plurality of 2D views 301-305 early so as to provide the user as soon as possible with a coarse overview of the 3D image data. Alternatively, a small distance between both 2D views may be a reason for transmitting the potential next one of the plurality of 2D views 301-305 early so as to provide the user as soon as possible with 2D views that are adjacent to the previously transmitted 2D view.

The transmitter 130 may be arranged for establishing a latency of transmitting the plurality of 2D views 301-305 to the image client 150, e.g., by measuring a time between transmission of one of the plurality of 2D views 301-305 by the image server 110 and receipt of said 2D view by the image client 150. The latency may relate to, or be indicative of, a current bandwidth of the communication channel, an amount of data that is transmitted per 2D view, the load, responsiveness, CPU of the image client 150 and/or the image server 110. Moreover, the processor 140 may be arranged for establishing the transmission order 330 in further dependence on the latency. For example, in case the latency is low, those of the plurality of 2D views 301-305 that are adjacent to the previously transmitted 2D view may be penalized. Moreover, the transmitter 130 may be further arranged for transmitting the plurality of 2D views 301-305 to the image client 150 with a degree of parallelism. The degree of parallelism may be established in dependence on the latency. Here, the term parallelism refers to two or more of the plurality of 2D views 301-305 being transmitted in parallel, e.g. over parallel communication channels. In case the latency is low, this may be indicative of a possibility to transmit two or more of the plurality of 2D views 301-305. Thus, the transmitter may be arranged for increasing a degree of parallelism, e.g., from sequential transmission to the transmission of two of the plurality of 2D views 301-305 in parallel, in dependence on the latency. Here, the transmission order established by the processor 140 may be the transmission order of those of the plurality of 2D views 301-305 that are transmitted over a single one of the parallel communication channels.

It is noted that the present invention does not interfere with established standards for transmitting 2D views, i.e., it may be applied to or within a framework of any existing standards for transmitting 2D views of 3D image data. Moreover, the present invention is independent of the data compression used for transmitting the plurality of 2D views to the image client. The present invention may be applied to so-termed distributed or client/server-based applications. In particular, it can be applied in thin- or zero-clients of web-applications. It may also be applied in established products that transmit image information via network nodes using the DICOM standard, e.g. PACS systems.

It will be appreciated that the invention also applies to computer programs, particularly computer programs on or in a carrier, adapted to put the invention into practice. The program may be in the form of a source code, an object code, a code intermediate source and object code such as in a partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. It will also be appreciated that such a program may have many different architectural designs. For example, a program code implementing the functionality of the method or system according to the invention may be sub-divided into one or more sub-routines. Many different ways of distributing the functionality among these sub-routines will be apparent to the skilled person. The subroutines may be stored together in one executable file to form a self-contained program. Such an executable file may comprise computer-executable instructions, for example, processor instructions and/or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the sub-routines may be stored in at least one external library file and linked with a main program either statically or dynamically, e.g. at run-time. The main program contains at least one call to at least one of the sub-routines. The sub-routines may also comprise function calls to each other. An embodiment relating to a computer program product comprises computer-executable instructions corresponding to each processing step of at least one of the methods set forth herein. These instructions may be sub-divided into sub- routines and/or stored in one or more files that may be linked statically or dynamically.

Another embodiment relating to a computer program product comprises computer-executable instructions corresponding to each means of at least one of the systems and/or products set forth herein. These instructions may be sub-divided into sub-routines and/or stored in one or more files that may be linked statically or dynamically.

The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a storage medium, such as a ROM, for example, a CD ROM or a semiconductor ROM, or a magnetic recording medium, for example, a hard disk. Furthermore, the carrier may be a transmissible carrier such as an electric or optical signal, which may be conveyed via electric or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such a cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted to perform, or to be used in the performance of, the relevant method.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. Image server (110) for transmitting, to an image client (150), a plurality of two-dimensional [2D] views (122, 301-305)) of three- or higher dimensional [3D] image data (300) for enabling a user operating the image client to navigate through the 3D image data by viewing different ones of the plurality of 2D views, the image server comprising:

- an input (120) for obtaining the plurality of 2D views;

a transmitter (130) for transmitting the plurality of 2D views in a transmission order (142, 330) to the image client; and

a processor (140) for (i) analyzing contents of the plurality of 2D views for obtaining a respective plurality of view properties, and (ii) establishing the transmission order in dependence on the plurality of view properties for transmitting the plurality of 2D views, based on a decreasing degree of information content of the 2D views.

2. Image server (110) according to claim 1, wherein the processor (140) is arranged for generating each of the plurality of view properties as being indicative of the information content of a respective one of the plurality of 2D views (122, 301-305).

3. Image server (110) according to claim 2, wherein the processor (140) is arranged for generating each of the plurality of view properties as being indicative of an entropy of the respective one of the plurality of 2D views (122, 301-305).

4. Image server (110) according to claim 3, wherein the processor (140) is arranged for:

performing predictive encoding of the plurality of 2D views (122, 301-305); and

- performing a rate distortion analysis of the predictive encoding for generating each of the plurality of view properties as being indicative of a rate distortion incurred by an omission of the respective one of the plurality of 2D views in the predictive encoding.

5. Image server (110) according to claim 4, wherein the processor (140) is arranged for iteratively establishing the transmission order (142, 330) by establishing, after having established a position of a previous one of the plurality of 2D views (122, 301-305) in the transmission order, the position of a next one of the plurality of 2D views in the transmission order in dependence on the rate distortion incurred by a joint omission of said previous one and said next one of the plurality of 2D views in the predictive encoding.

6. Image server (110) according to claim 4, wherein the 3D image data (300) comprises a region of interest (310), and wherein the processor (140) is arranged for detecting the region of interest in the plurality of 2D views (122, 301-305) for emphasizing the region of interest in the rate distortion analysis.

7. Image server (110) according to claim 2, wherein the 3D image data (300) comprises a region of interest (310), and wherein the processor (140) is arranged for detecting the region of interest in the plurality of 2D views (122, 301-305) for generating each of the plurality of view properties as being indicative of the visibility of the region of interest in the respective one of the plurality of 2D views.

8. Image server (110) according to claim 1, wherein the image client (150) is arranged for performing a view interpolation of received 2D views for enabling the user to navigate through the 3D image data (300) before receiving all of the plurality of 2D views (122, 301-305), and wherein, in the image server, the processor (140) is arranged for:

simulating the view interpolation for obtaining a plurality of interpolated 2D views;

- establishing a respective plurality of interpolation errors by comparing the plurality of interpolated 2D views to respective ones of the plurality of 2D views; and

establishing the transmission order (142, 330) in further dependence on the plurality of interpolation errors for transmitting the plurality of 2D views further based on a decreasing interpolation error.

9. Image server (110) according to claim 1, wherein the processor (140) is arranged for establishing, after having established the position of a previous one of the plurality of 2D views (122, 301-305) in the transmission order (142, 330), the position of a next one of the plurality of 2D views in the transmission order in further dependence on a distance between said previous one and said next one of the plurality of 2D views within the 3D image data (300).

10. Image server (110) according to claim 1, wherein the transmitter (130) is arranged for establishing a latency of transmitting the plurality of 2D views (122, 301-305) to the image client (150), and wherein the processor (140) is arranged for establishing the transmission order (330) in further dependence on the latency.

11. Image server (110) according to claim 10, wherein the transmitter (130) is further arranged for (i) transmitting the plurality of 2D views (122, 301-305) to the image client (150) with a degree of parallelism, and (ii) establishing the degree of parallelism in dependence on the latency.

12. Workstation or imaging apparatus comprising the image server of claim 1.

13. System (100) comprising the image server of claim 1 and the image client.

14. A method (200) of transmitting, to an image client, a plurality of two- dimensional [2D] views of three- or higher dimensional [3D] image data for enabling a user operating the image client to navigate through the 3D image data by viewing different ones of the plurality of 2D views, the method comprising:

obtaining (210) the plurality of 2D views;

analyzing (220) contents of the plurality of 2D views for obtaining a respective plurality of view properties; and

- establishing (230), in dependence on the plurality of view properties, a transmission order for transmitting the plurality of 2D views to the image client, based on a decreasing degree of information content of the 2D views.

15. A computer program product (242) comprising instructions for causing a processor system to perform the method according to claim 14.