EP0946919A1 - Real-time remote inspection of high-resolution images - Google Patents
Real-time remote inspection of high-resolution imagesInfo
- Publication number
- EP0946919A1 EP0946919A1 EP97947822A EP97947822A EP0946919A1 EP 0946919 A1 EP0946919 A1 EP 0946919A1 EP 97947822 A EP97947822 A EP 97947822A EP 97947822 A EP97947822 A EP 97947822A EP 0946919 A1 EP0946919 A1 EP 0946919A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- image
- images
- hierarchy
- level
- areas
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
- 238000007689 inspection Methods 0.000 title description 5
- 238000000034 method Methods 0.000 claims abstract description 50
- 230000005540 biological transmission Effects 0.000 claims abstract description 29
- 230000003139 buffering effect Effects 0.000 claims abstract description 3
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- 230000006837 decompression Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000013155 cardiography Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/21—Intermediate information storage
- H04N1/2166—Intermediate information storage for mass storage, e.g. in document filing systems
Definitions
- This invention relates to Real-Time Remote Inspection of High-Resolution Images and more particularly to coordinated storage, retrieval and transmission of digital images with different levels of resolution.
- the interactivity is handled by letting the user to select, within a low-resolution version of the image he/she is interested in, only a portion of the image to be viewed with higher resolution or higher priority with respect to the remainder: this implies a hierarchical structure of the image data as well as the definition of Region of Interest (Rol) that indicates a specific portion of the image to be viewed at higher resolution.
- Rol Region of Interest
- the Rol have two possible interpretations: in the first, the Rol is viewed in a display window independent from the low-resolution image while in the second, the Rol is displayed within the same display window but with an enhanced level of resolution for the specific region (EP-A-0 246 010 and WO-A-96 29818 teach about said second interpretation).
- the aim of the present invention is to provide coordinated integrated storage, retrieval and transmission methods to support the decoding operations required in particular by [1], although it is not exclusively targeted to the mentioned system.
- a system that receives a hierarchy of images at different resolution levels, either in original or coded format, and, by using only index numbers as identificators of portions of images, establishes direct relationships between corresponding areas in all consecutive levels of the hierarchy, before the storage in the database. It is also addressed a system for retrieving in a second time both full images and Rol that are selected by the user through the display apparatus. Finally, it is addressed a system for the interactive transmission of images or Rol as a sequence of consecutive elementary areas.
- the first object of this invention is to provide a suitable technique for the storage of high-resolution images that are generated by the image source as a hierarchy of digital images.
- the second object of this invention is to provide a suitable technique for retrieving images and portion of images that are organised in a hierarchical structure in a storage apparatus.
- the third object of this invention is to provide a suitable technique for the transmission to a display apparatus of images and portion of images that are retrieved from a storage apparatus and are organised in a hierarchical structure.
- a method for storage, retrieval and transmission comprising: the steps of receiving hierarchical digital image data from the source, where images are analysed and further decomposed as a set of elementary areas in which any such area in one image at a given level of the hierarchy corresponds to a group of elementary areas in the upper level of the hierarchy and in which for each image of the hierarchy, consecutive elementary areas are sorted out in a first file, whereas the position in said first file of each area is indexed in a second file and the length in bytes of each area is stored in a third file; the steps of passing and storing the index file and the length file of the areas in each image of the hierarchy in which these files are further used when retrieving from the storage means a portion of a given image in the hierarchy selected by a control signal issued by the display apparatus and where the selection refers to an images or a portion of an image at a coarser resolution level in the hierarchy; the steps of buffering the requested portion of images at higher resolution where the elementary areas are organised in order to form
- the first object of the invention is more particularly achieved by the steps of storing digital hierarchies of images corresponding to increasing resolution levels in which the whole information content of the image is aggregated as a set of indexes, each of them corresponding to one and only one elementary area and where:
- the step of storing digital hierarchies of images may be a step of storing hierarchies of uncoded images in the form of consecutive elementary areas of M x N pixels, independently from the hierarchical level, or a step of storing hierarchies of coded images where elementary areas are obtained through a mask composed of consecutive regular rectangles of M x N pixels.
- the image at level i may be stored as a set of consecutive regular elementary areas for which an index is produced in the increasing order from 0 to H j x V j , where H j and V j are respectively the number of areas in the horizontal and vertical directions.
- An index may be associated and used to identify a unique elementary area of the image at level i and in which this index corresponds to Zj 2 indexes of the image at level i-1 and each image may be decomposed as a set of indexed elementary areas and for each index the length in bytes may be written in a file at the corresponding index value.
- each image may be decomposed in a set of indexed elementary areas and for each index the position of the first byte of the information relative the indexed area may be written in a file at the corresponding index value.
- the second object of the invention is more particularly achieved by retrieving steps for stored hierarchies of images where:
- the hierarchy is stored as a unique file for each coded or uncoded image in the order of consecutive elementary areas
- the display apparatus queries images or portion of images in the increasing order of resolution level
- the image at the lowest resolution level n may be used to define a portion of the image at level n-1 in the form of a rectangular mask defined by the upper-left x 1 t y 1 and bottom-right x 2 , y 2 corners.
- any level i of the hierarchy may be defined a portion of the image at level i-1 to be retrieved as a rectangular area x 1 ( y ⁇ , x 2 , ⁇ 2 , until the level 1 of the hierarchy is reached.
- the queries may be received from the display apparatus as a query of a rectangular area x 1 , y 1 ; x 2 , y 2 of the image at level i that is mapped into a corresponding rectangular area x ⁇ , y.,', x 2 ', y 2 ' of the image i-1 to be retrieved, and a rectangular area x ⁇ , y ⁇ , x 2 ', y 2 ' of the image at level i-1 of the hierarchy may be mapped into a set of indexed areas for which the indexes are in the increasing order and the set of elementary areas covers the defined rectangle.
- the database may be queried for a portion of an image in the form of a set of rectangular elementary areas and this elementary areas may be identified as index numbers in the increasing order and in which the index is associated with the position and length of the elementary area to be retrieved.
- the database may be queried for a portion of an image in the form of a set of elementary areas, where the indexed elementary areas are identified by indexed positions and lengths to be retrieved without any specific ordering.
- a sequence of portions of the same reference image may be queried.
- the third object of the invention is more particularly achieved by the steps of temporarily storing the information relative to the elementary area to be transmitted in a buffer organised to store the exact amount of information to be transmitted when buffer is filled up.
- the buffer may be organised as consecutive indexed slots in the increasing indexing order, each slot corresponding to an indexed length.
- the transmission may be organised for sending the data contained in the buffer and data being readout in the increasing index order.
- Fig. 1 is a block diagram of a preferred embodiment of the method of the present invention.
- Fig. 2 represents the operations performed in storage mode, where the Y j x X j image of level i is decomposed in H j x V ( elementary areas of size M x N pixels.
- Fig. 3 represents the mapping of one M x N elementary area of level i with Z j 2 elementary areas of level i-1 , where Z j is the zooming factor between image at level i and image at level i-1 .
- Fig. 4 represents the selection of the active elementary areas in the image at level i- 1 once the selection is performed in the image at the level i .
- the rectangle x ⁇ , y-,', x 2 ', y 2 ' is derived from the reference image, then the indexes of the active elementary areas are computed.
- Fig. 5 represents the set of indexes that are queried to the database after the selection and the computation of active elementary areas on the reference image.
- Fig. 6 represents the buffer space allocated for each control signal received from the display apparatus, once the selection on the reference image has been performed, as well as the queries to the storage apparatus have been sent: the selected elementary areas are received from the database and placed in the appropriate slot, until the buffer is filled up.
- Block 1 is the source of images in hierarchical format
- block 2 is the storage system of the invention
- block 3 is the storage apparatus
- block 4 is the memory that stores the length and position of the indexed elementary areas for subsequent retrieval
- block 5 generates the control signals received from the display apparatus
- block 6 is the retrieval system of the invention
- block 7 is the memory to organise the buffer slots of the retrieved elementary areas
- block 9 is the transmission support
- block 10 is the display apparatus.
- the method for storage, retrieval and transmission of the present invention includes the following described methods in storage, retrieval and transmission modes.
- the storage apparatus is composed of one or several disks and/or CD juke-box that permanently store any digital information, in particular high-resolution images in the embodiment of the invention.
- the method in the storage mode is receiving the hierarchy of digitised images from the image source, either in original or coded format.
- the image with the lowest resolution level, say In is analysed first and subdivided into elementary areas of equal size M x N pixels to form a regular grid (Fig. 2).
- Each area of the grid is then indexed starting from the upper-left corner with the number zero and increasing the index for the consecutive areas in the horizontal direction or for the left area of the following line of the grid if the right image border is reached.
- the number of areas in the horizontal scan and in the vertical scan are, respectively:
- X n is the horizontal size in pixels
- Y n is the vertical size in pixels
- the method in storage mode For each indexed area, the method in storage mode generates a trace of the length l k , in number of bytes, occupied by the k-th indexed area of the image (such length is the same for all areas if the image is in uncompressed format).
- the elementary areas are then sent to the storage apparatus in their exact indexing order, so that the consecutive bytes of each area are adjacent for the subsequent retrieval and transmission.
- each given area of the image l j is covered by the low-resolution version of an aggregation of an integral number of elementary areas in the image lj. 1 t thereof if the zooming factor from image l j to image l is Z j in both the horizontal and vertical axes, each elementary area in l j corresponds to Z j 2 elementary areas in l ⁇ (Fig. 3).
- the method computes the position and the length, in number of bytes, of the indexed areas in the order they are sent to the storage apparatus. These positions and lengths are then recorded and sent as two distinct files to the retrieval module for subsequent hierarchical access to the images.
- the method in retrieval mode is remotely controlled by a user and is receiving control signals from the display apparatus.
- the storage apparatus contains all the information that can be potentially accessed at a given instant by the user in the hierarchy of resolution levels, to be displayed either as separate windows for separate resolution levels or in the same window with enhanced resolution levels.
- control signal issued by the user through the display apparatus represents a request for zooming within the image l j at the resolution level i , the rectangular region defined by his upper-left corner x ⁇ y ⁇ and his bottom-right corner x 2 , y 2 .
- the method operates as follows: for the image l n with the lowest resolution level, the retrieval system reads from the storage apparatus l k consecutive bytes at the position p k , where k ranging from 0 to (H n x V n )-1 is the index of the elementary areas in that image and p k and l k are respectively the position and the length in bytes of the k-th indexed elementary area as contained in the retrieval memory of the method; in subsequent operations and with reference to the image l j in the i-th level of the hierarchy, the retrieval is performed under control of the display apparatus that indicates simultaneously the level i of the reference image, the upper-left and bottom-right corner of the rectangular area x 1 t y ⁇ x 2 , y 2 to be viewed with magnification; the indexes of the elementary areas of the image l j .-, to be then displayed are computed from the index of the upper-left corner of the portion of image lj already displayed (Fig.
- x MOD y is the remainder of x/y
- DIV y is the integral part x/y
- index entries activate the corresponding entries in the position and length files of the image l ⁇ contained in the retrieval memory that queries the storage apparatus.
- the image data in the storage apparatus is accessed in the order of increasing index, reading in the storage apparatus the data file corresponding to the image l at the indexed position and at the indexed length in bytes.
- the following operation is the transmission of the elementary areas to the display apparatus.
- the method in the transmission mode controls a buffer in which the information relative to the elementary area being transmitted is temporarily stored before accessing the transmission means.
- the allocated buffer space corresponds to the sum of all the lengths of the elementary areas that have been requested by the retrieval module and each area and buffer slot are sorted in the order of their increasing index.
- the transmission operates as follows: the indexed areas enters the transmission buffer in their request order, if storage apparatus is synchronous with the retrieval module, or in a random order, if not, and are put in the corresponding location of the buffer since we know from the index the position and length of all the elementary areas (Fig. 6); when the buffer is filled up and ready to be transmitted, the bytes are then delivered to the transmission means and in turn to the display means. The system stops until the next control signal is received from the display apparatus.
- the invention has several domains of applicability, generally those in which the inspection of high resolution images is a current practice and in particular when such information have to be stored in a central database for future multiple access, such as:
- the required means are: a computer unit located near the storage apparatus to process digital images in hierarchical format where either original or incremental or coded formats are equally supported, since it is a task of the display apparatus to process the content of the images; storage means to store the information that is processed by the computer unit and accessed later; a computer unit to receive controls from the display apparatus and query data to the storage means; buffer means to structure the information to be sent to the display apparatus; transmission means to carry information from the buffer to the display apparatus and to carry control signals from the display apparatus to a computer unit.
- the benefits of using the method of the present invention include:
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Storing Facsimile Image Data (AREA)
Abstract
There is provided a method for storage, retrieval and transmission comprising: the steps of receiving hierarchical digital image data from the source (1), where images are analysed and further decomposed as a set of elementary areas in which any such area in one image at a given level of hierarchy corresponds to a group of elementary areas in the upper level of the hierarchy and in which for each image of the hierarchy, consecutive elementary areas are sorted out in a first file, where the position in said first file of each area is indexed in a second file and the length in bytes of each area is stored in a third file; the steps of passing and storing the index file and the length file of the areas in each image of the hierarchy in which these files are further used when retrieving from the storage means (3) a portion of a given image in the hierarchy selected by a control signal (5) issued by the display apparatus (10) and where the selection refers to an image or portion of an image at a coarser resolution level in the hierarchy; the steps of buffering the requested portion of the images at higher resolution where the elementary areas are organised in order to form a sequence of consecutive areas to be transmitted across the transmission means (9) and that are suitable for the display apparatus.
Description
Real-Time Remote Inspection of High-Resolution Images
FIELD OF THE INVENTION
This invention relates to Real-Time Remote Inspection of High-Resolution Images and more particularly to coordinated storage, retrieval and transmission of digital images with different levels of resolution.
BACKGROUND OF THE INVENTION
There is the need of accessing high resolution image information in several fields of pure and applied technical and scientific environments, such as cartography, medicine, industrial design and professional editing.
In the past this information was typically stored in large "film rooms" in the form of printed films or papers. The available technologies permit today to have and store the same information in large database systems containing images in digital format.
There is also the additional need to remotely access this data from a display station that is connected with long-distance transmission means to the storage apparatus; the most relevant user requirement in this case is the so called distance independence, i.e., the ability to remotely perform display operations like browsing, navigation and zooming in a natural way. This requirement is usually addressed as real-time interactivity.
The interactivity is handled by letting the user to select, within a low-resolution version of the image he/she is interested in, only a portion of the image to be viewed with higher resolution or higher priority with respect to the remainder: this implies a hierarchical structure of the image data as well as the definition of Region of Interest (Rol) that indicates a specific portion of the image to be viewed at higher resolution. It turns out that the Rol have two possible interpretations: in the first, the Rol is viewed in a display window independent from the low-resolution image while in the second, the Rol is displayed within the same display window but with an enhanced level of resolution for the specific region (EP-A-0 246 010 and WO-A-96 29818 teach about said second interpretation). The coder defined in "JPEG Committe Draft, Digital Compression and Coding of Continuous-tone Still Images, Jan. 1991" also teaches about multi-resolution hierarchical coding.
The concept of Rol makes the system interactive because it reduces drastically the amount of information and thus the delay for the display and long-distance transmission at a given instant. Moreover, the extensive use of coding-decoding techniques, in particular targeted to hierarchical image structures further increases the degree of interactivity. The publication of H. Torbey, D. Freilander, J. Barda, "Selective Decompression on a Hierarchically Coded Image", in Multidimensional Signal Processing, San Francisco, Mar. 1992 [1] teaches about mathematical relations enabling decoding/decompression for image hierarchies that operates on a user selectable Rol.
The aim of the present invention is to provide coordinated integrated storage, retrieval and transmission methods to support the decoding operations required in particular by [1], although it is not exclusively targeted to the mentioned system.
For the scope of this invention, it is addressed a system that receives a hierarchy of images at different resolution levels, either in original or coded format, and, by using only index numbers as identificators of portions of images, establishes direct relationships between corresponding areas in all consecutive levels of the hierarchy, before the storage in the database. It is also addressed a system for retrieving in a second time both full images and Rol that are selected by the user through the display apparatus. Finally, it is addressed a system for the interactive transmission of images or Rol as a sequence of consecutive elementary areas.
It is noticeable that, although this invention fits both with coded and uncoded images, in most of the implemented systems it will operate on coded material and thus it will select elementary areas as blocks of the coded image, defined by appropriate block headers in the coded stream.
SUMMARY OF THE INVENTION
The first object of this invention is to provide a suitable technique for the storage of high-resolution images that are generated by the image source as a hierarchy of digital images.
The second object of this invention is to provide a suitable technique for retrieving images and portion of images that are organised in a hierarchical structure in a storage apparatus.
The third object of this invention is to provide a suitable technique for the transmission to a display apparatus of images and portion of images that are retrieved from a storage apparatus and are organised in a hierarchical structure.
In order to achieve the above objects, there is provided a method for storage, retrieval and transmission comprising: the steps of receiving hierarchical digital image data from the source, where images are analysed and further decomposed as a set of elementary areas in which any such area in one image at a given level of the hierarchy corresponds to a group of elementary areas in the upper level of the hierarchy and in which for each image of the hierarchy, consecutive elementary areas are sorted out in a first file, whereas the position in said first file of each area is indexed in a second file and the length in bytes of each area is stored in a third file; the steps of passing and storing the index file and the length file of the areas in each image of the hierarchy in which these files are further used when retrieving from the storage means a portion of a given image in the hierarchy selected by a control signal issued by the display apparatus and where the selection refers to an images or a portion of an image at a coarser resolution level in the hierarchy; the steps of buffering the requested portion of images at higher resolution where the elementary areas are organised in order to form a sequence of consecutive areas to be transmitted across the transmission means and that are suitable for the display apparatus.
The first object of the invention is more particularly achieved by the steps of storing digital hierarchies of images corresponding to increasing resolution levels in which the whole information content of the image is aggregated as a set of indexes, each of them corresponding to one and only one elementary area and where:
- images are received from a source of images that independently generates the image hierarchy
- one elementary area at a given level of the hierarchy matches exactly a down- sampled aggregation of elementary areas in the adjacent image at higher resolution
- a relationship between one elementary area and the group of matching areas of the upper resolution level is established through index numbers with a preprocessing phase prior of the storage.
The step of storing digital hierarchies of images may be a step of storing hierarchies of uncoded images in the form of consecutive elementary areas of M x N pixels, independently from the hierarchical level, or a step of storing hierarchies of coded
images where elementary areas are obtained through a mask composed of consecutive regular rectangles of M x N pixels.
The image at level i may be stored as a set of consecutive regular elementary areas for which an index is produced in the increasing order from 0 to Hj x Vj , where Hj and Vj are respectively the number of areas in the horizontal and vertical directions.
Each image at level i may be decomposed in Hj x Vj consecutive elementary areas and in which H = Zj x Hj and Vj.^Zj x Vs are the number of elementary areas in the upper level i-1 of the hierarchy and Z( is the zooming factor between image lj and lj..,.
An index may be associated and used to identify a unique elementary area of the image at level i and in which this index corresponds to Zj2 indexes of the image at level i-1 and each image may be decomposed as a set of indexed elementary areas and for each index the length in bytes may be written in a file at the corresponding index value. In a variant, each image may be decomposed in a set of indexed elementary areas and for each index the position of the first byte of the information relative the indexed area may be written in a file at the corresponding index value.
The second object of the invention is more particularly achieved by retrieving steps for stored hierarchies of images where:
- the hierarchy is stored as a unique file for each coded or uncoded image in the order of consecutive elementary areas
- the display apparatus queries images or portion of images in the increasing order of resolution level
- relationships between elementary areas of successive levels of the hierarchy are maintained in the form of files that represents position and length of each individual area within the stored image.
The image at the lowest resolution level n may be used to define a portion of the image at level n-1 in the form of a rectangular mask defined by the upper-left x1 t y1 and bottom-right x2, y2 corners.
At any level i of the hierarchy may be defined a portion of the image at level i-1 to be retrieved as a rectangular area x1 ( yη, x2, ^2 , until the level 1 of the hierarchy is reached.
The queries may be received from the display apparatus as a query of a rectangular area x1, y1 ; x2, y2 of the image at level i that is mapped into a corresponding rectangular area x^, y.,', x2', y2' of the image i-1 to be retrieved, and a rectangular area x^, y^, x2', y2' of the image at level i-1 of the hierarchy may be mapped into a set of indexed areas for which the indexes are in the increasing order and the set of elementary areas covers the defined rectangle.
The database may be queried for a portion of an image in the form of a set of rectangular elementary areas and this elementary areas may be identified as index numbers in the increasing order and in which the index is associated with the position and length of the elementary area to be retrieved. In a variant the database may be queried for a portion of an image in the form of a set of elementary areas, where the indexed elementary areas are identified by indexed positions and lengths to be retrieved without any specific ordering.
A sequence of portions of the same reference image may be queried.
The third object of the invention is more particularly achieved by the steps of temporarily storing the information relative to the elementary area to be transmitted in a buffer organised to store the exact amount of information to be transmitted when buffer is filled up.
The buffer may be organised as consecutive indexed slots in the increasing indexing order, each slot corresponding to an indexed length.
The transmission may be organised for sending the data contained in the buffer and data being readout in the increasing index order.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 is a block diagram of a preferred embodiment of the method of the present invention.
Fig. 2 represents the operations performed in storage mode, where the Yj x Xj image of level i is decomposed in Hj x V( elementary areas of size M x N pixels.
Fig. 3 represents the mapping of one M x N elementary area of level i with Zj 2 elementary areas of level i-1 , where Zj is the zooming factor between image at level i and image at level i-1 .
Fig. 4 represents the selection of the active elementary areas in the image at level i- 1 once the selection is performed in the image at the level i . The rectangle x^, y-,', x2', y2' is derived from the reference image, then the indexes of the active elementary areas are computed.
Fig. 5 represents the set of indexes that are queried to the database after the selection and the computation of active elementary areas on the reference image.
Fig. 6 represents the buffer space allocated for each control signal received from the display apparatus, once the selection on the reference image has been performed, as well as the queries to the storage apparatus have been sent: the selected elementary areas are received from the database and placed in the appropriate slot, until the buffer is filled up.
DETAILED DESCRIPTION OF THE INVENTION
A block diagram of a preferred embodiment of the method of the present invention is represented on Fig. 1. Block 1 is the source of images in hierarchical format, block 2 is the storage system of the invention, block 3 is the storage apparatus, block 4 is the memory that stores the length and position of the indexed elementary areas for subsequent retrieval, block 5 generates the control signals received from the display apparatus, block 6 is the retrieval system of the invention, block 7 is the memory to organise the buffer slots of the retrieved elementary areas, block 9 is the transmission support and block 10 is the display apparatus.
The method for storage, retrieval and transmission of the present invention includes the following described methods in storage, retrieval and transmission modes.
Method in storage mode
The storage apparatus is composed of one or several disks and/or CD juke-box that permanently store any digital information, in particular high-resolution images in the embodiment of the invention.
The method in the storage mode is receiving the hierarchy of digitised images from the image source, either in original or coded format. The image with the lowest resolution level, say In, is analysed first and subdivided into elementary areas of equal size M x N pixels to form a regular grid (Fig. 2). Each area of the grid is then indexed starting from the upper-left corner with the number zero and increasing the index for the consecutive areas in the horizontal direction or for the left area of the following line of the grid if the right image border is reached.
The number of areas in the horizontal scan and in the vertical scan are, respectively:
Hn = Xn/N Vn = Yn/M where:
Xn is the horizontal size in pixels Yn is the vertical size in pixels
This gives Hn x Vn adjacent areas indexed from 0 to (Hn x Vn)-1 .
For each indexed area, the method in storage mode generates a trace of the length lk, in number of bytes, occupied by the k-th indexed area of the image (such length is the same for all areas if the image is in uncompressed format).
The elementary areas are then sent to the storage apparatus in their exact indexing order, so that the consecutive bytes of each area are adjacent for the subsequent retrieval and transmission.
Similarly, the method applies for all the images with higher resolution ln.-,, ..., ^ , where I-, is the image with highest resolution in the hierarchy. All elementary areas have M x N pixels and in order to provide the hierarchical structure, each given area of the image lj is covered by the low-resolution version of an aggregation of an integral number of elementary areas in the image lj.1 t thereof if the zooming factor from image lj to image l is Zj in both the horizontal and vertical axes, each elementary area in lj corresponds to Zj 2 elementary areas in lμι (Fig. 3).
Further, for each digital image of the hierarchy and for each indexed elementary area, the method computes the position and the length, in number of bytes, of the indexed areas in the order they are sent to the storage apparatus. These positions
and lengths are then recorded and sent as two distinct files to the retrieval module for subsequent hierarchical access to the images.
Method in retrieval mode
The method in retrieval mode is remotely controlled by a user and is receiving control signals from the display apparatus. The storage apparatus contains all the information that can be potentially accessed at a given instant by the user in the hierarchy of resolution levels, to be displayed either as separate windows for separate resolution levels or in the same window with enhanced resolution levels.
For the object of this invention, it is assumed that the control signal issued by the user through the display apparatus represents a request for zooming within the image lj at the resolution level i , the rectangular region defined by his upper-left corner x^ y^ and his bottom-right corner x2, y2 . The method operates as follows: for the image ln with the lowest resolution level, the retrieval system reads from the storage apparatus lk consecutive bytes at the position pk , where k ranging from 0 to (Hn x Vn)-1 is the index of the elementary areas in that image and pk and lk are respectively the position and the length in bytes of the k-th indexed elementary area as contained in the retrieval memory of the method; in subsequent operations and with reference to the image lj in the i-th level of the hierarchy, the retrieval is performed under control of the display apparatus that indicates simultaneously the level i of the reference image, the upper-left and bottom-right corner of the rectangular area x1 t y^ x2, y2 to be viewed with magnification; the indexes of the elementary areas of the image lj.-, to be then displayed are computed from the index of the upper-left corner of the portion of image lj already displayed (Fig. 4), say Cj , from the values of x^ y1, x2, y2 and from the zooming factor between the two images Zj , first by determining the absolute position x-)', y^, X2 1, V2 in tne image lj_ι of the corresponding rectangle x-|, yl t x2, y2 of the image lj as:
x = Zj x (Cj MOD Hj + x ) y-,' = Zj x (Cj DIV Hj + y.,) x2' = Zj x (Cj MOD Hj + x2) y2' = Zj x (Cj DIV Hj + y2)
where: x MOD y is the remainder of x/y x DIV y is the integral part x/y
and second by determining the index entries relative to elementary areas of the image Ij.-i as (Fig. 5):
y1'DIV HM+ y1'DIV HM+ y DIV Hi-l÷
÷x^DIV N, +(x1'+N)DIV N, ... +x2'DIV N,
(y1'+M)DIV HM+ (y1'+M)DIV HM+ '+MJDIV HM+
+x1'DIV N, +(x1'+N)DIV N, ... +x2'DIV N,
y2'DIV Hi-1+ y2 ,DIV HM+ y2'DIV Hμι+
+x1 ,DIV N, +(x1'+N)DIV N, ... +x2T IV N,
that cover the rectangle x-|', y.,', x2', y2' ; these index entries activate the corresponding entries in the position and length files of the image lμι contained in the retrieval memory that queries the storage apparatus.
The image data in the storage apparatus is accessed in the order of increasing index, reading in the storage apparatus the data file corresponding to the image l at the indexed position and at the indexed length in bytes.
The following operation is the transmission of the elementary areas to the display apparatus.
Method in transmission mode
The method in the transmission mode controls a buffer in which the information relative to the elementary area being transmitted is temporarily stored before accessing the transmission means.
The allocated buffer space corresponds to the sum of all the lengths of the elementary areas that have been requested by the retrieval module and each area and buffer slot are sorted in the order of their increasing index. The transmission operates as follows: the indexed areas enters the transmission buffer in their request order, if storage apparatus is synchronous with the retrieval module, or in a random order, if not, and are put in the corresponding location of the buffer since we know from the index the position and length of all the elementary areas (Fig. 6); when the buffer is filled up and ready to be transmitted, the bytes are then delivered to the transmission means and in turn to the display means.
The system stops until the next control signal is received from the display apparatus.
DOMAINS OF APPLICABILITY
The invention has several domains of applicability, generally those in which the inspection of high resolution images is a current practice and in particular when such information have to be stored in a central database for future multiple access, such as:
- database systems for the design of industrial plants and buildings,
- database systems for the design and further inspection of mechanical engines,
- database systems for the computer assisted consultation of cartographic material,
- medical systems to store and access images produced via digital microscopes,
- medical systems to store and access images obtained by radiation,
- medical systems to store and access cardiography films.
The required means are: a computer unit located near the storage apparatus to process digital images in hierarchical format where either original or incremental or coded formats are equally supported, since it is a task of the display apparatus to process the content of the images; storage means to store the information that is processed by the computer unit and accessed later; a computer unit to receive controls from the display apparatus and query data to the storage means; buffer means to structure the information to be sent to the display apparatus; transmission means to carry information from the buffer to the display apparatus and to carry control signals from the display apparatus to a computer unit.
The benefits of using the method of the present invention include:
- user ability to select instantaneously the portion of images to be viewed with magnification,
- system ability to select fast and precisely the information required at a given instant by the user,
- system ability to reduce the amount of information that have to cross the transmission means,
- system ability to co-exist with any present and future method for image processing.
Claims
1. A method for storage, retrieval and transmission of digital images with different levels of resolution, characterised by the steps of storing digital hierarchies of images corresponding to increasing resolution levels in which the whole information content of the image is aggregated as a set of indexes, each of them corresponding to one and only one elementary area and where:
- images are received from a source of images that independently generates the image hierarchy
- one elementary area at a given level of the hierarchy matches exactly a down- sampled aggregation of elementary areas in the adjacent image at higher resolution
- a relationship between one elementary area and the group of matching areas of the upper resolution level is established through index numbers with a preprocessing phase prior of the storage.
2. A method as in Claim 1 , characterised by storing hierarchies of uncoded images in the form of consecutive elementary areas of M x N pixels, independently from the hierarchical level.
3. A method as in Claim 1 , characterised by storing hierarchies of coded images where elementary areas are obtained through a mask composed of consecutive regular rectangles of M x N pixels.
4. A method as in any one of the preceding Claims, characterised by storing hierarchies of images where the image at level i is stored as a set of consecutive regular elementary areas for which an index is produced in the increasing order from 0 to Hj x Vj , where H( and Vj are respectively the number of areas in the horizontal and vertical directions.
5. A method as in Claim 4, characterised by storing a hierarchy of coded or uncoded image data where each image at level i is decomposed in Hs x Vj consecutive elementary areas and in which Hj.^ Zj x Hj and Vj.1=Zi x Vs are the number of elementary areas in the upper level i-1 of the hierarchy and Zj is the zooming factor between image lj and l^.
6. A method as in Claims 4 or 5, characterised by storing hierarchies of images where an index is associated and used to identify a unique elementary area of the image at level i and in which this index corresponds to Zj 2 indexes of the image at level i-1 .
7. A method as in Claim 6, characterised by storing a hierarchy of images where each image is decomposed as a set of indexed elementary areas and for each index the length in bytes is written in a file at the corresponding index value.
8. A method as in Claim 6, characterised by storing a hierarchy of images where each image is decomposed in a set of indexed elementary areas and for each index the position of the first byte of the information relative the indexed area is written in a file at the corresponding index value.
9. A method as in anyone of the preceding claims, characterised by retrieving steps for stored hierarchies of images where:
- the hierarchy is stored as a unique file for each coded or uncoded image in the order of consecutive elementary areas
- the display apparatus queries images or portion of images in the increasing order of resolution level
- relationships between elementary areas of successive levels of the hierarchy are maintained in the form of files that represents position and length of each individual area within the stored image.
10. A method for storage, retrieval and transmission of digital images with different levels of resolution, characterised by retrieving steps for stored hierarchies of images where:
- the hierarchy is stored as a unique file for each coded or uncoded image in the order of consecutive elementary areas
- the display apparatus queries images or portion of images in the increasing order of resolution level
- relationships between elementary areas of successive levels of the hierarchy are maintained in the form of files that represents position and length of each individual area within the stored image.
11. A method as in Claim 10, characterised by stored hierarchies of images where the image at the lowest resolution level n is used to define a portion of the image at level n-1 in the form of a rectangular mask defined by the upper-left x1 f y1 and bottom-right x2, y2 corners.
12. A method as in Claim 10, characterised by stored hierarchies of images where at any level i of the hierarchy is defined a portion of the image at level i-1 to be retrieved as a rectangular area x1 t y1, x2, y2 , until the level 1 of the hierarchy is reached.
13. A method as in anyone of the Claims 10, 11 or 12, characterised by stored hierarchies of images where the queries are received from the display apparatus as a query of a rectangular area x-,, y1 t x2, y2 of the image at level i that is mapped into a corresponding rectangular area x^, y^, x2', y2' of the image i-1 to be retrieved.
14. A method as in Claim 13, characterised by stored hierarchies of images in which a rectangular area x.,', y.,', x2', y2' of the image at level i-1 of the hierarchy is mapped into a set of indexed areas for which the indexes are in the increasing order and the set of elementary areas covers the defined rectangle.
15. A method as in Claims 13 or 14, characterised by stored hierarchies of images that queries the database for a portion of an image in the form of a set of rectangular elementary areas and this elementary areas are identified as index numbers in the increasing order and in which the index is associated with the position and length of the elementary area to be retrieved.
16. A method as in Claims 13 or 14, characterised by stored hierarchies of images that queries the database for a portion of an image in the form of a set of elementary areas, where the indexed elementary areas are identified by indexed positions and lengths to be retrieved without any specific ordering.
17. A method as in anyone of the Claims 11, 12, 13, 14, 15 or 16, characterised by stored hierarchies of images that queries a sequence of portions of the same reference image.
18. A method as in anyone of the preceding Claims, where the information relative to the elementary area to be transmitted is temporarily stored in a buffer organised to store the exact amount of information to be transmitted when buffer is filled up.
19. A method as in Claims 18, where the buffer is organised as consecutive indexed slots in the increasing indexing order and each slots corresponds to an indexed length.
20. A method as in Claims 18 or 19, characterised by transmitting portions of images that is organised for sending the data contained in the buffer and in which data is readout in the increasing index order.
21. A method for storage, retrieval and transmission of digital images with different level of resolution, where the information to be transmitted is temporarily stored in a buffer and transmitted when the buffer is filled up and where the buffer is organised to store the exact amount of information to be transmitted.
22. A method as in Claims 21 , where the buffer is organised as consecutive indexed slots in the increasing indexing order and each slot corresponds to an indexed length.
23. A method as in Claims 21 or 22, characterised by transmitting portions of images that is organised for sending the data contained in the buffer and in which data is readout in the increasing index order.
24. A method for storage, retrieval and transmission of digital images comprising: the steps of receiving hierarchical digital image data from the source, where images are analysed and further decomposed as a set of elementary areas in which any such area in one image at a given level of the hierarchy corresponds to a group of elementary areas in the upper level of the hierarchy and in which for each image of the hierarchy, consecutive elementary areas are sorted out in a first file, whereas the position in said first file of each area is indexed in a second file and the length in bytes of each area is stored in a third file; the steps of passing and storing the index file and the length file of the areas in each image of the hierarchy in which these files are further used when retrieving from the storage means a portion of a given image in the hierarchy selected by a control signal issued by the display apparatus and where the selection refers to an images or a portion of an image at a coarser resolution level in the hierarchy; the steps of buffering the requested portion of images at higher resolution where the elementary areas are organised in order to form a sequence of consecutive areas to be transmitted across the transmission means and that are suitable for the display apparatus.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GBGB9627037.6A GB9627037D0 (en) | 1996-12-30 | 1996-12-30 | Real-time interactive remote inspection of high-resolution images |
GB9627037 | 1996-12-30 | ||
PCT/IB1997/001607 WO1998029819A1 (en) | 1996-12-30 | 1997-12-30 | Real-time remote inspection of high-resolution images |
Publications (1)
Publication Number | Publication Date |
---|---|
EP0946919A1 true EP0946919A1 (en) | 1999-10-06 |
Family
ID=10805102
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP97947822A Ceased EP0946919A1 (en) | 1996-12-30 | 1997-12-30 | Real-time remote inspection of high-resolution images |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP0946919A1 (en) |
AU (1) | AU5405898A (en) |
GB (1) | GB9627037D0 (en) |
WO (1) | WO1998029819A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2432739A (en) * | 2005-11-04 | 2007-05-30 | Bosch Gmbh Robert | Storing video data in different hierarchy levels in network |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AUPR678401A0 (en) * | 2001-08-02 | 2001-08-23 | Singfield, Christian | Network image server |
US7343052B2 (en) * | 2002-04-09 | 2008-03-11 | Sonic Solutions | End-user-navigable set of zoomed-in images derived from a high-resolution master image |
US20080095447A1 (en) * | 2005-06-30 | 2008-04-24 | Naohiro Fukuyama | Retrieval System and Retrieval Method |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4742558A (en) * | 1984-02-14 | 1988-05-03 | Nippon Telegraph & Telephone Public Corporation | Image information retrieval/display apparatus |
US5165103A (en) * | 1986-09-12 | 1992-11-17 | Hitachi, Ltd. | Image data processing method for selective partial image storage |
JPH02195480A (en) * | 1989-01-25 | 1990-08-02 | Hitachi Ltd | Retrieving system for image data |
JPH03223966A (en) * | 1989-12-26 | 1991-10-02 | Fuji Xerox Co Ltd | Filing device and file retrieving method applying its filing device |
US5321520A (en) * | 1992-07-20 | 1994-06-14 | Automated Medical Access Corporation | Automated high definition/resolution image storage, retrieval and transmission system |
-
1996
- 1996-12-30 GB GBGB9627037.6A patent/GB9627037D0/en active Pending
-
1997
- 1997-12-30 WO PCT/IB1997/001607 patent/WO1998029819A1/en not_active Application Discontinuation
- 1997-12-30 AU AU54058/98A patent/AU5405898A/en not_active Abandoned
- 1997-12-30 EP EP97947822A patent/EP0946919A1/en not_active Ceased
Non-Patent Citations (1)
Title |
---|
See references of WO9829819A1 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2432739A (en) * | 2005-11-04 | 2007-05-30 | Bosch Gmbh Robert | Storing video data in different hierarchy levels in network |
GB2432739B (en) * | 2005-11-04 | 2008-12-03 | Bosch Gmbh Robert | Method for handling content information |
Also Published As
Publication number | Publication date |
---|---|
AU5405898A (en) | 1998-07-31 |
WO1998029819A1 (en) | 1998-07-09 |
GB9627037D0 (en) | 1997-02-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP0500883B1 (en) | Image database incorporating low resolution index image data | |
AU642415B2 (en) | Method and apparatus for manipulating digital video data | |
US4661811A (en) | Video map display | |
EP1024444B1 (en) | Image information describing method, video retrieval method, video reproducing method, and video reproducing apparatus | |
CA1315391C (en) | System for processing various types of information in easily usable form | |
US4665555A (en) | Computer based drawing management system | |
KR20020052190A (en) | Image data displaying system and method | |
JP2001184802A (en) | Device and method for recording and accessing meta-data | |
Hill Jr et al. | Interactive image query system using progressive transmission | |
CN1251514C (en) | Storage of compressed data items | |
WO1998029819A1 (en) | Real-time remote inspection of high-resolution images | |
KR100563085B1 (en) | Method for compositively displaying digital map and photo image | |
JPH0589176A (en) | Image retrieving device | |
CN1033549C (en) | Electronic mapped GPS receiver | |
KR20010035015A (en) | Method and system for displaying integrated image of multiple data | |
JPH05502568A (en) | reader | |
US5107342A (en) | System and a process for recording cartographic information on a videodisk | |
JPS648854B2 (en) | ||
JPH0877328A (en) | Map data retrieval method | |
KR20010000208A (en) | Method and system for large image display | |
JP3450346B2 (en) | Different-scale map management device | |
JPS62243067A (en) | Image file device | |
Shaffer et al. | An in-core hierarchical data structure organization for a geographic database | |
JPH05103233A (en) | Video manufacturing device for data video | |
GB2034941A (en) | Information storage and retrieval system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 19990705 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): CH DE FI FR GB IT LI NL SE |
|
17Q | First examination report despatched |
Effective date: 20000309 |
|
GRAG | Despatch of communication of intention to grant |
Free format text: ORIGINAL CODE: EPIDOS AGRA |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED |
|
18R | Application refused |
Effective date: 20011130 |