CN111938567A

CN111938567A - Deep learning-based ophthalmologic parameter measurement method, system and equipment

Info

Publication number: CN111938567A
Application number: CN202010655960.7A
Authority: CN
Inventors: 翟广涛; 杨小康; 朱文瀚; 朱向阳
Original assignee: Shanghai Jiaotong University
Current assignee: Shanghai Jiaotong University
Priority date: 2020-07-09
Filing date: 2020-07-09
Publication date: 2020-11-17
Anticipated expiration: 2040-07-09
Also published as: CN111938567B

Abstract

The invention provides a human eye parameter measuring method and system based on deep learning, which comprises the following steps: acquiring a face picture, and extracting left and right eye images in the picture; identifying different positions in the left eye image and the right eye image by adopting a deep neural network, wherein the different positions comprise positions of a cornea, a sclera and inner and outer canthus; a plurality of eye parameters are calculated for different locations in the left and right eye images. Meanwhile, equipment based on the human eye parameter measuring method and system is provided. The human eye parameter measuring method, the system and the equipment based on deep learning can realize automatic identification and positioning of different parts of human eyes; the automatic measurement of the eye parameters which are often required to be measured by an ophthalmologist is realized; provides parameter assistance and support for the ophthalmologist to analyze the eye disease condition.

Description

Deep learning-based ophthalmologic parameter measurement method, system and equipment

Technical Field

The invention relates to a medical image processing technology in the technical field of artificial intelligence, in particular to a method, a system and equipment for measuring ophthalmic parameters based on deep learning.

Background

At present, the problems of unbalanced supply and demand of high-quality medical resources, long culture period of doctors, high misdiagnosis rate, quick change of disease spectrum, advanced technology, aggravated aging of population, growth of chronic diseases and the like in China are to be solved. With the increase of the health attention of people, the rapid development of medical AI is promoted due to a great demand.

To date, AI has developed significantly in more segments of our medical field, for example, 2016 (10 months) and hundred degree release of "Baidu medical cerebrum", which is a product of the same class as IBM and google. The Baidu medical brain is particularly applied to the medical field, acquires and analyzes medical professional documents and medical data in large quantity, and gives a final proposal of diagnosis and treatment based on user symptoms through a simulated inquiry flow.

In 7 months in 2017, the ali health promulgated medical AI system, vector You, includes a clinical medical research and diagnosis platform, a medical auxiliary detection engine, and the like. In addition, the Ali health is also cooperated with external institutions such as governments, hospitals and scientific research institutions, and intelligent diagnosis engines for 20 common and frequent diseases are developed, including diabetes, lung cancer prediction, fundus screening and the like.

In 11 months in 2018, the digital diagnosis and treatment equipment special item which is borne by Tencent is started, the item is one of 6 special items for trial and development which are started in the first batch of a national key development plan, and the exploration and the assisted medical service upgrading are performed based on AI + CDSS (artificial intelligence clinical assistant decision support technology).

The AI has many combination points with the medical field, and the summary analysis is carried out through the application condition of the AI in the medical field, and the AI is mainly applied to five fields at present, which are respectively as follows: medical imaging, assisted diagnosis, drug development, health management, disease prediction.

By means of development advantages of medical image big data and image recognition technology, the medical image becomes the most mature field combining artificial intelligence and medical treatment in China and can be applied to medical treatment fields such as pulmonary tuberculosis, eyeground, breast cancer, cervical cancer and the like. In ophthalmic examinations, doctors are often required to take accurate measurements of a patient's eye for diagnosis and disease status determination. They typically measure manually a number of ocular parameters such as the height of the eye cleft, the longitudinal and transverse diameter of the cornea, the distance between the inner can and the outer distance and the distance between the pupils. Currently, the usual method is for the doctor to measure these parameters directly on the patient's eye using a ruler. This is very cumbersome for the doctor and the result is difficult to convince, while physical contact risks infection for both the doctor and the patient. To solve this problem, those skilled in the art try to automatically calculate these parameters of the eye from the eye image by a specific algorithm, which can make it unnecessary for the ophthalmologist to measure the patient's eye with a ruler with time and effort, and for the patient to take a strange posture to facilitate the doctor and to take the risk of infection. In recent years, with the rapid development of deep learning, CNN-based image object detection has achieved high accuracy, which is applied to medical image analysis by more and more researchers, but the techniques and studies applied to the measurement of eye diseases are still limited.

At present, no explanation or report of the similar technology of the invention is found, and similar data at home and abroad are not collected.

Disclosure of Invention

In view of the above-mentioned shortcomings in the prior art, the present invention provides a method, system and device for measuring human eye parameters based on deep learning.

The invention is realized by the following technical scheme.

According to one aspect of the invention, a human eye parameter measurement method based on deep learning is provided, and comprises the following steps:

acquiring a face picture, and extracting left and right eye images in the face picture;

identifying different parts in the left eye image and the right eye image, including positions of cornea, sclera and inner and outer canthus by adopting a deep neural network to obtain an identification result;

and calculating a plurality of eye parameters of different positions and different states in the left eye image and the right eye image according to the identification result.

Preferably, the extracting left and right eye images in the face picture includes:

processing the face picture by adopting open source engineering, extracting a plurality of eye key points in the face picture, and extracting complete left-eye and right-eye images according to the positions of the eye key points.

Preferably, the deep neural network adopts a multitask convolutional neural network, which comprises a corneal block, a scleral block and inner and outer angular blocks; wherein:

the corneal block predicts a bounding box and a confidence for each pixel position using pixel-based prediction, the confidence reflecting the probability of whether the pixel is within the cornea. After non-maximum inhibition, calculating a weighted average value of all the bounding boxes to be used as a final prediction result;

the scleral block employs a full convolution network, wherein the input feature map is up-sampled by a factor of m to obtain a classification score map;

the inner and outer canthus block adopts a design structure of three layers behind a YoLO (You Only Look one) target detection neural network, an image is divided into n x n grids, the grids where the inner and outer canthus fall and coordinate offset relative to the center of the grids are predicted, and then the coordinates of key points are directly predicted.

Preferably, the ocular parameters are 10, including: palpebral fissure height, corneal sagittal diameter, corneal transverse diameter, corneal onversion, supraeyelid retraction, infraeyelid retraction, pupil diameter, inferior limiting amount, superior limiting amount, and external limiting amount.

Preferably, the eye parameters are calculated by fitting a quadratic function to the eyelid and calculating the scale of the physical world and the image size in combination with the sticker.

Preferably, the method of fitting a quadratic function to an eyelid comprises:

after obtaining the identification result of the sclera, fitting the edge of the sclera by using a quadratic function to obtain the position of the eyelid, and setting the center of the cornea as the vertex of a quadratic curve, wherein the expression of the quadratic curve is as follows:

p(y-cy)＝±(x-cx)² (1)

where (cx, cy) is the center of the cornea and (x, y) is the position coordinates of the pixel points in the image. The equation is used to fit the lower eyelid when a positive sign is used on the right side of the equation, and the equation can fit the upper eyelid if a negative sign is used. The value of p is determined by:

p＝d₁+d₂ (2)

wherein d is₁And d₂Pixel distance dimensions representing the sclera width and the maximum flare or cornea diameter at which the sclera cannot engage due to the cornea location, respectively; wherein, for the sclera recognition result of the unilateral shielding cornea of the upper eyelid and the lower eyelid, d₁And d₂Taking values according to the pixel distance of the width of the sclera and the maximum open pixel distance which is caused by the position of the cornea and can not be connected with the sclera; scleral identification of Co-occluded cornea by Upper and lower eyelids, d₂Taking values according to the average value of the pixel distance of the up-and-down opening of the sclera; scleral identification of unoccluded cornea for both upper and lower eyelids, d₂The value is taken in terms of the pixel distance of the corneal diameter.

Preferably, the method of calculating the scale of the physical world and the image size using the sticker includes:

pasting two identical standard round stickers on the forehead of a patient and recognizing the stickers to obtain a scale S between the picture size and the real world physical size_scaleThe method specifically comprises the following steps:

first, two stickers in the picture are identified by Hough transform, and the diameters of the two stickers are extracted, assuming that the diameters of the two stickers obtained from the picture are respectively D_AAnd D_BThe average diameter of the two stickers is then recorded as D_image：

Physical size of the sticker in the real world is D_realScale S between picture size and real world physical size_scaleComprises the following steps:

preferably, the method for calculating the ocular parameter is as follows:

height of eyelid fissure: after fitting the eyelid edge, calculating the distance between the highest point and the lowest point of the eyelid;

corneal sagittal and transverse corneal diameter: calculating directly according to the recognition result;

covering the cornea: after fitting the eyelid edge, calculating the distance between the highest point of the eyelid and the highest point of the cornea;

amount of retraction on eyelid: after fitting the eyelid edge, calculating the distance between the highest point of the eyelid and the highest point of the cornea;

amount of retraction under eyelid: after fitting the eyelid edge, calculating the distance between the lowest point of the eyelid and the lowest point of the cornea;

diameter of pupil: after the cornea is identified, directly calculating after binarization is carried out on the cornea;

lower turnover limit: calculating the distance between the inner and outer canthus connecting lines and the lowest point of the cornea;

upper limiting limit: calculating the distance between the inner and outer canthus connecting lines and the lowest point of the cornea;

limited amount of outward turning: the distance between the outer canthus and the outermost point of the cornea is calculated.

According to a second aspect of the present invention, there is provided a human eye parameter measurement system based on deep learning, comprising:

the eye image extraction module is used for extracting left and right eye images in the face image according to the acquired face image;

the eye part identification module adopts a deep neural network to identify different parts in the left eye image and the right eye image, including positions of a cornea, a sclera and inner and outer canthus, so as to obtain an identification result;

and the eye parameter calculation module is used for calculating a plurality of eye parameters of different positions and different states in the left eye image and the right eye image according to the recognition result obtained by the eye part recognition module.

Preferably, the eye image extraction module processes the face image by using an open source engineering, extracts a plurality of eye key points in the face image, and extracts a complete left eye image and a complete right eye image according to the positions of the eye key points.

Preferably, in the eye part identification module, the deep neural network adopts a multitask convolutional neural network, which comprises a corneal block, a scleral block and inner and outer angular blocks; wherein:

the corneal block predicts a boundary box and a confidence coefficient for each pixel position by adopting pixel-based prediction, and calculates a weighted average value of all the boundary boxes after non-maximum suppression to serve as a final prediction result;

the inner and outer canthus blocks adopt a design structure of a posterior three-layer of a YOLO target detection neural network, an image is divided into n multiplied by n grids, the grids where the inner and outer canthus fall and coordinate deviation relative to the center of the grids are predicted, and then coordinates of key points are directly predicted.

According to a third aspect of the present invention, there is provided an apparatus comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor being operable to perform any of the methods described above when executing the computer program.

Due to the adoption of the technical scheme, the invention has at least one of the following beneficial effects:

the human eye parameter measuring method, the system and the equipment based on deep learning can realize automatic identification and positioning of different parts of human eyes.

The method, the system and the equipment for measuring the eye parameters based on deep learning provided by the invention realize the automatic measurement of the eye parameters which are usually required to be measured by ophthalmologists.

The method, the system and the equipment for measuring the human eye parameters based on deep learning provide parameter assistance and support for an ophthalmologist to analyze the eye disease condition.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a diagram illustrating different parts of a human eye and an example of a human eye picture according to an embodiment of the present invention.

Fig. 2 is a schematic structural diagram of a multitasking neural network in the embodiment of the present invention.

Fig. 3 is a standard sticker in an embodiment of the present invention.

Fig. 4 is a schematic diagram of a post-processing picture for calculating eye parameters according to an embodiment of the present invention.

Fig. 5 is a flowchart of a method for measuring parameters of a human eye based on deep learning according to an embodiment of the present invention.

Detailed Description

The following examples illustrate the invention in detail: the embodiment is implemented on the premise of the technical scheme of the invention, and a detailed implementation mode and a specific operation process are given. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention.

An embodiment of the invention provides an ophthalmic parameter measuring method based on deep learning, which uses a deep learning technology and an image processing algorithm to process an image of a human eye, so as to identify different parts of the human eye and complete measurement of ophthalmic parameters. As shown in fig. 5, the method includes the steps of:

step 1: collecting high-definition pictures of the face of a patient, and automatically extracting pictures of the left eye and the right eye of the patient from the pictures;

as a preferred embodiment, the left and right eye photographs are processed separately;

step 2: identifying different parts of the eye in the left and right eye photographs, including the cornea, sclera, inner and outer canthus (i.e. canthus);

and step 3: the result processing algorithm and the eye parameter calculation method are used for calculating 10 common parameters of the eyes in the left eye photo and the right eye photo. Wherein the parameters are the basis for the ophthalmologist to analyze the eye condition.

The measured eye parameters can assist an ophthalmologist to analyze the eye condition.

As a preferred embodiment, step 1, using currently existing open source engineering (for example, dlib open source library, 68 key points in the picture can be extracted as required) to process the human face and extract a plurality of facial key points, where these facial key points also include key parts such as human eyes, and then extracting a picture of human eyes according to these key points.

In some embodiments of the present invention, the image of the human eye to be identified is shown in fig. 1, where (a) is an illustration of different parts of the human eye, and (b) and (c) are examples of several different eye disease images, respectively, in fig. 1.

As a preferred embodiment, step 2, comprises the following sub-steps:

the cornea, sclera and inner and outer canthus have different characteristics and therefore different methods are needed to identify them separately. The sclera is irregular in shape and greatly different among different patients, so that the sclera segmentation is suitable for semantic segmentation at a pixel level, and the range of the sclera can be accurately identified. The shape of the cornea is regular, roughly circular or elliptical, and can be located and delimited by a rectangular box and then extracted by an ellipse. The medial and lateral canthus are two points on either side of the eye and it is therefore desirable to locate their coordinates directly. In addition, the medial and lateral canthus are on either side of and adjacent to the sclera, a relatively easy to find constraint. This is the basic idea of establishing a deep neural network in the embodiment of the present invention.

The backbone network of the multitask convolutional neural network, based on VGG16, was used, as shown in fig. 2, to identify the corneal, scleral and inner and outer angular portions mentioned above. Fig. 3 is a structure of 3 blocks of CNN, including: cornea (Cornea) mass, Sclera (Sclera) mass and Canthus (inner and outer Canthus) mass, wherein:

predicting a boundary box and a confidence coefficient for each pixel position by adopting pixel-based prediction, and solving a weighted average value of all the boundary boxes after non-maximum suppression to obtain a final prediction result;

the scleral block employs a full convolution network, in which the input feature map is up-sampled by a factor of m to obtain a classification score map; as a preferred embodiment, m is 8;

coordinates of key points are directly predicted for inner and outer canthus blocks, and the image is divided into n × n grids by using a neural network for target detection (refer to Redmon, Joseph, Santosh Divvala, Ross Girshick, and Ali Farhadi, the latter three-layer design structure of "You Only local: Unifield, real-time object detection," In Proceedings of the IEEE conference on component vision and mapping, pp.779-788.2016), to predict In which grid the inner and outer canthus fall and coordinate offset relative to the grid center. In this way, all key parts of the eye, such as sclera, cornea and inner and outer canthus, are obtained, and the positions of the eyelids can be deduced; as a preferred embodiment, n is 7.

As a preferred embodiment, step 3, comprises the following steps:

selecting 10 parameters which are usually required to be measured by an ophthalmologist, wherein the parameters provide objective basis for the ophthalmologist to analyze the eye condition; these 10 parameters include: palpebral Fissure Height (PFH), corneal sagittal diameter, corneal transverse diameter, corneal onversion, episcleral recession, subscleral recession, pupil diameter, inferior restricted, superior restricted, and external restricted.

To obtain the ratio between the size of the picture and the physical size of the real world, a diameter D is applied to the forehead of the patient_realAs shown in fig. 3, two examples of standard stickers are shown as (a) and (b). When the sticker is placed, the sticker is leveled as much as possible and parallel to the camera lens to minimize errors caused by tilting and wrinkling. Let the average diameter of two stickers be D_imagePer pixel, scale S of image size to physical world size_scaleComprises the following steps:

Physical size of the sticker in the real world is D_realThen the scale between the picture size and the physical size of the real world is denoted as S_scaleIt can be calculated from equation (10):

the edge of the sclera is fitted using a quadratic function to obtain the position of the eyelid, as shown in fig. 4, where (a) is an example of several eye pictures, (b) is an illustration of the parameters of the fitted sclera edge, and (c) is a schematic of the method of fitting the sclera edge. Setting the center of the cornea as the vertex of a quadratic curve, wherein the expression of the quadratic curve is as follows:

p(y-cy)＝±(x-cx)² (5)

p＝d₁+d₂ (6)

wherein d is₁And d₂As shown in FIG. 4(b), d₁And d₂Pixel distance dimensions representing the sclera width and the maximum flare or cornea diameter at which the sclera cannot engage due to the cornea location, respectively; for the scleral identification results of the single-edge occlusion of the cornea by the upper and lower eyelids in the first and second images of FIG. 4(b), d₁And d₂Taking values according to the pixel distance of the width of the sclera and the maximum open pixel distance which is caused by the position of the cornea and can not be connected with the sclera; for the scleral identification result of the third image in which the upper and lower eyelids together occlude the cornea, d₂According to the pixel distance of the sclera spreading up and downTaking the value of the mean value; for the sclera recognition result that neither upper eyelid nor lower eyelid of the fourth image obstructs the cornea, d₂The value is taken in terms of the pixel distance of the corneal diameter.

As a preferred embodiment, the method for calculating the ocular parameter comprises:

In another embodiment of the present invention, a system for measuring parameters of human eyes based on deep learning is provided, which includes:

the eye image extraction module is used for extracting left and right eye images in the image according to the acquired face image;

the eye part identification module adopts a deep neural network to identify different parts in the left eye image and the right eye image, including the positions of the cornea, the sclera and the inner and outer canthus;

and the eye parameter calculation module is used for calculating a plurality of eye parameters in different positions and different states in the human eye image according to the recognition result.

As a preferred embodiment, the eye image extraction module uses the existing open source engineering (the patent uses Dlib open source library and the Haar detection algorithm of OpenCV jointly, Dlib open source library is used for processing the whole face image and can extract 68 key points, and the Haar detection algorithm can process partial face images and position human eyes) to process the face and extract a plurality of face key points, wherein the face key points comprise key parts such as human eyes, and then the human eye images are extracted according to the key points.

In a preferred embodiment, in the eye region identification module, the deep neural network employs a multitask neural network, including a corneal block, a scleral block, and inner and outer canthus blocks, and all key regions of the eye, including the sclera, the cornea, and the inner and outer canthus, can be identified according to characteristics of different eye regions. Wherein:

the corneal block uses pixel-based prediction, predicting a bounding box and a confidence for each pixel location, the confidence reflecting the probability of whether the pixel is within the cornea. After non-maximum inhibition, calculating a weighted average value of all the bounding boxes to be used as a final prediction result;

the scleral block employs a full convolution network, in which the input feature map is up-sampled by a factor of m to obtain a classification score map;

the inner and outer canthus blocks adopt a design structure of three layers behind a YoLO (You Only Look one) target detection neural network, an image is divided into n x n grids, the grids where the inner and outer canthus fall and coordinate offset relative to the center of the grids are predicted, and then the coordinates of key points are directly predicted.

In a preferred embodiment, the eye parameter calculation module first selects 10 parameters that the ophthalmologist usually needs to measure, including the Palpebral Fissure Height (PFH), the longitudinal diameter of the cornea, the transverse diameter of the cornea, the corneal onlay, the episcleral recession, the subscleral recession, the pupil diameter, the lower limitation, the upper limitation, and the outer limitation. As a preferred embodiment, the module can paste a standard round sticker on the forehead of the patient and automatically recognize the sticker to obtain the ratio between the picture size and the real world physical size; the eyelids were then fitted using a quadratic function and ocular parameters were calculated.

In a third embodiment of the present invention, an apparatus is provided, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor, when executing the computer program, can be used to execute any one of the deep learning based eye parameter measurement methods.

Optionally, a memory for storing a program; a Memory, which may include a volatile Memory (RAM), such as a Random Access Memory (SRAM), a Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), and the like; the memory may also comprise a non-volatile memory, such as a flash memory. The memory 62 is used to store computer programs (e.g., applications, functional modules, etc. that implement the above-described methods), computer instructions, etc., which may be stored in one or more memories in a partitioned manner. And the computer programs, computer instructions, data, etc. described above may be invoked by a processor.

The computer programs, computer instructions, etc. described above may be stored in one or more memories in a partitioned manner. And the computer programs, computer instructions, data, etc. described above may be invoked by a processor.

A processor for executing the computer program stored in the memory to implement the steps of the method according to the above embodiments. Reference may be made in particular to the description relating to the preceding method embodiment.

The processor and the memory may be separate structures or may be an integrated structure integrated together. When the processor and the memory are separate structures, the memory, the processor may be coupled by a bus.

According to the method, the system and the equipment for measuring the human eye parameters based on the deep learning, provided by the embodiment of the invention, the left eye image and the right eye image in the image are extracted by acquiring the human face image; identifying different positions in the left eye image and the right eye image by adopting a deep neural network, wherein the different positions comprise positions of a cornea, a sclera and inner and outer canthus; calculating a plurality of eye parameters of different positions in the left eye image and the right eye image; the automatic identification and positioning of different parts of human eyes can be realized; the automatic measurement of the eye parameters which are often required to be measured by an ophthalmologist is realized; provides parameter assistance and support for the ophthalmologist to analyze the eye disease condition.

It should be noted that, the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, and the like in the system, and those skilled in the art can implement the step flow of the method by referring to the technical scheme of the system, that is, the embodiment in the system can be understood as a preferred example of the implementation method, and details are not described herein.

Those skilled in the art will appreciate that, in addition to implementing the system and its various devices provided by the present invention in purely computer readable program code means, the method steps can be fully programmed to implement the same functions by implementing the system and its various devices in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices thereof provided by the present invention can be regarded as a hardware component, and the devices included in the system and various devices thereof for realizing various functions can also be regarded as structures in the hardware component; means for performing the functions may also be regarded as structures within both software modules and hardware components for performing the methods.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.

Claims

1. A human eye parameter measurement method based on deep learning is characterized by comprising the following steps:

2. The method for measuring human eye parameters based on deep learning of claim 1, wherein the extracting left and right eye images in a human face picture comprises:

3. The method according to claim 1, wherein the deep neural network employs a multitasking convolutional neural network including corneal block, scleral block and medial and lateral canthal blocks; wherein:

the corneal block predicts a bounding box and a confidence at each pixel position using pixel-based prediction, the confidence reflecting the probability of whether the pixel is within the cornea. After non-maximum inhibition, calculating the average value of all the bounding boxes to be used as a final prediction result;

4. The method for measuring human eye parameters based on deep learning of claim 1, wherein the number of the eye parameters is 10, comprising: palpebral fissure height, corneal sagittal diameter, corneal transverse diameter, corneal onversion, supraeyelid retraction, infraeyelid retraction, pupil diameter, inferior limiting amount, superior limiting amount, and external limiting amount.

5. The method of measuring human eye parameters based on deep learning of claim 4, wherein the eye parameters are calculated by fitting a quadratic function to the eyelids and calculating the scale of the physical world and the image size in combination with the sticker.

6. The deep learning based human eye parameter measurement method of claim 5, wherein the quadratic function fitting eyelid method comprises:

p(y-cy)＝±(x-cx)² (1)

p＝d₁+d₂ (2)

A method for calculating a scale of a physical world and an image size using a sticker, comprising:

the method for calculating the eye parameters comprises the following steps:

7. A human eye parameter measurement system based on deep learning, comprising:

8. The system according to claim 7, wherein the eye image extraction module processes the face image by using open source engineering and extracts a plurality of eye key points in the face image, and extracts complete left and right eye images according to positions of the eye key points.

9. The deep learning-based eye parameter measurement system according to claim 7, wherein in the eye part identification module, the deep neural network is a multitask convolutional neural network comprising a corneal block, a scleral block and inner and outer angular blocks; wherein:

10. An apparatus comprising a memory, a processor and a computer program stored on the memory and operable on the processor, wherein the computer program, when executed by the processor, is operable to perform the method of any of claims 1 to 6.