GB2325318A - Processing information in a plurality of processing stages - Google Patents

Abstract

Processing digital information, such as image or audio information, using a data flow type architecture. Input data elements are sequentially processed by a plurality of nodes, 10, 2, 12, and at a final stage the processed data is output 12, to storage or for remote transmission. Each node is activated by means of one or more triggers. Each input-only node 10 or input-output node 2 can trigger one or more successive input-output or output-only nodes, thus enabling parallel processing. Also, a processing node may require more than one trigger to start processing, thus the results of previous processing stages can be combined. Other possible node triggers include self triggers which can be activated internally within a node and external (e.g. user-initiated) triggers. Buffer means (6, fig. 1) associated with the nodes store the processed data elements to prevent corruption by subsequent processing. Security and fire alarm applications are described (figs. 5 and 6).

Description

APPARATUS FOR TEE MWNAGEMkw-s OF DIGITAL INFORMATION This invention relates to apparatus for the management of digital information.

Apparatus for the management of digital information is known. A limiting factor in the performance of the known apparatus is the communication means used to pass data between processing stages. This data may be the data being processed by the processing stages, or it may be some external controlling data that is influencing the processing stages in some way. Also, the limiting factor may influence the ability of the apparatus to operate when the volume of data being processed exceeds some limit. This is known as congestion and it is a similar problem to the management of car traffic on a road system.

It is an aim of the present invention to obviate or reduce the above mentioned problems.

Accordingly, in one non-limiting embodiment of the present invention there is provided apparatus for the management of digital information, which apparatus comprises means for creating a plurality of data elements, a plurality of node means for processing the data elements in a data flow architecture, trigger means for inhibiting operation of the node means, and buffer means for storing the data elements such that processed data elements are not corrupted by subsequent processing by the node means.

The apparatus of the present invention is able to operate without the above mentioned limitations.

Thus, the apparatus of the present invention is suited to situations where large amounts of data may need to be processed by large numbers of processing stages.

The apparatus is able to operate by creating an image, passing that image to a processing stage, processing the image at the processing stage and passing the processed image on to the next processing stage, and repeating the processing until a final stage. The final stage may be a viewing stage. One stage can be effective to trigger more than one next stage so that in essence the processing of the images can proceed in parallel. Similarly, a processing stage may require more than one trigger to start processing. Thus the results of previous processing stages can be combined.

This type of processing of data images is known as data flow in computer jargon. It is believed that concept of using data flow for the processing of images has not previously been used. The concept of using the data flow for the processing of images enables the apparatus of the present invention to provide the building blocks required to build complex systems, for example for the Internet.

The term data flow describes a computer architecture where the outputs of one node trigger activity in subsequent nodes. Thus the apparatus of the present invention enables information to be transmitted between nodes, with each node having the task of calculating some function upon its inputs and producing an output containing a result.

The apparatus of the present invention offers a number of advantage over conventional computer architecture because it 'is inherently parallel in nature. This enables the apparatus of the present invention to afford the considerable performance gains over classical Von Nuemann computer architectures.

The apparatus of the present invention makes it relatively easy to create and modify complex systems for the management of digital information.

The data elements may be image elements. There may be several types of image elements. An image element can contain any type of digital information.

The image element may be a graphical image of an entire scene or part of a scene.

The data elements may alternatively be such that they contain some other desired type of digital information, for example audio information so that the data elements would then be audio elements. If desired, the data elements may be multimodal data elements.

Preferably, the apparatus is such that the data elements are held in a format which is such that the data elements are compressed, uncompressed, provided with monochrome colour etc. This format may however vary.

If desired, additional data structures may accompany the data elements. The additional data structures may vary depending upon the type of information described in a data element, and the operations that will be performed upon that data element.

The node means may comprise input only nodes input-output nodes, and output only nodes.

The data elements pass between the various nodes.

The functions performed upon the data elements are digital transforms. The mechanisms used to trigger the passage of the data elements between nodes are appropriate for digital processing.

As indicated above, the data elements are not limited to image elements. Data representing any kind of external event may be passed between the nodes. An example of non-image elements is an audio sample containing megabytes of data, or the state of a light switch which is one data bit long.

The input only nodes may be such as to originate a data element. The data element may be entirely computer generated or it may be a data element acquired from a real world scene, for example from a video camera with digital capture. Once a data element has been created by the input only node, the data element will be moved to an output data element buffer. This will trigger the next node in the sequence to accept this output data element as an input data element.

The input-output nodes will normally be triggered by an input data element. The input-output nodes will each perform a transformation upon the input data element to generate an output data element. This will trigger the next node in the sequence to accept this output data element as an input data element.

The output only node will represent the final stage in a sequence of nodle stages. The output only node will be triggered by an input data element. The output only node may perform a transformation upon the input data element. The data element will then be despatched to its final destination or destinations.

The resultant data element may be archived to some storage medium or it may be transmitted over a communications medium to some remote point.

The buffer means is employed to ensure that a buffer that is being processed by one node, is not corrupted while it is being processed by subsequent nodes in a sequence or graph of nodes.

Each input only node or input-output node can trigger one or more successive input-output or output only nodes. Thus, a data element can be manipulated, transformed or transmitted by one or more nodes.

The apparatus may be one in which the trigger means comprises self triggers which are able to be activated internally within a node, and/or external triggers which are activated externally, and/or template triggers which describe operational characteristics of a node, and/or feedback triggers, and/or lazy triggers that can be ignored when desired.

The self-triggers may be activated internally within a node. The triggering may be generated based on the completion of an external physical event such as a timer, or the end of a video capture.

The external triggers may be triggered from outside the local environment where, for example, the information passes into a node from a user entering a command at a keyboard.

The template triggers may be special triggers which describe operational characteristics of a node.

An example is a node that compresses an input data element. A template trigger would describe the parameters affecting compression ratios.

The feedback triggers may be special triggers that, when activating a node, can be used by the activated node to signal that the processing of that triggered event is complete. The feedback triggers can be used to ensure that a node completes its processing successfully before the next trigger is raised.

The lazy triggers are special triggers that can be ignored. The lazy triggers are used because, in image processing, skipping an image frame is often acceptable. What is more important is that the current image being processed, is the most recent image. An example is a system that triggers an image capture 25 times a second but where the next node can only process 5 images a second. The effect of using lazy triggers is that the second stage can skip elements.

The trigger means are designed to inform a node that there is a data element available for processing.

The trigger means provides information about the location of the nodes input data element, and also status and control information regarding operations that may be applied to the data element, and operations that are being applied to a data element by a node.

Depending upon the status of a trigger, a node will perform appropriate operations on the data element. Status information may define whether the data element is read-only or read-write. A read-only data element would be one where multiple downstream nodes may be triggered to process that data element and, because of this, no one node may alter the output data element as any such transforms would conflict with the processing by other nodes. Read-write nodes would be ones where only a single node is required to process the data element, and there would be a performance advantage in manipulating the node in situ. Other types of status information may be assigned to a nodes data element.

The trigger means is effectively the mechanism by which a node is activated to start processing a data element or data elements. The nodes may have multiple input triggers. Node processing will not start until a predetermined subset of triggers has been activated.

This subset might be all the input triggers or just one.

Once a node has been triggered, the node accesses its input data elements. The trigger status is readonly, then no transforms can be performed on the data element. Instead the node has to perform any such transforms in its own memory space, that is the node would have to manipulate a copy of the data element.

Once processing of a node's data element has been completed, then the node assigns this data element to an output area and raises a trigger to initiate the next node or nodes in the sequence. The key features are thus that one node can be triggered by one or more input triggers, and also one node can trigger one or more subsequent nodes. There may be several types of triggering events, and these may or may not contain references to data elements.

The apparatus of the present invention can be used to describe complex image processing mechanisms in a simple way. The resulting descriptions can be implemented in highly parallel computer architectures.

The apparatus is not limited to the processing of image information and it can be applied to a number of other types of related information processing such as audio. 'Whilst data flow architectures are not novel, it is believed that using data flow architectures for digital processing and manipulation of data such as image and audio data is novel.

Embodiments of the invention will now be described solely by way of example and with reference to the accompanying drawings in which: Figure 1 illustrates schematically the operation of an input-output node; Figure 2 illustrates schematically the operation of an input node; Figure 3 illustrates schematically the operation of an ouput node; Figure 4 illustrates schematically the flow of information through various nodes in apparatus for the management of digital information; Figure 5 shows first apparatus for the management of digital information; and Figure 6 shows second apparatus for the management of digital information.

Referring to Figure 1, there is shown an input-output node 2 receiving inputs via lines 4. The input-output node 2 is connected to a buffer or storage device 6. The input-output node 2 can have inputs and it can output. The input-output node 2 can signal an event dependent upon one or more inputs.

The input-output node 2 can have internal storage, usually a memory. The inputs on the lines 4 will be from prior nodes.

Referring now to Figure 2, there is shown a buffer or storage device 6 inputting information via lines 8 to an input node 10. The inputs are physical, for example from a camera, a disc or a memory. The input node 10 will usually be continuously running.

The input node 10 may signal an event. The signalled event will usually be dependent upon time.

Referring now to Figure 3, there is shown an output node 12 receiving inputs via lines 14 and outputing to a buffer or storage device 6 via line 16.

The output node 12 is such that its output is always physical, for example to'a disc, a visual display unit or a memory. Inputs to the output node 12 are from a prior node. There is no signalling due to there being no subsequent nodes after the output nodes 12.

Referring now to Figure 4, there is shown a simple system using an input node 10, an input-output node 2, and an output node 12. Storage devices 18, 20 are shown. As can be seen from Figure 4, the input to the storage device 18 is physical, for example from a camera, a disc or a memory. The input node 10 i8 such that there is no prior node in the apparatus. The input-out node 2 is able to receive an input and is able to output. The output node 12 receives an input only. There is no subsequent node after the output node 12. The storage device 20 can be any suitable and appropriate physical storage device such for example as a disc, a visual display unit or a memory.

Referring now to Figure 5, there is shown first apparatus 22 for the management of digital information. The apparatus 22 comprises two cameras 24, 26 and an infrared camera 28. The cameras 24, 26, 28 input information via lines 30, 32, 34 to a web-cam 36. As shown in Figure 5, the web-cam 36 is able to output to a disk 38, a telephone 40, a fire alarm 42 or the Internet 44. The telephone 40 is connected to a pager 46. The pager 46 may be on a security guard 48. The fire alarm 42 is shown connected to a fire station 50. The Internet 44 is shown connected to a GSM telephone 52 and the telephone 52 is shown connected to a lap top computer 54. The telephone 52 and the lap top computer 54 can be installed in a fire engine, so that images can be down loaded in the fire engine as the fire engine is on its way to a fire.

The apparatus enclosed by the broken line 56 in Figure 5 can all be located in a building such for example as a school, an office or a factory. The cameras 24, 26, 28 can offer substantially automatic security surveillance. If an emergency situation occurs, then the appropriate persons/organisations are informed, for example the security guard 48 and the fire station 50 in the event of a fire.

Figure 6 shows second apparatus 58 for the management of digital information. The apparatus 58 shown in Figure 6 is somewhat similar to the apparatus 22 shown in Figure 5 and similar parts have been given the same reference numerals for ease of comparison and understanding.

Figure 6 shows the presence of a passive infrared detector 60 and a heat sensor 62. The camera 24 is shown linked to a processing node 62. The processing node 62 is a motion node which detects movement in an image and reports time and snapshot of the image. The cameras 24, 26 are shown linked to a processing node 64. The processing node 64 is an overlay node which takes images and overlays them in order to produce one image output.

A colour filter 66 is employed to remove all colours except specified colours. The colour filter 66 is linked to a percentage area node 68 which calculates the area covered by individual colours as a percentage. An OR gate 70 produces an output from an input that triggered. An If-node 72 operates such that if one input triggers, then the second input on a main path 74 is used as an output.

As shown in Figure 6, nodes 76 act as image out nodes, each of which produces computer viewable images. Three video recorders 78 are employed as shown.

A contract logger 80 logs information from triggered inputs to the disk 38. A pager 82 pages a standard number for the telephone 40 if the input is triggered.

An override 84 displays an input 1 unless an input 2 on line 86 is triggered, in which case the override 84 will then output the input 2. The output goes to a visual display unit 88.

An alarm node 88 goes to the fire alarm 42. The laptop 54 is for use in a fire engine 90. A television tuner 92 inputs to the override 84.

In order that the operation of Figure 6 may be fully understood, the following scenario is given by way of example only.

A building is fitted with the two cameras 24, the infrared camera 28, the passive infrared device 60 and the heat sensor 62. All of these devices require certain actions to be taken in certain circumstances, for example as follows.

1. If there is an intruder in the building, then the security guard 48 needs informing via the pager 46.

2. If a fire starts, then the fire brigade needs to be called.

3. If circumstances No.1 or No.2 above occur, then the appropriate cameras need recording to video tape.

4. One of the cameras will cover the main entrance to the building and this camera needs to be recording on to tape continuously.

5. If the fire brigade has an Internet-based viewing system that requires the fire brigade to see all the cameras 24, 28, then the infrared camera image should advantageously be overlaid on to a normal image in order to aid location of the fire.

6. If the security guard 48 should be playing games on the computer 86, then the computer 86 is configured automatically to show video camera images in the event that alarms should be raised. This will be effected regardless of the current task of the computer 86.

7. The apparatus 58 is configured such that if the infrared camera 28 has more than a certain percentage of its image white (white is used to represent "very hot") or if the heat sensor 62 is triggered, then the alarm will sound, and the video camera (overlaid with the infrared camera 28) will be recorded. If the passive infrared device 60 detects movement or if the apparatus 58 can detect movement in the video camera image, then the security guard 48 is paged and the camera will operate in a recording mode.

The apparatus of the present invention is suited to situations where large amounts of data may need to be processed by large numbers of processing stages.

By way of example, it is mentioned that a video capture device is capturing frames of television images at 25 frames per second. These images (data elements) are transferred to a second processing element that is comparing the current and a previous element to identify changes between the two images, ie looking for motion. The second processing element is only able to process five images per second. The apparatus of the present invention is able to use a lazy trigger between the two processing stages. The second processing stage would then ignore all triggered data inputs except the latest one. This is acceptable when processing video and other real time data elements as the apparatus is only interested in the latest information, and all prior information is of lesser value. The overall result is that the apparatus is limited by the slowest stage to process images at 5 frames per second.

It is to be appreciated that the embodiments of the invention described above with reference to the accompanying drawings have been given by way of example only and that modifications may be effected.

Thus, for example, the apparatus of the present invention can be used in a wide range of security applications.

Claims (11)

1. Apparatus for the management of digital information, which apparatus comprises means for generating a plurality of data elements, a plurality of node means for processing the data elements in a data flow architecture, trigger means for initiating operation of the node means, and buffer means for storing the data elements such that processed data elements are not corrupted by subsequent processing by the node means.

2. Apparatus according to claim 1 in which the data elements are image elements.

3. Apparatus according to claim 1 in which the data elements are audio elements.

4. Apparatus according to claim 1 in which the data elements are multimodal data elements.

5. Apparatus according to any one of the preceding claims 'in which the node means comprises input only nodes, input-output nodes, and output only nodes.

6. Apparatus according to any one of the preceding claims in which the trigger means comprises self triggers which are able to be activated internally within a node.

7. Apparatus according to any one of the preceding claims in which the trigger means comprises external triggers which are activated externally.

8. Apparatus according to any one of the preceding claims in which the trigger means comprises template triggers which describe operational characteristics of a node.

9. Apparatus according to any one of the preceding claims in which the trigger means comprises feedback triggers.

10. Apparatus according to any one of the preceding claims in which the trigger means comprises lazy triggers which are able to be ignored when desired.

11. Apparatus for the management of digital information, substantially as herein described with reference to the accompanying drawings.