GB2471481A - Data path establishment for client initiated source to sink data transfer - Google Patents
Data path establishment for client initiated source to sink data transfer Download PDFInfo
- Publication number
- GB2471481A GB2471481A GB0911338A GB0911338A GB2471481A GB 2471481 A GB2471481 A GB 2471481A GB 0911338 A GB0911338 A GB 0911338A GB 0911338 A GB0911338 A GB 0911338A GB 2471481 A GB2471481 A GB 2471481A
- Authority
- GB
- United Kingdom
- Prior art keywords
- data
- data path
- clients
- module
- path process
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000012546 transfer Methods 0.000 title claims abstract description 10
- 238000000034 method Methods 0.000 claims abstract description 59
- 230000008569 process Effects 0.000 claims abstract description 48
- 238000004891 communication Methods 0.000 claims abstract description 11
- 238000004590 computer program Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 description 8
- 239000000872 buffer Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 230000035484 reaction time Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011982 device technology Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
- G06F13/14—Handling requests for interconnection or transfer
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mobile Radio Communication Systems (AREA)
Abstract
A data path module establishes a data path process for transferring data between data sources and data sinks; and a client initiates data transfer. This may be used in a computing device which must meet soft real-time deadlines, for example in Voice over IP (VoIP) or digital video (DVB) on a mobile phone which suffer from delay, jitter and latency. The data path process may be isolated from the clients and service modules and use asynchronous communications. The service modules may establish the data path process as a layer between the kernel and the middleware.
Description
AN APPARATUS AND A METHOD FOR DATA TRANSFER
TECHNICAL FIELD
This invention relates to an apparatus and a method for data transfer.
BACKGROUND TO THE INVENTION
Certain operations carried out by a computing device must comply with soft real-time deadlines. That is, deadlines which should be complied with in order for a user to perceive satisfactory device performance. If such deadlines are missed, perceived device performance decreases, but the device will continue to operate. In contrast, hard real-time deadlines are ones which result in device failure if missed.
Operations which includes soft real-time deadlines include Voice over IP and digital video such as DVB. Real-time operations have three main characteristics; namely delay, latency and jitter. Delay is the time taken for the system to respond to an event. It should be less than the deadline for that event. Latency is the system reaction time, i.e. the time taken for the system to respond to an event. Jitter is the unwanted variation in delay and latency.
In the above examples, data is transferred between two hardware components. In order for data to be transferred between two hardware components in a computing device, a data path must be established between those components. Data transfer and the data path should arranged to reduce delay, latency and jitter.
SUMMARY OF EXAMPLES OF THE INVENTION
An example of the invention provides an apparatus comprising: a plurality of data sources and a plurality of data sinks; a data path module arranged to establish a data path process for instantiating a data path for transferring data between said data sources and said data sinks; and a client arranged to initiate data transfer from said data sources to said data sinks.
The data path process may be isolated from said clients.
The apparatus may further comprise at least one service module and control of said data path process is provided by said at least one service module.
The apparatus may further comprise an operating system, having a kernel which includes a plurality of interfaces to said data sources and data sinks.
The data path process may be arranged to communicate with said plurality of interfaces.
Communication between the service modules and the data path process may be asynchronous.
The data path process may provide soft real-time guarantees.
An instantiated data path may not pass through the service modules or the clients.
A plurality of data paths may be instantiated at any one time and each data path may be isolated from the other data paths.
The service modules may be further arranged to establish said data path process as a layer between the kernel and the middleware.
The clients may only access a control plane of the data path.
The clients may be isolated from a data plane of the data process.
The clients and the data path may include capabilities and the clients do not require the same level of capabilities required by the data path.
The data path process may be a single process. The data path process may be a single thread.
The apparatus may be a mobile phone.
Another example of the invention provides a method for transferring data on an apparatus, the apparatus comprising: a plurality of data sources and a plurality of data sinks; a data path module; and a client, the method comprising: receiving a request from said client to establish a data path from the data source to the data sink; the data path module establishing a data path process; and the data path process instantiating a data path.
Another example of the invention provides a computing device which may comprise: a memory, arranged to store instructions; a processor, arranged to carry out said instructions; a plurality of said clients, stored in said memory.
A further example of the invention provides a computer program or a suite of computer programs arranged such that when executed by a computer they cause the computer to operate in accordance with the method described above.
A further example of the invention provides a computer readable medium storing the computer program, or at least one of the suites of computer programs.
A further example of the invention provides an operating system for causing a computing device to operate in accordance with a method described above.
A further example of the invention provides a device substantially as described herein and as shown in Figures 1 to 3.
This summary provides examples of the invention which are not intended to be limiting on the scope of the invention. The features of the invention described above and recited in the claims may be combined in any suitable manner. The combinations described above and recited in the claims are not intended to limit the scope of the invention.
Features and advantages associated with the examples of the invention will be apparent from the following description of some examples of the invention.
BRIEF DESCRIPTION OF THE DRAWINGS
Examples of the invention are hereinafter described with reference to the accompanying diagrams where: Figure 1 is a schematic diagram of the components of a mobile telephone in an example of the invention; Figure 2 is a functional diagram showing some components of the device shown in Figure 1; and Figure 3 is a flow chart showing method in an example of the invention.
DESCRIPTION OF EXAMPLES OF THE INVENTION
The following example of the invention is described in the context of a mobile telephone. It will be appreciated that the invention may encompass other computing devices. Figure 1 is a schematic diagram showing some of the components of a mobile telephone 101. The components of the mobile telephone 101 include a processor 102, which is arrange to carry out instructions stored as computer programs on the telephone. The telephone 101 also includes a system bus 103 which connects the processor 102 to other components of the device. The bus 103 allows the components to communicate with each other. Here, the components are shown to communicate via a single system bus 103, however, in practice, the mobile telephone 101 may include several buses to connect the various components.
The telephone 101 also includes an speaker 104, a microphone 105, a keypad 106 and a display 107. These components may also include respective device controllers. The mobile telephone 101 also memory components including a ROM 108, a RAM 109 and a storage device 110. The storage device may be an internal hard drive or a removable storage device such as a flash memory card. The ROM 108 has an operating system stored thereon. The operating system is for controlling the operation of the device. The RAM 109 is used while the device is switched on to store temporary data. The telephone 101 also includes a radio 111 and an antenna 112. The radio 111 and antenna 112 aflow the telephone to communicate with a mobile phone network in a manner familiar to the person skilled in the art. In addition, the telephone 101 includes a DVB receiver 113, for receiving digital video signals.
This description of the components of a telephone 101 is one example of the manner in which the components may be arranged. Many variations are possible to including different components and different arrangements of those components.
The invention is not limited to any particular set of components nor to any particular combination of those components. Advances in computing device technology may result in certain components being replaced by others which perform the same function. Such a device could also embody the invention.
The mobile telephone 101 includes an operating system (OS) which is stored in ROM 108. The OS may be the Symbian OS. The device also includes other software applications which may be stored in ROM or which may be stored in the storage device 110. The processor 102 is arranged to execute instructions of the OS and of the applications. Execution of these instructions causes mobile device 101 to carry out particular functions by controlling the various hardware components of the device.
Figure 2 is a functional diagram showing the logical links between software and hardware components of the telephone 101. The operating system includes a kernel 202. The kernel 202 is arranged to manage the mobile telephone's hardware resources and communications between hardware and software stored on the device. The kernel 202 includes a sound driver 203, for controlling communications with the speaker 104, a graphics driver 204, for controlling communications with the display 105, and DVB driver 205, for controlling communications with the DVB receiver 113. The kernel may include other modules. For example, the kernel may include other device drivers.
The operating system also includes various middleware components including a multimedia services module 206 and a comms services module 207. The multimedia services module 206 is for controlling the multimedia based components of the telephone 101; for example the speaker 104 and display 105. The comms services modules 207 is for controlling communications based components of the telephone 101, for example the DVB receiver 113. Each module includes a control component and a management component. In this case, the multimedia services module 206 includes a multimedia control component 206a and a multimedia management component 206b. The comms services modules 207 includes a comms control component 207a and a comms management component 207b.
As will be seen in more detail below, the control components are for setting up, controlling and closing data paths based on metadata prepared by the management components and instructions received from multimedia clients. The management components include data path metadata, configuration settings and use-case/application priorities These elements will be described in more detail below.
As can be seen, the control and management functions are separated within each service module. Control functions are referred to generally as the control plane and management functions are referred to generally as the management plane. As will be seen below, data path functions are referred to as the data plane..
The telephone 101 also includes clients. Clients may be part of the operating system or may be separate to it. In this example, the telephone includes a DVB client 208. The DVB client 208 is for selecting digital content received via the DVB receiver 113 and for playing this to a user. The DVB client 208 communicates with the multimedia module 206 and the comms module 207 via APIs of those modules.
The DVB client 208 communicates with the comms module 207 to control the DVB receiver 113. The DVB client 208 communicates with the multimedia module 206 to control playback of digital content via the display 107 and speaker 104. In particular, the DVD client 208 sends instructions to the control components of each services module.
The telephone also includes a data path module 209. The data path module 209 is for establishing, controlling and ending the transfer of data between different hardware components. The control components of the multimedia module 206 and comms module 207 instruct the data path module to establish a data path when required. More than one data path may be established at any one time. Data paths are isolated from each other. Communications between the control components of the multimedia module 206 and the comms module 207 and the data path module 209 are asynchronous.
The data path module 209 represents the data plane. The data path module 209 isolates the middleware and clients from the physical data path. In this sense, the middleware and clients are only exposed to the control and management planes.
Messaging between the data plane and the control plane is asynchronous. This prevents and non-deterministic messaging interrupting the data flow in the data path.
An example of how the above described components operate will now be described with reference to Figure 3. This example shows how a data path is created between different hardware components. In particular, it shows the streaming of data from the DVB receiver 113 to the speaker 104 and display 107.
In the present example, a user of the telephone 101 may choose digital video to watch using the DVB client 208. The DVB client 208 may include a browser which enables a user to see what video content is available to view via the DVB receiver 113. The received video data must be transferred to the display 107 and to the speaker 104. A data path process must be established by the data path module 209 to control this flow.
The DVB client 208 sends instructions to the multimedia control component 206a to establish a data path process for the new data flow (block 301). The multimedia control component 206a send instructions to the data path module 209 to instantiate a data path process for the new data flow (block 302). The data path process then creates the new data path process suitable for receiving data from the DVD receiver 113 (the data source) and sending data to the display 107 and speaker 104 (the data sinks) (block 303). The data path module 209 then signals back to the multimedia control component 206a that the data path process is established and ready to commence data flow (block 304).
The multimedia control component 206a then signals the DVD client 208 that data path process is established (block 305). The DVB client 208 sends instructions to the comms control component 207a indicating that the data path process is established (block 306). The comms control component 207a the instructs the data path module 209 to begin data transfer (block 308). The DVB receiver 113 then starts sending data, via the data path process, to the speaker 104 and display 107 (block 309).
As noted above, communications between the multimedia control component 206a, the comms control component 207a and the data path module 209 are asynchronous. Asynchronous communication reduces the likelihood of any control signals between the multimedia module 206, the comms module 207 and the data path process interfering with the flow of data from the DVB receiver 113 to the display 107 and speaker 104.
The kernel 202 provides a shared buffer memory in RAM 109. This allows device drivers in the kernel to allocate and own their own memory chunks. This means that device drivers can provision aligned buffers so that memory address and memory length can be set such that direct memory access (DMA) transfer can occur. This prevents additional data copy within the drivers. In the present case, the DVB driver 205 owns a first memory chunk and the graphics driver 204 owns a second memory chunk. In the data path process, a single copy between the two memory chunks occurs. This enables hardware acceleration or hardware off-load in the data path between the two drivers to occur.
An alternative to the above is that the control components of the multimedia and comms modules handle memory allocation. In the present case, the multimedia control component owns a first memory chunk and the comms control component owns a second memory chunk. DMA buffers are provided by each device driver. In the data path process, a first copy between the DVB driver 205 DMA buffer and the second memory chunk occurs. A second copy between the second and first memory chunks occurs. A third copy between the first memory chunk and DMA buffer of the graphics driver 204 occurs. The same memory chunk can be used regardless of which driver is source or sink of the data. Traffic outside the data path process can share the same pools. This makes for a simpler implementation.
The data flow may be terminated either by the user or by the video content reaching an endpoint. In either case, the multimedia module 206 instructs the data path module 209 to stop the data flow. Once the data flow has stopped, the data process is ended.
In the above example, a data flow between a DVB receiver and a display and speaker is described. Data flows may also occur between a camera device, a GPS receiver, a file, the baseband, a WiFI unit, amongst others.
A benefit of this example is that the data flow is isolated from the middleware (multimedia and comms services). The data is transferred in a so-called data plane which is hidden from the middleware and application layer. This arrangement is preferred for real-time services in which data flow interruptions must be kept to a minimum.
The use of a separate data path process is beneficial for real-time services. Real-time services may be characterised in three ways. Firstly, system delay must be low. This is the time from an event occurring to a system response. This must be shorter or equal to a given deadline. Secondly, latency must be low. Latency is the reaction time of the system. Finally, jitter must be low. Jitter is variation in an event period, delay or latency and should be avoided. Using a separate data path mitigates each of these factors.
Real-time services includes, Voice over IP, LBS location update, playing music through a speaker from a local file or remote source, streaming video, etc. Non-real time services include music and other file downloads. Using a separate data path enables real-time services to continue without interruption from non-real-time services.
As will be appreciated from the above, the functions of the telephone include data, control and management. These functions can be thought of as planes which operate in conjunction but separately from each other. The data path process operates in the data plane and is isolated from the clients and middleware. The middleware and clients only have access to the control and management planes.
The above examples of the invention describe a software implementation of the invention. Other examples of the invention include a hardware only implementation.
A example of the invention includes a component on a chip which provides the functionality described above in connection with the software implementation.
An example of the invention is an apparatus as defined in the claims. This apparatus may be a component provided as part of a chip on an electronic circuit board. Alternatively the apparatus may be a chip on an electronic circuit board. As a further alternative, the apparatus may be a computing device, such as a mobile phone. The features defined in the claims may be implemented in hardware.
Alternatively, the features may be implemented using software instructions which may be stored in a memory provided on the component, chip or computing device.
A further example of the invention provides an apparatus having means for establishing a data path process for instantiating a data path from a data source to a data sink, in response to instructions from a client.
An advantage of the above examples is that, by confining processing of a data flow within a single process, the performance of that flow can be better controlled and thread priority contention problems can be mitigated.
A data path module may be a dedicated module for establishing, controlling and stopping data path processes. Alternatively, a data path module may be part of another module, for example the multimedia services module or comms module.
Another advantage of isolating the data path from the applications relates to secunty.
For example, if an application is used for streaming a DRM (digital rights management) protected stream, it would not be able to gain access to the content, as this is isolated from the application.
Various modifications, changes, and/or alterations may be made to the above described examples to provide further examples which use the underlying inventive concept, falling within the spirit and/or scope of the invention. Any such further examples are intended to be encompassed by the appended claims.
Claims (21)
- Claims 1. An apparatus comprising: a plurality of data sources and a plurality of data sinks; a data path module arranged to establish a data path process for instantiating a data path for transferring data between said data sources and said data sinks; and a client arranged to initiate data transfer from said data sources to said data sinks.
- 2. An apparatus according to claim 1, wherein said data path process is isolated from said clients.
- 3. An apparatus according to claim 1, further comprising at least one service module and wherein control of said data path process is provided by said at least one service module.
- 4. An apparatus according to claim 3, further comprising an operating system, having a kernel which includes a plurality of interfaces to said data sources and data sinks.
- 5. An apparatus according to claim 4, wherein said data path process is arranged to communicate with said plurality of interfaces.
- 6. An apparatus according to any proceeding claim, wherein communication between the service modules and the data path process is asynchronous.
- 7. An apparatus according to claim 1, wherein said data path process ensures soft real-time guarantees.
- 8. An apparatus according to claim 1, wherein an instantiated data path does not pass through the service modules or the clients.
- 9. An apparatus according to claim 1, wherein a plurality of data paths may be instantiated at any one time and each data path is isolated from the other data paths.
- 10. An apparatus according to claim 4, wherein said service modules are further arranged to establish said data path process as a layer between the kernel and the middleware.
- 11. An apparatus according to claim 1 wherein said clients may only access a control plane of said data path.
- 12. An apparatus according to claim 11, wherein said clients are isolated from a data plane of said data process.
- 13. An apparatus according to claim 1, wherein said clients and said data path require includes capabilities and said clients do not require the same level of capabilities required by said data path.
- 14. An apparatus according to claim 1, wherein said data path process is a single process.
- 15. An apparatus according to claim 13, wherein said data path process is a single thread.
- 16. A computing device according to any preceding claim, wherein said apparatus is a mobile phone.
- 17. A method for transferring data on an apparatus, the apparatus comprising: a plurality of data sources and a plurality of data sinks; a data path module; and a client, the method comprising: receiving a request from said client to establish a data path from the data source to the data sink; the data path module establishing a data path process; and the data path process instantiating a data path.
- 18. A computer program or suite of computer programs arranged such that when executed by a computer they cause the computer to operate in accordance with the method of claim 17.
- 19. A computer readable medium storing the computer program, or at least one of the suites of computer programs, according to claim 18.
- 20. An operating system for causing a computing device to operate in accordance with a method as claimed in claim 17.
- 21. An apparatus substantially as described hereinbefore and as shown in Figures 1 to 3.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB0911338A GB2471481A (en) | 2009-06-30 | 2009-06-30 | Data path establishment for client initiated source to sink data transfer |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB0911338A GB2471481A (en) | 2009-06-30 | 2009-06-30 | Data path establishment for client initiated source to sink data transfer |
Publications (2)
Publication Number | Publication Date |
---|---|
GB0911338D0 GB0911338D0 (en) | 2009-08-12 |
GB2471481A true GB2471481A (en) | 2011-01-05 |
Family
ID=41008514
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
GB0911338A Withdrawn GB2471481A (en) | 2009-06-30 | 2009-06-30 | Data path establishment for client initiated source to sink data transfer |
Country Status (1)
Country | Link |
---|---|
GB (1) | GB2471481A (en) |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4627070A (en) * | 1981-09-16 | 1986-12-02 | Fmc Corporation | Asynchronous data bus system |
US6253245B1 (en) * | 1997-04-25 | 2001-06-26 | U.S. Philips Corporation | Transmission system with buffer between independently controlled data paths |
US6938093B2 (en) * | 2001-07-06 | 2005-08-30 | Juniper Networks, Inc. | Bandwidth allocation for a data path |
US20070226529A1 (en) * | 2006-03-21 | 2007-09-27 | Mediatek Inc. | Memory controller and device with data strobe calibration |
EP2034655A2 (en) * | 2007-05-30 | 2009-03-11 | Silicon Storage Technology, Inc. | Systems and methods for transmission and reception of data Including processing and buffering features |
US20090132747A1 (en) * | 2007-11-19 | 2009-05-21 | International Business Machines Corporation | Structure for universal peripheral processor system for soc environments on an integrated circuit |
KR20090054780A (en) * | 2007-11-27 | 2009-06-01 | 한국전자통신연구원 | Apparatus and method for communication between processors in multiprocessor soc system |
-
2009
- 2009-06-30 GB GB0911338A patent/GB2471481A/en not_active Withdrawn
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4627070A (en) * | 1981-09-16 | 1986-12-02 | Fmc Corporation | Asynchronous data bus system |
US6253245B1 (en) * | 1997-04-25 | 2001-06-26 | U.S. Philips Corporation | Transmission system with buffer between independently controlled data paths |
US6938093B2 (en) * | 2001-07-06 | 2005-08-30 | Juniper Networks, Inc. | Bandwidth allocation for a data path |
US20070226529A1 (en) * | 2006-03-21 | 2007-09-27 | Mediatek Inc. | Memory controller and device with data strobe calibration |
EP2034655A2 (en) * | 2007-05-30 | 2009-03-11 | Silicon Storage Technology, Inc. | Systems and methods for transmission and reception of data Including processing and buffering features |
US20090132747A1 (en) * | 2007-11-19 | 2009-05-21 | International Business Machines Corporation | Structure for universal peripheral processor system for soc environments on an integrated circuit |
KR20090054780A (en) * | 2007-11-27 | 2009-06-01 | 한국전자통신연구원 | Apparatus and method for communication between processors in multiprocessor soc system |
Non-Patent Citations (1)
Title |
---|
Multipath routing with adaptive playback scheduling for Voice over IP in Service Overlay networks, Hong Li, L.Mason, Sarnoff Symposium, 2008, IEEE 28-30 April 2008 pages 1-5 ISBN: 978-1-4244-1843-5, INSPEC Accession number 10195337 * |
Also Published As
Publication number | Publication date |
---|---|
GB0911338D0 (en) | 2009-08-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
USRE48323E1 (en) | Media processing method and device | |
US8200479B2 (en) | Method and system for asymmetric independent audio rendering | |
WO2021254330A1 (en) | Memory management method and system, client, server and storage medium | |
US9411637B2 (en) | Adaptive process importance | |
US9191417B2 (en) | Cross-process media handling in a voice-over-internet protocol (VOIP) application platform | |
KR102029806B1 (en) | System-on-chip and application processor including fifo buffer and mobile device comprising the same | |
US10334118B2 (en) | Method and system for providing video multimedia ringtone | |
US12003934B2 (en) | Controlling dual-mode bluetooth low energy multimedia devices | |
US7965660B2 (en) | Conference where mixing is time controlled by a rendering device | |
TW201841530A (en) | System and method of sending data via additional secondary data lines on a bus | |
KR20150028588A (en) | Electronic device and method for providing streaming service | |
WO2014059578A1 (en) | Embedded multimedia card partitioned storage space adjustment method and terminal | |
JP5146046B2 (en) | Information processing apparatus and information processing apparatus control method | |
US7286823B2 (en) | Mobile multimedia engine | |
CN107534616B (en) | Quality of service for universal serial bus | |
US20130343373A1 (en) | Voice-over-internet protocol (voip) application platform | |
WO2023005880A1 (en) | Audio control method, electronic device, audio device, and storage medium | |
GB2471481A (en) | Data path establishment for client initiated source to sink data transfer | |
US20110125902A1 (en) | Apparatus And A Method For Resource Management | |
KR100517723B1 (en) | TASK SCHEDULLING METHOD FOR MAINTAINING QoS OF MULTIMEDIA TASK IN WIRELESS TERMINAL | |
CN110019629B (en) | Map snapshot method and intelligent terminal | |
CN118555274A (en) | Audio transmission method, device, equipment and storage medium | |
EP2135164A2 (en) | Integrated circuit having data processing stages and electronic device including the integrated circuit | |
KR20070010432A (en) | Apparatus using two-way sharing memory and its method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WAP | Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1) |