GB2359438A - Method of changing bit rate in accordance with user selected storage mode - Google Patents

Method of changing bit rate in accordance with user selected storage mode Download PDF

Info

Publication number
GB2359438A
GB2359438A GB0004061A GB0004061A GB2359438A GB 2359438 A GB2359438 A GB 2359438A GB 0004061 A GB0004061 A GB 0004061A GB 0004061 A GB0004061 A GB 0004061A GB 2359438 A GB2359438 A GB 2359438A
Authority
GB
United Kingdom
Prior art keywords
bitstream
storage
transcoding
bit rate
received
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB0004061A
Other versions
GB0004061D0 (en
Inventor
Bruce Fairbairn Devlin
Michael James Knee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Snell Advanced Media Ltd
Original Assignee
Snell and Wilcox Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Snell and Wilcox Ltd filed Critical Snell and Wilcox Ltd
Priority to GB0004061A priority Critical patent/GB2359438A/en
Publication of GB0004061D0 publication Critical patent/GB0004061D0/en
Priority to PCT/GB2001/000724 priority patent/WO2001063936A1/en
Priority to AU37538/01A priority patent/AU3753801A/en
Publication of GB2359438A publication Critical patent/GB2359438A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • H04N19/107Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/147Data rate or code amount at the encoder output according to rate distortion criteria
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/152Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/162User input
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440254Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering signal-to-noise parameters, e.g. requantization
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/7921Processing of colour television signals in connection with recording for more than one processing mode
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/78Television signal recording using magnetic recording
    • H04N5/782Television signal recording using magnetic recording on tape
    • H04N5/783Adaptations for reproducing at a rate different from the recording rate

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Television Signal Processing For Recording (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

A method of processing a compressed digital video bitstream wherein the bitstream is received and is transcoded to a bitstream having a different bit rate reflecting the storage mode selected by a user. The storage modes available may include long play (having a lower bit rate than the received bitstream), compress-to-fit (having a bit rate reflecting available storage space) and rapid access (having an increased bit rate , with P frames being recoded as I frames to allow high speed searching). The transcoding may include the steps of decoding the received signal, extracting the coding decisions and recoding using the extracted decisions.

Description

COMPRESSED BITSTREAMS 2359438 This invention relates to compressed
bitstreams and in the most important example to the processing of compressed bitstreams prior to storage.
The television viewer has, through the video cassette recorder (VCR) become accustomed to time-shifting broadcast television programmes. that is to say recording a television programme off air for viewing at a more convenient time. The VCR has also created an alternative distribution channel for video content in the form of the sold or hired video tape. The VCR has been developed to offer a range of additional features. Typically, a video tape can be played at an increased speed with a recognisable picture still being displayed, to enable particular material to be located. Advanced VCRs will often provide the capability of freeze-framing or of stepping frame-by-frame through selected scenes. Another feature that is frequently available is that of 1ong play", where the user elects to compromise the recording quality in return for a longer recording interval on a given length of tape.
in the move to digital television, proposals are already being made to imitate the functionality of a VCR through other storage media such as hard disks or re-writable CDs or DVIDs. It has already been recognised that new storage media offer capabilities considerably beyond those of the VCR, such as the ability to record and play back simultaneously to give a "pause" effect in essentially real time viewing.
The compressed nature of broadcast digital video does, however, present challenges in providing functionality analogous to that of an advanced VCR. Generally, a bitstream delivered to the viewer (whatever the method of distribution) will be a relatively low bitrate and in the MPEG case will contain relatively few intra-coded frames.
It is an object of one aspect of the present invention to address this difficulty.
Accordingly, the present invention consists in one aspect in a method of processing a compressed digital video bitstream, comprising the steps of receiving a bitstream; establishing a plurality of user-defined storage modes; on user selection of a given one or more of said storage modes, transcoding the received bitstream to a transcoded bitstream having a different bitrate, reflecting the selected storage mode and storing the transcoded bitstream.
The invention will now be described by way of example with reference to the accompanying drawings in which:- Figure 1 is a block diagram illustrating an embodiment of the present invention; and Figure 2 is a block diagram of one form of the transcoder shown in Figure 1.
Referring initially to Figure 1, a compressed bitstream is received at terminal 100 and, instead of passing directly to the store 102 is received by a transcoder 104. In accordance with the invention, the user is provided the option of selecting storage modes through mode selection block 106. One such storage mode is 1ong play" and upon selection of this mode, the transcoder receives from the mode selection block a control signal. In accordance with this control signal, the transcoder operates to transcode the received bitstream to a transcoded bitstream having a selected lower bitrate. In this way, the programme material represented by the received bitstream can be accommodated by using a lesser portion of the store 102.
In certain known arrangements, a user interface is provided, enabling the user to instruct recording of a given television programme. In such a circumstance, the arrangement according to this example of the invention can operate to determine first the amount of available storage capability in the store 102. If this is less than that required to record the entire programme at the broadcast bitrate, the mode selection block 106 can recalculate a bitrate at which the entire programme can be recorded. This recalculated bitrate is provided to the transcoder which performs the transcoding operation as before.
In another storage mode, the user can select to store the received bitstream at the lowest bitrate consistent with a defined quality of video image.
Using, for example, the quality measure disclosed in PCT/G1399103356, the bitstrearn can be analysed and a quality measure determined. Using this measure in feedback, the transcoder will in this storage mode reduce the bitrate of the transcoded bitstream until the quality measure reaches a defined threshold.
In one form of this invention, the transcoder 104 operates in accordance with the disclosure of EP 0 765 576 and EP 0 913 058. As disclosed in these references, an incoming bitstream can be decoded to an uncompressed or less compressed bitstream, with the coding decisions being extracted. The uncompressed or less compressed signal can then be re-encoded at the required bitrate utilising the same coding decisions, so as to minimise or remove losses in re-encoding. The transcoding operation can also usefully take advantage of the disclosure of EP 0 963 657.
In another form of the present invention, the mode selector 106 permits selection of a mode in which the transcoder operates to increase bitrate. As has been explained, the received bitstream will typically take the form of a long GOP MPEG bitstream, that is to say a bitstream in which there are relatively few intra-coded or 1 frames and a relatively large number of predicted frames which may take the form of forward predicted frames (P frames) or both forward and backward predicted frames (B frames). If such a long GOP bitstream is written directly to store, operations such as a highspeed search for picture material, freeze frame and so on will be complex and time consuming.
Unless the available functionality is to be confined to the relatively infrequent 1 frames, some form of decoding operation will be required within each trick mode function.
In accordance with the present invention, this difficulty is overcome, where storage capacity permits, by modifying the received bitstream before it is written into the store. Thus the received bitstream is modified by reducing the amount of prediction coding. In one example, identified P frames in the received bitstream will be decoded and re-encoded as 1 frames. Thus as shown in Figure 2, the incoming bitstream is received by a controller 200. This is under the control of the mode selection block 106. Independence upon the user preference and the amount of available storage, it is elected to convert all or a given proportion of P frames into 1 frames. Thus the controller passes some or all P frames to a decoder 202 where, with information of course from the associated 1 frame, a decompressed picture signal is derived. This is reencoded in 1 frame encoder 204 to provide a sequence of 1 frames which are multilplexed in multiplexer 206 with the remaining, undisturbed frames, to form a transcoded bitstrearn output. Advantageously, appropriate coding decisions are forwarded from the decoder 102 to the 1 frame encoder 204.
I n another form of the invention, the number of intra-coded macroblocks within a picture is increased by a defined proportion. Typically, a given number of P type macroblocks will be converted to 1 type macroblocks. This feature has particular advantage in those trick modes which utilise a portion only of any one picture. Thus, by analogy with a VCR shuttle mode, it can be arranged that a recognisable picture is provided in a search mode by combining respective different picture regions from different incoming frames. By generally increasing the number of intra-coded macroblocks and by specifically ensuring that those picture regions which are to be used to produce a combined picture are coded as 1 type macroblocks, the present invention can considerably facilitate such an arrangement.
It should be understood that this invention has been described by way of example only and that a wide variety of further modifications are possible without departing from the scope of the invention.

Claims (9)

1. A method of processing a compressed digital video bitstream, comprising the steps of receiving a bitstream, establishing a plurality of userdefined storage modes; on user selection of a given one or more of said storage modes, transcoding the received bitstream to a transcoded bitstream having a different bitrate, reflecting the selected storage mode and storing the transcoded bitstream.
2. A method according to Claim 1, wherein the received bitstream is compliant with a given compression standard and the transcoded bitstrearn is compliant with the same standard.
3. A method according to Claim 1 or Claim 2, wherein the bitrate of the transcoded bitstream is less than that of the received bitstream.
4. A method according to any one of the preceding claims, wherein the transcoding step comprises the steps of decoding to an uncompressed or less compressed signal, extracting coding decisions and re-encoding the uncompressed or less compressed signal utilising some or all of said extracted coding decisions.
5. A method according to any one of the preceding claims, comprising the steps, on user selection of a given one of said storage modes, of determining an amount of available storage and transcoding the received bitstream to the bitrate calculated with reference to said storage.
6. A method according to any one of the preceding claims, wherein the step of transcoding, comprises reducing the degree of prediction within the received bitstream.
7. A method according to Claim 1, wherein the step of transcoding comprising the conversion of at least a portion of the prediction coded matter into intra-coded matter.
8. A method according to Claim 7. wherein said portion is a macroblock.
9. A method according to Claim 7, wherein said method is a frame.
GB0004061A 2000-02-21 2000-02-21 Method of changing bit rate in accordance with user selected storage mode Withdrawn GB2359438A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
GB0004061A GB2359438A (en) 2000-02-21 2000-02-21 Method of changing bit rate in accordance with user selected storage mode
PCT/GB2001/000724 WO2001063936A1 (en) 2000-02-21 2001-02-21 Compressed video bitstreams processing
AU37538/01A AU3753801A (en) 2000-02-21 2001-02-21 Compressed video bitstreams processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB0004061A GB2359438A (en) 2000-02-21 2000-02-21 Method of changing bit rate in accordance with user selected storage mode

Publications (2)

Publication Number Publication Date
GB0004061D0 GB0004061D0 (en) 2000-04-12
GB2359438A true GB2359438A (en) 2001-08-22

Family

ID=9886107

Family Applications (1)

Application Number Title Priority Date Filing Date
GB0004061A Withdrawn GB2359438A (en) 2000-02-21 2000-02-21 Method of changing bit rate in accordance with user selected storage mode

Country Status (3)

Country Link
AU (1) AU3753801A (en)
GB (1) GB2359438A (en)
WO (1) WO2001063936A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004040910A1 (en) * 2002-10-28 2004-05-13 Philippe Guenebaud Device for the trans-compression of a digital audiovisual stream in order to adapt same to existing television equipment
WO2008139120A2 (en) * 2007-04-13 2008-11-20 Streamwide Multimedia flow processing architecture
EP2192691A1 (en) * 2008-11-27 2010-06-02 Samsung Electronics Co., Ltd. Image recording apparatus and method of recording image

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8295619B2 (en) * 2010-04-05 2012-10-23 Mediatek Inc. Image processing apparatus employed in overdrive application for compressing image data of second frame according to first frame preceding second frame and related image processing method thereof

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08214058A (en) * 1995-02-07 1996-08-20 Victor Co Of Japan Ltd Recorder for telephone set
US5982723A (en) * 1996-09-30 1999-11-09 Laser Dynamics, Inc. Data recording and reproducing method for multi-layered optical disk system

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5887115A (en) * 1993-01-13 1999-03-23 Hitachi America, Ltd. Method and apparatus for implementing a video tape recorder for recording digital video signals having either a fixed or variable data transmission rate
KR950014858B1 (en) * 1993-06-04 1995-12-16 대우전자주식회사 Digital image recording apparatus
US5889561A (en) * 1994-11-04 1999-03-30 Rca Thomson Licensing Corporation Method and apparatus for scaling a compressed video bitstream
EP0893027B1 (en) * 1996-04-12 2004-08-25 Imedia Corporation Compressed-video distribution system having a statistical multiplexer with transcoders
US6674802B2 (en) * 1996-07-15 2004-01-06 Snell & Wilcox Limited Video process where part of compressed version of video signal accompanies video signal itself

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08214058A (en) * 1995-02-07 1996-08-20 Victor Co Of Japan Ltd Recorder for telephone set
US5982723A (en) * 1996-09-30 1999-11-09 Laser Dynamics, Inc. Data recording and reproducing method for multi-layered optical disk system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
'Seamless concatenation - a 21st century dream' <URL:http://www.bbc.co.uk/atlantic/montpap.htm> *
WPI Abstract Accession No. 1996-431064 & JP 08 214 058 A *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004040910A1 (en) * 2002-10-28 2004-05-13 Philippe Guenebaud Device for the trans-compression of a digital audiovisual stream in order to adapt same to existing television equipment
CN100385950C (en) * 2002-10-28 2008-04-30 菲利普·格纳博 Device for the trans-compression of a digital audiovisual stream in order to adapt same to existing television equipment
WO2008139120A2 (en) * 2007-04-13 2008-11-20 Streamwide Multimedia flow processing architecture
WO2008139120A3 (en) * 2007-04-13 2008-12-31 Streamwide Multimedia flow processing architecture
EP2192691A1 (en) * 2008-11-27 2010-06-02 Samsung Electronics Co., Ltd. Image recording apparatus and method of recording image

Also Published As

Publication number Publication date
GB0004061D0 (en) 2000-04-12
WO2001063936A1 (en) 2001-08-30
AU3753801A (en) 2001-09-03

Similar Documents

Publication Publication Date Title
US7046910B2 (en) Methods and apparatus for transcoding progressive I-slice refreshed MPEG data streams to enable trick play mode features on a television appliance
CA2432170C (en) Method and apparatus for storing a stream of video data on a storage medium
US6389218B2 (en) Method and apparatus for simultaneously producing compressed play and trick play bitstreams from a video frame sequence
JP3742431B2 (en) Method for generating a signal representing an MPEG compatible digital image, method for recording the signal, and method for generating and recording the signal
US5991502A (en) Optical recording device which calculates distances between I-frames and records I-frame addresses in a sector
US7340150B2 (en) Digital video signal record and playback device and method for selectively reproducing desired video information from an optical disk
US7359615B2 (en) Method and device for ensuring storage time for digital broadcast
JP3825719B2 (en) Image reproduction method, image reproduction apparatus, and image recording apparatus
KR20080044828A (en) Method and system for providing instant replay
KR20070049098A (en) Recording apparatus and method, reproducing apparatus and method, recording medium, and program
JPH08506229A (en) Digital video tape recorder for digital HDTV
KR20060088303A (en) Apparatus and method recording/playing moving picture in digital broadcasting receiver
US7924916B2 (en) Method and apparatus for decoding encoded groups of pictures of a video sequence and presenting said video sequence and said groups of pictures in temporally backward direction
US20030169817A1 (en) Method to encode moving picture data and apparatus therefor
JP2006509393A (en) Allocation and scheduling strategies that improve trick play performance and temporal scalability
US20080152313A1 (en) Image processing apparatus
KR100975170B1 (en) Image data reproducing device and method
GB2359438A (en) Method of changing bit rate in accordance with user selected storage mode
JP4462559B2 (en) Trick playback content creation method and device, trick playback compressed video data transmission method and device, and trick playback content creation program
JP3922490B2 (en) Video signal re-encoding device and method and video signal recording and re-encoding device and method
CA2206454A1 (en) Method and apparatus for coding digital video signals
JP4510254B2 (en) Recording apparatus and encoding apparatus
JP3800819B2 (en) Image synthesizer
JP2006187037A (en) Image reproducing method
US20060115234A1 (en) Moving picture compression-encoding apparatus and storage method

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)