KR101897945B1 - 서브 트랙 피처를 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램 - Google Patents

서브 트랙 피처를 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램 Download PDF

Info

Publication number
KR101897945B1
KR101897945B1 KR1020187016192A KR20187016192A KR101897945B1 KR 101897945 B1 KR101897945 B1 KR 101897945B1 KR 1020187016192 A KR1020187016192 A KR 1020187016192A KR 20187016192 A KR20187016192 A KR 20187016192A KR 101897945 B1 KR101897945 B1 KR 101897945B1
Authority
KR
South Korea
Prior art keywords
track
tile
tracks
box
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020187016192A
Other languages
English (en)
Korean (ko)
Other versions
KR20180067713A (ko
Inventor
프랑크 드누알
프레데릭 마즈
훼브르 쟝 르
씨릴 꽁꼴라또
Original Assignee
캐논 가부시끼가이샤
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 캐논 가부시끼가이샤 filed Critical 캐논 가부시끼가이샤
Publication of KR20180067713A publication Critical patent/KR20180067713A/ko
Application granted granted Critical
Publication of KR101897945B1 publication Critical patent/KR101897945B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/167Position within a video image, e.g. region of interest [ROI]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/438Interfacing the downstream path of the transmission network originating from a server, e.g. retrieving encoded video stream packets from an IP network
    • H04N21/4382Demodulation or channel decoding, e.g. QPSK demodulation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4728End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/65Transmission of management data between client and server
    • H04N21/658Transmission by the client directed to the server
    • H04N21/6587Control parameters, e.g. trick play commands, viewpoint selection
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845Structuring of content, e.g. decomposing content into time segments
    • H04N21/8451Structuring of content, e.g. decomposing content into time segments using Advanced Video Coding [AVC]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • H04N21/85406Content authoring involving a specific file format, e.g. MP4 format

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Information Transfer Between Computers (AREA)
KR1020187016192A 2013-07-23 2014-07-21 서브 트랙 피처를 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램 Active KR101897945B1 (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
GB1313157.8 2013-07-23
GB1313157.8A GB2516826B (en) 2013-07-23 2013-07-23 Method, device and computer program for encapsulating partitioned timed media data by creating tracks to be independently encapsulated in at least one media f
PCT/EP2014/065654 WO2015011108A1 (en) 2013-07-23 2014-07-21 Method, device, and computer program for encapsulating partitioned timed media data using sub-track feature

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
KR1020167003814A Division KR101867570B1 (ko) 2013-07-23 2014-07-21 서브 트랙 피처를 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램

Publications (2)

Publication Number Publication Date
KR20180067713A KR20180067713A (ko) 2018-06-20
KR101897945B1 true KR101897945B1 (ko) 2018-09-12

Family

ID=49119166

Family Applications (2)

Application Number Title Priority Date Filing Date
KR1020187016192A Active KR101897945B1 (ko) 2013-07-23 2014-07-21 서브 트랙 피처를 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램
KR1020167003814A Active KR101867570B1 (ko) 2013-07-23 2014-07-21 서브 트랙 피처를 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램

Family Applications After (1)

Application Number Title Priority Date Filing Date
KR1020167003814A Active KR101867570B1 (ko) 2013-07-23 2014-07-21 서브 트랙 피처를 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램

Country Status (7)

Country Link
US (2) US10212491B2 (enExample)
EP (1) EP3025508B1 (enExample)
JP (2) JP6395828B2 (enExample)
KR (2) KR101897945B1 (enExample)
CN (2) CN109982154B (enExample)
GB (1) GB2516826B (enExample)
WO (1) WO2015011108A1 (enExample)

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10349077B2 (en) * 2011-11-21 2019-07-09 Canon Kabushiki Kaisha Image coding apparatus, image coding method, image decoding apparatus, image decoding method, and storage medium
US10721530B2 (en) 2013-07-29 2020-07-21 Koninklijke Kpn N.V. Providing tile video streams to a client
GB2519746B (en) 2013-10-22 2016-12-14 Canon Kk Method, device and computer program for encapsulating scalable partitioned timed media data
KR20160105793A (ko) * 2014-01-07 2016-09-07 캐논 가부시끼가이샤 인터-계층 종속성을 인코딩하기 위한 방법, 디바이스, 및 컴퓨터 프로그램
WO2015197818A1 (en) 2014-06-27 2015-12-30 Koninklijke Kpn N.V. Hevc-tiled video streaming
GB2539462B (en) * 2015-06-16 2019-04-03 Canon Kk Obtaining media data and metadata from encapsulated bit-streams wherein operating point descriptors can be dynamically set
US20180213216A1 (en) * 2015-06-16 2018-07-26 Lg Electronics Inc. Media data transmission device, media data reception device, media data transmission method, and media data rececption method
WO2017029400A1 (en) * 2015-08-20 2017-02-23 Koninklijke Kpn N.V. Forming one or more tile streams on the basis of one or more video streams
US10674185B2 (en) 2015-10-08 2020-06-02 Koninklijke Kpn N.V. Enhancing a region of interest in video frames of a video stream
US20170111642A1 (en) * 2015-10-14 2017-04-20 Qualcomm Incorporated Support of random access and switching of layers and sub-layers in multi-layer video files
EP3293981A1 (en) * 2016-09-08 2018-03-14 Koninklijke KPN N.V. Partial video decoding method, device and system
US11197040B2 (en) * 2016-10-17 2021-12-07 Mediatek Inc. Deriving and signaling a region or viewport in streaming media
US10805620B2 (en) * 2017-01-11 2020-10-13 Mediatek Inc. Method and apparatus for deriving composite tracks
WO2018155939A1 (ko) * 2017-02-22 2018-08-30 에스케이텔레콤 주식회사 영상 복호화 방법 및 장치
KR102424829B1 (ko) * 2017-02-22 2022-07-25 에스케이텔레콤 주식회사 비디오 데이터가 부호화된 비트스트림을 처리하는 방법
US11360770B2 (en) 2017-03-20 2022-06-14 Intel Corporation Systems, methods, and apparatuses for zeroing a matrix
CN110431847B (zh) * 2017-03-24 2022-07-22 联发科技股份有限公司 视频处理方法及装置
GB2560921B (en) * 2017-03-27 2020-04-08 Canon Kk Method and apparatus for encoding media data comprising generated content
US10924822B2 (en) * 2017-04-04 2021-02-16 Qualcomm Incorporated Segment types as delimiters and addressable resource identifiers
US10778993B2 (en) * 2017-06-23 2020-09-15 Mediatek Inc. Methods and apparatus for deriving composite tracks with track grouping
GB2563865A (en) * 2017-06-27 2019-01-02 Canon Kk Method, device, and computer program for transmitting media content
US11275588B2 (en) 2017-07-01 2022-03-15 Intel Corporation Context save with variable save state size
GB2567625B (en) * 2017-10-12 2020-07-15 Canon Kk Method, device, and computer program for generating timed media data
US11743479B2 (en) 2017-12-06 2023-08-29 V-Nova International Limited Methods and apparatuses for encoding and decoding a bytestream
JP7430197B2 (ja) * 2019-03-11 2024-02-09 華為技術有限公司 エンコーダ、デコーダ及び対応する方法
US11581022B2 (en) * 2019-05-29 2023-02-14 Nokia Technologies Oy Method and apparatus for storage and signaling of compressed point clouds
US11523185B2 (en) 2019-06-19 2022-12-06 Koninklijke Kpn N.V. Rendering video stream in sub-area of visible display area
PH12022551544A1 (en) 2019-12-31 2023-04-24 Nokia Technologies Oy A method, an apparatus and a computer program product for video encoding and video decoding
KR102876734B1 (ko) * 2020-02-07 2025-10-27 삼성전자 주식회사 이미지를 저장하는 전자 장치 및 방법
KR20220164549A (ko) * 2020-04-02 2022-12-13 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. 효율적인 관심영역, 스트림 액세스 및 파라미터 세트 처리를 허용하는 파일 포맷 체계
GB2597642B (en) * 2020-06-16 2023-08-02 Canon Kk Method, device, and computer program for encapsulating media data into a media file
EP3972278A1 (en) 2020-09-17 2022-03-23 Lemon Inc. Subpicture tracks in coded video
US11930295B2 (en) 2020-09-17 2024-03-12 Lemon Inc. Handling of non-VCL NAL units in picture unit construction
EP3985974B1 (en) * 2020-10-13 2023-05-10 Axis AB An image processing device, a camera and a method for encoding a sequence of video images
JP7641766B2 (ja) * 2021-03-01 2025-03-07 日本放送協会 コンテンツ生成装置、受信装置、及びプログラム
CN114817647A (zh) * 2022-04-28 2022-07-29 清华大学 一种子图检索的方法、装置及电子设备
US20230362415A1 (en) * 2022-05-05 2023-11-09 Lemon Inc. Signaling of Preselection Information in Media Files Based on a Movie-level Track Group Information Box
GB2631722B (en) * 2023-07-10 2025-09-17 Canon Kk Method, device, and computer program for improving indexing, filtering, and repairing of portions of encapsulated media data

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012168365A1 (en) 2011-06-08 2012-12-13 Koninklijke Kpn N.V. Spatially-segmented content delivery

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6833840B2 (en) * 2000-02-14 2004-12-21 Optibase Ltd PROTO implementation in MPEG-4
AU2001286466A1 (en) * 2000-08-11 2002-02-25 Holomage, Inc. Method of and system for generating and viewing multi-dimensional images
CA2542026C (en) * 2002-04-29 2012-11-20 Nokia Corporation Indicating regions within a picture
EP1557837A1 (en) * 2004-01-26 2005-07-27 Sony International (Europe) GmbH Redundancy elimination in a content-adaptive video preview system
US7668914B2 (en) * 2005-03-28 2010-02-23 Alcatel Lucent Milestone synchronization in broadcast multimedia streams
US8019167B2 (en) * 2007-01-03 2011-09-13 Human Monitoring Ltd. Compressing high resolution images in a low resolution video
US20100232504A1 (en) * 2009-03-13 2010-09-16 The State of Oregon acting by and through the State Board of Higher Education on behalf of the Supporting region-of-interest cropping through constrained compression
JP5542912B2 (ja) * 2009-04-09 2014-07-09 テレフオンアクチーボラゲット エル エム エリクソン(パブル) メディア・コンテナ・ファイル管理
KR101802273B1 (ko) * 2010-03-05 2017-11-28 삼성전자주식회사 복수 개의 스트림으로 구성된 컨텐츠 파일 송수신 장치 및 방법
EP2532736A1 (en) * 2011-06-08 2012-12-12 National University of Ireland, Galway Engineered living tissue substitute
US9584819B2 (en) * 2011-10-24 2017-02-28 Qualcomm Incorporated Grouping of tiles for video coding
US10349077B2 (en) * 2011-11-21 2019-07-09 Canon Kabushiki Kaisha Image coding apparatus, image coding method, image decoding apparatus, image decoding method, and storage medium
US9354799B2 (en) * 2012-06-13 2016-05-31 Sonic Ip, Inc. Systems and methods for adaptive streaming systems with interactive video timelines
US9838686B2 (en) * 2013-07-12 2017-12-05 Intel Corporation Techniques for inclusion of thumbnail images in compressed video data

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012168365A1 (en) 2011-06-08 2012-12-13 Koninklijke Kpn N.V. Spatially-segmented content delivery

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JEAN LE FEUVRE et al., "Support for efficient tile access in the HEVC File Format", 104th MPEG meeting, ISO/IEC JTCI/SC29/WG11 MPEG2012/M29231, April 2013.
'Text of ISO/IEC FDIS 14496-15:2010 3rd edition Carriage of NAL unit structured video in the ISO BMFF', 104. MPEG Meeting; Incheon(22-4-2031); (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11)

Also Published As

Publication number Publication date
JP2018182772A (ja) 2018-11-15
KR20160034951A (ko) 2016-03-30
US20160165321A1 (en) 2016-06-09
CN105556982A (zh) 2016-05-04
CN109982154A (zh) 2019-07-05
JP2016527810A (ja) 2016-09-08
JP6643430B2 (ja) 2020-02-12
GB201313157D0 (en) 2013-09-04
KR101867570B1 (ko) 2018-06-15
US10212491B2 (en) 2019-02-19
WO2015011108A1 (en) 2015-01-29
GB2516826A (en) 2015-02-11
GB2516826B (en) 2016-06-22
JP6395828B2 (ja) 2018-09-26
US10547914B2 (en) 2020-01-28
KR20180067713A (ko) 2018-06-20
EP3025508B1 (en) 2021-09-08
US20190124419A1 (en) 2019-04-25
EP3025508A1 (en) 2016-06-01
CN105556982B (zh) 2019-04-02
CN109982154B (zh) 2021-06-08

Similar Documents

Publication Publication Date Title
KR101897945B1 (ko) 서브 트랙 피처를 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램
KR101800561B1 (ko) 코딩 종속성들에 대한 일반 시그널링을 이용하여 분할된 시간 설정형 미디어 데이터를 캡슐화하는 방법, 디바이스 및 컴퓨터 프로그램
KR101965374B1 (ko) 비디오 데이터의 재생 방법 및 비디오 데이터를 재생하기 위한 디바이스
GB2551296A (en) Method, device, and computer program for encapsulating partitioned timed media data
GB2546027A (en) Method, device, and computer program for encapsulating partioned timed media data

Legal Events

Date Code Title Description
A107 Divisional application of patent
PA0104 Divisional application for international application

Comment text: Divisional Application for International Patent

Patent event code: PA01041R01D

Patent event date: 20180607

Application number text: 1020167003814

Filing date: 20160215

A201 Request for examination
PA0201 Request for examination

Patent event code: PA02012R01D

Patent event date: 20180608

Comment text: Request for Examination of Application

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

Patent event code: PE07011S01D

Comment text: Decision to Grant Registration

Patent event date: 20180619

PG1501 Laying open of application
GRNT Written decision to grant
PR0701 Registration of establishment

Comment text: Registration of Establishment

Patent event date: 20180905

Patent event code: PR07011E01D

PR1002 Payment of registration fee

Payment date: 20180905

End annual number: 3

Start annual number: 1

PG1601 Publication of registration
PR1001 Payment of annual fee

Payment date: 20210820

Start annual number: 4

End annual number: 4

PR1001 Payment of annual fee

Payment date: 20220822

Start annual number: 5

End annual number: 5

PR1001 Payment of annual fee

Payment date: 20240826

Start annual number: 7

End annual number: 7