CN111292424A - Multi-view 360-degree VR content providing system - Google Patents

Multi-view 360-degree VR content providing system Download PDF

Info

Publication number
CN111292424A
CN111292424A CN201910017120.5A CN201910017120A CN111292424A CN 111292424 A CN111292424 A CN 111292424A CN 201910017120 A CN201910017120 A CN 201910017120A CN 111292424 A CN111292424 A CN 111292424A
Authority
CN
China
Prior art keywords
content
content providing
spherical image
dimensional modeling
modeling data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910017120.5A
Other languages
Chinese (zh)
Inventor
洪昇模
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Postmedia Co ltd
Original Assignee
Postmedia Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Postmedia Co ltd filed Critical Postmedia Co ltd
Publication of CN111292424A publication Critical patent/CN111292424A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/006Mixed reality
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/816Monomedia components thereof involving special video data, e.g 3D video
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/003Navigation within 3D models or images
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/21805Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Geometry (AREA)
  • Databases & Information Systems (AREA)
  • Processing Or Creating Images (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention discloses a system capable of providing multi-view 360-degree VR content in a virtual space. The multi-view 360-degree VR content providing system of the invention is a multi-view 360-degree VR content providing system including a content providing server, a content providing device and a content playing device, the content providing server stores a spherical image, position information of the spherical image, three-dimensional modeling data of an object located at a specific location, and position information of the three-dimensional modeling data by splicing one or more images captured at a specific capturing location, the content providing device requests the spherical image and the three-dimensional modeling data with the position information in the specific area and receives the spherical image and the three-dimensional modeling data from the content providing server to selectively provide the spherical image and the three-dimensional modeling data to the content playing device, the content playing device provides the position information in the virtual space input by the user to the content providing device, and receives the content corresponding to the position from the content providing device.

Description

Multi-view 360-degree VR content providing system
Technical Field
The present invention relates to a 360-degree VR content providing system, and more particularly, to a system capable of providing multi-viewpoint content that can provide a user with a high degree of freedom and thus can cope with a free viewpoint change.
Background
Due to the development of technology, in the case of contents (contents) including images or videos, all scenes such as 360 ° of panoramic contents are expressed beyond just expressing a certain scene, or a Virtual space is created by mapping (mapping) contents such as VR (Virtual Reality) contents in the panoramic contents to a spherical or cylindrical shape through a processing process, so that an experiencer can experience a space that does not exist actually.
However, the 360 ° virtual space played by VR contents photographed at a specific position has a disadvantage that only photographed images seen at the specific position can be provided, and image contents of an area beyond the photographing position cannot be provided.
[ Prior art documents ]
[ patent document ]
(patent document 0001) granted patent publication No. 10-1807439
Disclosure of Invention
Technical problem
The invention aims to provide a system capable of providing multi-viewpoint 360-degree VR content in a virtual space.
The present invention is not limited to the above-mentioned problems, and other problems not mentioned can be clearly understood by the skilled person from the following description.
Technical scheme
The multi-view 360-degree VR content providing system according to the present invention for solving the above-mentioned problems is a multi-view 360-degree VR content providing system including a content providing server, a content providing device, and a content playback device, the content providing server concatenates (Stitching) one or more images captured at a specific capturing location and stores a spherical image, position information of the spherical image, three-dimensional modeling data of an object located at the specific capturing location, and position information of the three-dimensional modeling data, the content providing device requests the spherical image and the three-dimensional modeling data with the position information in the specific area and receives the spherical image and the three-dimensional modeling data from the content providing server to selectively provide the spherical image and the three-dimensional modeling data to the content playing device, the content playing device provides the position information in the virtual space input by the user to the content providing device, and receives the content corresponding to the position from the content providing device.
According to an embodiment of the present invention, the content providing apparatus may receive, from the content providing server, a request for a spherical image and three-dimensional modeling data having position information within a radius set in advance centering on the position within the virtual space.
According to another embodiment of the present invention, when the position within the virtual space is inside a preset region, the content providing apparatus may receive from the content providing server a request for a spherical image and three-dimensional modeling data having position information within the preset region.
According to an embodiment of the present invention, when the position in the virtual space received from the content playback device is within a preset reference radius with reference to the position where the spherical image is captured although the position is beyond the position where the spherical image is captured, the content providing device may provide the content playback device with a spherical image of a new viewpoint generated by video-based rendering from among the spherical images.
Further, the content providing device may provide the three-dimensional modeling data to the content playback device when the position in the virtual space received from the content playback device is outside a preset reference radius with reference to the position where the spherical image is captured.
Other embodiments of the invention are included in the detailed description and the accompanying drawings.
Technical effects
According to one aspect of the present invention, in the virtual space, it is possible to freely select a position according to a user's selection in addition to a shooting location, and to provide a content from a viewpoint of observation at a position determined according to the user's selection.
According to another aspect of the present invention, a story-based (storytelling) VR content can be developed according to a user's free viewpoint selection.
The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned are clearly understood by the skilled person from the following description.
Drawings
Fig. 1 is a configuration diagram of a multi-view 360-degree VR content providing system according to the present invention.
Fig. 2 is a reference diagram for a spherical image according to the present invention.
Fig. 3 is a reference diagram based on image modeling.
FIG. 4 is a reference diagram for a range of data requests within a virtual space.
Fig. 5 is a reference diagram for the position where the spherical image is captured.
Fig. 6 is a reference diagram of three-dimensional modeling data.
Reference numerals
100: multi-view 360-degree VR content providing system
110: content providing server
120: content providing device
130: content playback apparatus
Detailed Description
The advantages and features of the invention described in this invention and the manner of attaining them will become apparent by reference to the following detailed description of embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be embodied in various forms different from each other, and the embodiments are provided to fully inform those skilled in the art (hereinafter, referred to as "practitioners") of the present invention of the scope of the present invention, so that the scope of the present invention is defined only by the claims.
The terms used in the present invention are used for describing embodiments, and are not used for limiting the scope of the present invention. In the present invention, the singular forms also include the plural forms as long as not specifically mentioned in the sentence. The use of "comprising" and/or "comprising" in the specification does not preclude the presence or addition of one or more other components in addition to the mentioned components. Throughout the specification, the same reference numerals refer to the same constituent elements, and "and/or" includes each of the constituent elements mentioned and all combinations of one or more. Although the terms "first", "second", and the like are used to describe various components, it is needless to say that these components are not limited by these terms. These terms are used only to distinguish one constituent element from another constituent element. Therefore, the first component mentioned below is within the technical idea of the present invention, and may be a second component.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, generally used dictionary-defined terms have to be interpreted normally or excessively as long as they are not specifically defined.
Embodiments of the present invention will be described below with reference to the accompanying drawings.
Fig. 1 is a configuration diagram of a multi-view 360-degree VR content providing system according to the present invention.
Referring to fig. 1, the multi-view 360-degree VR content providing system 100 of the present invention may include a content providing server 110, a content providing apparatus 120, and a content playing apparatus 130.
The content providing server 110 and the content providing apparatus 120 may be connected through a communication network, and may receive and transmit data through the communication network. The communication network is various communication networks capable of receiving and transmitting data, such as a wired communication network and a wireless communication network, and the method of transmitting data between the content providing server 110 and the content providing apparatus 120 is not limited.
The content providing server 110 may store a spherical image, position information of the spherical image, three-dimensional modeling data of an object located at a specific location, and position information of the three-dimensional modeling data by Stitching (Stitching) one or more images captured at a specific capturing location.
Fig. 2 is a reference diagram for a spherical image according to the present invention.
Referring to fig. 2, a process of generating a sphere (sphere) image by Stitching (Stitching) N images can be confirmed. The spherical image can be displayed on the content playing device 130 through the content providing device 120, and the user can view the image in the corresponding view direction according to the user's sight line direction.
The spherical images are matched with position information of the shot place. Thus, the spherical image may correspond to a particular location within the virtual space.
The three-dimensional modeling data of the object located at the specific location means three-dimensional data that renders (renders) and represents a real object such as an actual building or structure. The three-dimensional Modeling data may be obtained by three-dimensionally scanning an actual object, or may be obtained by Image based Modeling (Image based Modeling).
Fig. 3 is a reference diagram based on image modeling.
If referring to fig. 3, an example of modeling based on an image of a car or a building can be confirmed. The example at the upper end of fig. 3 is an example modeled based on images photographed in a plurality of directions of a subject, and the example at the lower end of fig. 3 is an example modeled based on an image photographed in a certain direction of the subject. In the three-dimensional modeling data according to the upper end example of fig. 3, the user can freely observe at any position, but in the three-dimensional modeling data according to the lower end example of fig. 3, the user can observe only in a limited range centered on the shooting position.
The three-dimensional modeling data is also matched with position information of the object. Thus, the three-dimensional modeling data may correspond to a particular location within the virtual space.
The content providing device 120 may request and receive the spherical image and the three-dimensional modeling data having the position information within the specific area from the content providing server 110 to selectively provide to the content playing device 130.
The content playing device 130 may transmit information about a location in the virtual space according to the user's input to the content providing device 120, and receive content corresponding to the location from the content playing device 130.
The content providing device 120 and the content playing device 130 are devices connected by wire or wireless, and the content provided by the content providing device 120 can be output through a display device included in the content playing device 130.
The content playing device 130 may be embodied in a Head Mounted Display (HMD) form. The HMD type is a display system that is worn on the head and displays an image directly in front of the eyes of a user. The content playing device 130 may output visual content. The visual content means content provided by the content providing server 110 through the content providing device 120. Applications, images, videos, icons, user interfaces, and the like may also be included in the visual content.
In addition, the content playing device 130 may output virtual reality content. When the content playing device 130 is embodied in an HMD, virtual reality content may be output in a light-shielded (see-closed) manner. The light-shielding type corresponds to a type of using contents in a state where a user cannot recognize an external environment. In the present invention, a case where the content playing device 130 is embodied in an HMD is assumed, and content is output in a light-shielding type.
On the other hand, the content playing device 130 may sense various inputs of the user and the surrounding environment, and transmit the sensing result to the content providing device. Various inputs of the user may be received through a camera, a microphone, a touch sensor, an acceleration sensor, a gyro sensor, a GPS (global positioning system) sensor, a geomagnetic sensor, and the like. In the present invention, in the case where the content playing apparatus 130 is embodied as an HMD, the moving angle, direction, speed, etc. of the user wearing the HMD may be sensed.
The content providing server 110 may store spherical images and three-dimensional modeling data for various locations, various positions, various buildings, and the like. It is very inefficient to transmit all the spherical images and the three-dimensional modeling data stored in the content providing server 110 to the content providing device 120. Therefore, it is more efficient for the content playing device 130 to provide the relevant spherical image and three-dimensional modeling data to the content providing device 120 corresponding to the position in the virtual space.
FIG. 4 is a reference diagram for a range of data requests within a virtual space.
Referring to fig. 4, the area of the virtual space displayed on the content playback device 130 may be referred to. For convenience of understanding, a cultural heritage of a building is taken as an example. The user using the content playing device 130 is located in a virtual space embodying the cultural relic. In fig. 4, the large black dots are the positions in the virtual space where the user is now located. Therefore, a spherical image and three-dimensional modeling data for a region seen in the virtual space are required.
According to an embodiment of the present invention, the content providing apparatus 120 may request and receive a spherical image and three-dimensional modeling data having position information within a radius set in advance centering on the position within the virtual space from the content providing server 110. Fig. 4 (a) is a reference diagram of the embodiment.
According to another embodiment of the present invention, when the position in the virtual space is within a preset area, the content providing apparatus 120 may request the spherical image and the three-dimensional modeling data having the position information in the preset area to receive the spherical image and the three-dimensional modeling data from the content providing server. Fig. 4 (b) is a reference diagram of the embodiment.
On the other hand, the multi-view 360 degree VR content providing system 100 of the present invention gives a high degree of freedom to a user, providing free movement in a virtual space and thus multi-view content. At this time, the user may be beyond the position where the spherical image is captured within the virtual space. At this time, since the viewpoint viewed by the user changes, the viewpoint of the provided content also needs to be changed.
Fig. 5 is a reference diagram for the position of the spherical image capturing.
Referring to fig. 5, the place indicated by the black circle in the virtual space is the position where the spherical image is captured. Therefore, when the user goes beyond the location indicated by the blue circle in the virtual space, it is necessary to provide contents corresponding to the viewpoint change.
According to an embodiment of the present invention, when the position in the virtual space received from the content playback device 130 exceeds the position where the spherical image is captured, but is within a preset reference radius with respect to the position where the spherical image is captured, the content providing device 120 may provide the spherical image of the new viewpoint generated by the video-based rendering in the spherical image to the content playback device 130.
The video-Based Rendering is Image-Based Rendering (Image Based Rendering), and means a method of generating a video from a combination of two-dimensional videos with a new viewpoint. The image-based rendering is a technique well known to practitioners, and thus a detailed description is omitted. The image-based rendering may generate a limited range of view point images in a basic image, and thus may be provided only when the content playing apparatus 130 is located within a preset reference radius with reference to a location where the spherical image is photographed. The preset reference radius may refer to the part of the area indicated by the thick dark grey line shown as a black circle in fig. 5.
If the user is outside the preset reference radius in the virtual space, the user needs to change to another content.
According to an embodiment of the present invention, the content providing apparatus 120 may provide the three-dimensional modeling data to the content playback apparatus 130 when the position in the virtual space received from the content playback apparatus 130 is outside a preset reference radius with reference to the place where the spherical image is captured.
Fig. 6 is a reference diagram of three-dimensional modeling data.
Referring to fig. 6 (a), it can be confirmed that the position of the user in the virtual space is indicated by a large black dot. The position of the user shown in fig. 6 (a) is not included in the position where the spherical image shown in fig. 5 is captured or the position of the preset reference radius. Accordingly, three-dimensional modeling data according to the above-described embodiments can be provided. Fig. 6 (b) is an example of three-dimensional modeling data displayed in the content playback device 130 from the viewpoint of the user at the user position shown in fig. 6 (a).
While the embodiments of the present invention have been described with reference to the drawings, it will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The embodiments described above are therefore to be considered in all respects as illustrative and not restrictive.

Claims (5)

1. A multi-view 360-degree VR content providing system includes a content providing server, a content providing device, and a content playing device,
the content providing server stores a spherical image, position information of the spherical image, three-dimensional modeling data of an object located at a specific location, and position information of the three-dimensional modeling data by splicing one or more images captured at a specific capturing location,
the content providing device requests the spherical image and the three-dimensional modeling data with the position information in the specific area and receives the spherical image and the three-dimensional modeling data from the content providing server to selectively provide the spherical image and the three-dimensional modeling data to the content playing device,
the content playing device provides the position information in the virtual space input by the user to the content providing device, and receives the content corresponding to the position from the content providing device.
2. The multi-view 360 degree VR content provision system of claim 1 wherein,
the content providing device receives a request for a spherical image and three-dimensional modeling data having position information within a radius set in advance with the position within the virtual space as a center from the content providing server.
3. The multi-view 360 degree VR content provision system of claim 1 wherein,
when the position in the virtual space is within a preset region, the content providing device requests the spherical image and the three-dimensional modeling data having the position information in the preset region to receive the spherical image and the three-dimensional modeling data from the content providing server.
4. The multi-view 360 degree VR content provision system of claim 1 wherein,
the content providing device provides the content playing device with a spherical image of a new viewpoint generated by video-based rendering from the spherical images, when the position in the virtual space received from the content playing device is within a preset reference radius with reference to the position where the spherical image is captured, although the position is beyond the position where the spherical image is captured.
5. The multi-view 360 degree VR content provision system of claim 4 wherein,
the content providing device provides the three-dimensional modeling data to the content playback device when the position in the virtual space received from the content playback device is outside a preset reference radius with reference to the position where the spherical image is captured.
CN201910017120.5A 2018-12-06 2019-01-08 Multi-view 360-degree VR content providing system Pending CN111292424A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2018-0156056 2018-12-06
KR1020180156056A KR102200115B1 (en) 2018-12-06 2018-12-06 System for providing multi-view 360 angle vr contents

Publications (1)

Publication Number Publication Date
CN111292424A true CN111292424A (en) 2020-06-16

Family

ID=71021525

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910017120.5A Pending CN111292424A (en) 2018-12-06 2019-01-08 Multi-view 360-degree VR content providing system

Country Status (2)

Country Link
KR (1) KR102200115B1 (en)
CN (1) CN111292424A (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102574677B1 (en) * 2021-12-14 2023-09-06 엘아이지넥스원 주식회사 Method continuously acquiring and reproducing 360-degree virtual reality images according to viewpoint movement
KR102564522B1 (en) 2023-02-02 2023-08-07 엠앤앤에이치 주식회사 Multi-view shooting apparatus and method for creating 3D volume object

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105989573A (en) * 2015-02-16 2016-10-05 上海钧铭商务咨询有限公司 Method and system for providing exhibition hall guide information based on 360-degree digital panoramic technology
JP2017146762A (en) * 2016-02-17 2017-08-24 株式会社Acw−Deep Image display type simulation service providing system and image display type simulation service providing method
CN107169924A (en) * 2017-06-14 2017-09-15 歌尔科技有限公司 The method for building up and system of three-dimensional panoramic image
US20180181196A1 (en) * 2016-12-22 2018-06-28 Samsung Electronics Co., Ltd. Method for displaying image, storage medium, and electronic device
US20180184000A1 (en) * 2016-12-23 2018-06-28 Samsung Electronics Co., Ltd. Method and device for managing thumbnail of three-dimensional contents
WO2018135730A1 (en) * 2017-01-23 2018-07-26 고범준 Vr image generation method, vr image processing method, and vr image processing system

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5196472B2 (en) * 2007-10-23 2013-05-15 株式会社ロケーションビュー Image display processing program and image display processing system
KR20120137607A (en) * 2011-05-24 2012-12-24 이엔지정보기술 주식회사 Web service system and method using spherical image, web map, gis database
EP3044951A4 (en) 2013-09-11 2017-09-06 Varex Imaging Corporation Pixel circuit with constant voltage biased photodiode and related imaging method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105989573A (en) * 2015-02-16 2016-10-05 上海钧铭商务咨询有限公司 Method and system for providing exhibition hall guide information based on 360-degree digital panoramic technology
JP2017146762A (en) * 2016-02-17 2017-08-24 株式会社Acw−Deep Image display type simulation service providing system and image display type simulation service providing method
US20180181196A1 (en) * 2016-12-22 2018-06-28 Samsung Electronics Co., Ltd. Method for displaying image, storage medium, and electronic device
US20180184000A1 (en) * 2016-12-23 2018-06-28 Samsung Electronics Co., Ltd. Method and device for managing thumbnail of three-dimensional contents
WO2018135730A1 (en) * 2017-01-23 2018-07-26 고범준 Vr image generation method, vr image processing method, and vr image processing system
CN107169924A (en) * 2017-06-14 2017-09-15 歌尔科技有限公司 The method for building up and system of three-dimensional panoramic image

Also Published As

Publication number Publication date
KR102200115B1 (en) 2021-01-08
KR20200069004A (en) 2020-06-16

Similar Documents

Publication Publication Date Title
US9858643B2 (en) Image generating device, image generating method, and program
US10999412B2 (en) Sharing mediated reality content
US7817104B2 (en) Augmented reality apparatus and method
US8253649B2 (en) Spatially correlated rendering of three-dimensional content on display components having arbitrary positions
EP3629309A2 (en) Drone real-time interactive communications system
EP2305358A1 (en) Portable type game device and method for controlling portable type game device
JP2012084146A (en) User device and method providing augmented reality (ar)
US20210037214A1 (en) Output control apparatus, display terminal, remote control system, control method, and non-transitory computer-readable medium
CN110825333B (en) Display method, display device, terminal equipment and storage medium
EP3229482B1 (en) Master device, slave device, and control method therefor
KR102197615B1 (en) Method of providing augmented reality service and server for the providing augmented reality service
CN106133582A (en) The method and system of real scene image stream are combined in wearing display device
JP6409861B2 (en) Information processing apparatus, information processing system, control method thereof, and program
CN111292424A (en) Multi-view 360-degree VR content providing system
JP6649010B2 (en) Information processing device
JP2018056889A (en) Display terminal, display method, and program
TWI764366B (en) Interactive method and system based on optical communication device
WO2018139073A1 (en) Display control device, second display device, method for controlling display control device, and control program
EP3542877A1 (en) Optimized content sharing interaction using a mixed reality environment
EP3805899A1 (en) Head mounted display system and scene scanning method thereof
JP2018094086A (en) Information processing device and image formation method
JP6376251B2 (en) Information processing apparatus, information processing system, control method thereof, and program
KR20190112407A (en) Operating method for Holoportation contents
WO2022172335A1 (en) Virtual guide display device, virtual guide display system, and virtual guide display method
US11928775B2 (en) Apparatus, system, method, and non-transitory medium which map two images onto a three-dimensional object to generate a virtual image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200616