CN110830550A - Computer cluster and diskless starting method thereof - Google Patents
Computer cluster and diskless starting method thereof Download PDFInfo
- Publication number
- CN110830550A CN110830550A CN201910943120.8A CN201910943120A CN110830550A CN 110830550 A CN110830550 A CN 110830550A CN 201910943120 A CN201910943120 A CN 201910943120A CN 110830550 A CN110830550 A CN 110830550A
- Authority
- CN
- China
- Prior art keywords
- servers
- file system
- server
- root file
- client
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1017—Server selection for load balancing based on a round robin mechanism
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/45—Network directories; Name-to-address mapping
- H04L61/4505—Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols
- H04L61/4511—Network directories; Name-to-address mapping using standardised directories; using standardised directory access protocols using domain name system [DNS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L61/00—Network arrangements, protocols or services for addressing or naming
- H04L61/50—Address allocation
- H04L61/5007—Internet protocol [IP] addresses
- H04L61/5014—Internet protocol [IP] addresses using dynamic host configuration protocol [DHCP] or bootstrap protocol [BOOTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Abstract
The invention discloses a computer cluster and a diskless starting method thereof, wherein the method comprises the following steps: s1, compressing the mirror image of the root file system; s2, the client downloads the compressed root file system mirror image from one of the servers in a load balancing mode; s3, the client decompresses the root file system image to complete the boot. The technical scheme of the invention accelerates the starting process and optimizes the downloading process of the root file system. The diskless startup time of a high performance computing cluster can be greatly reduced.
Description
Technical Field
The invention relates to the technical field of servers, in particular to a computer cluster and a diskless starting method thereof.
Background
Modern servers typically support the PXE boot approach. PXE is a pre-boot execution environment promoted by Intel, and can support downloading system images from a remote server without passing through a system on a local hard disk. By means of PXE technology, high performance computing clusters can achieve diskless boot. Compared with local hard disk startup, diskless startup has the following advantages: a local hard disk is not needed, so that the cost in the aspect of hard disks is reduced; the risk of damage to the local hard disk is avoided; and the consistency of the system is maintained, all the nodes are started through a uniform system mirror image, and the system is kept consistent.
Since the system image needs to be downloaded from the remote server during diskless startup, the network bandwidth of the remote server and the network bandwidth of the system are likely to become bottlenecks. When the whole cluster is started, all nodes need to access the server where the system mirror image is located, all nodes share the outlet bandwidth of the mirror image server, and the downloading speed of the mirror image is greatly reduced. The mirror server is generally configured with gigabit ethernet ports, and the network rate is 1 Gb/s. Diskless booted system images are typically around 4GB in size. When a single node is started, the starting time is about 32 s. When 50 nodes start up simultaneously, the start-up time is increased by 50 times to 1600 s. Moreover, all system mirror image data need to be sent through the ethernet port of the mirror image server, and the data volume sent by the mirror image server reaches 200 GB. Such large-scale data transmission is liable to cause Ethernet port abnormality and data transmission error.
Disclosure of Invention
Aiming at the problems in the related art, the invention provides a computer cluster and a diskless starting method thereof, which can optimize the downloading process of a root file system.
The technical scheme of the invention is realized as follows:
according to an aspect of the present invention, there is provided a diskless boot method of a computer cluster, comprising:
s1, compressing the mirror image of the root file system;
s2, the client downloads the compressed root file system mirror image from one of the servers in a load balancing mode;
s3, the client decompresses the root file system image to complete the boot.
According to an embodiment of the present application, S2 includes: setting a plurality of servers as the same domain name; the domain name is resolved to multiple servers in a round robin fashion.
According to an embodiment of the application, resolving domain names to a plurality of servers in a polling manner includes: when applying for resolving the domain name, the domain name is resolved into the IP addresses of a plurality of servers in sequence.
According to the embodiment of the application, the plurality of servers comprise a load scheduler and a server pool, and the load scheduler is used for sending the request of the client to the server in the server pool to execute.
According to an embodiment of the present application, S2 includes: configuring virtual IPs for the load scheduler and the servers in the server pool, wherein the virtual IPs of the load scheduler are visible to the outside, and the virtual IPs of the servers in the server pool are invisible to the outside; the client requests the system image through the virtual IP, and the load scheduler starts service on the server in the server pool to provide the downloaded system image.
According to an embodiment of the application, compressing a root file system image includes: the root file system image is compressed into the squashfs format.
According to another aspect of the present invention, a computer cluster is provided, which includes a client and a plurality of servers, wherein the client downloads a compressed root file system image from one of the servers in a load balancing manner; and the client decompresses the root file system image to complete the boot.
According to the embodiment of the application, a plurality of servers are set to be the same domain name; and, each time a domain name resolution is applied, the domain name is sequentially resolved into the IP addresses of a plurality of servers.
According to the embodiment of the application, the plurality of servers comprise a load scheduler and a server pool, wherein the load scheduler is used for sending the request of the client to the server in the server pool for execution; the load scheduler and the servers in the server pool are provided with virtual IPs, wherein the virtual IPs of the load scheduler are visible to the outside, and the virtual IPs of the servers in the server pool are invisible to the outside; the client requests the system image through the virtual IP, and the load scheduler starts service on the server in the server pool to provide the downloaded system image.
According to an embodiment of the application, compressing a root file system image includes: the root file system image is compressed into the squashfs format.
By the technical scheme, the starting process is accelerated, and the downloading process of the root file system is optimized. The diskless startup time of a high performance computing cluster can be greatly reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a flow diagram of a high performance computer cluster diskless boot according to an embodiment of the present invention;
fig. 2 is a flowchart of a diskless boot method of a computer cluster according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
Fig. 1 is a flow chart of a high-performance computer cluster startup without a disk. The starting process mainly comprises the following steps:
s11, the BIOS loads a PXE client (client) program into the memory, and the PXE client acquires the dynamic IP, the host name, the TFTP server address and the DNS server address through the DHCP server;
s12, the PXE client downloads a boot loader (typically pxelinux.0) to the memory and executes it through the TFTP server address acquired at step S11. Pxelinux.0 is a boot bootstrap, similar to the grub bootstrap. Pxelinux.0 will download a configuration file (usually pxelinux.cfg/default) and select the kernel to load according to the configuration file.
S13, the boot program is started to download the kernel of the operating system and initramfs into the memory through TFTP, and the control right is given to the kernel.
S14, the kernel completes its decompression and mounting initramfs, and performs basic hardware initialization, and then downloads the root file system image through network (usually through http).
S15, the kernel mounts the root file system, executes init (systematic) initialization program, starts each service, and finishes system startup.
In the steps, the main files downloaded by the client node comprise a pxelinux.0 boot loader, a linux kernel, initramfs and a root file system image. With boot sizes typically in the hundreds of KB, linux kernel and initramfs totaling tens of MB, and root file system images typically around a few GB. Root file system download is the longest time consuming part of the whole boot process. The diskless starting method provided by the invention mainly accelerates the starting process by the following two aspects, and optimizes the downloading process of the root file system:
1. and the mirror image of the root file system is compressed, and the volume of the system mirror image is reduced.
2. Adding a starting server to accelerate the starting speed in a load balancing mode
Fig. 2 is a flowchart of a diskless boot method of a computer cluster according to an embodiment of the present invention. The diskless boot method according to an embodiment of the present invention may include the steps of:
s21, compressing the mirror image of the root file system;
s22, the client downloads the compressed root file system mirror image from one of the servers in a load balancing mode;
s23, the client decompresses the root file system image to complete the boot.
In one embodiment, at step S21, compressing the root file system image may include: the root file system image is compressed into the squashfs format. Specifically, the root file system image is generally saved as a file, and is formatted into an ext4 format through mkfs. And then various system software is installed in the image file. Generally, after various basic software packages are installed, the volume of the image file is about several GB, and the image file is too large and is inconvenient to transmit. By compressing the mirror image of the root file system into the squashfs format, the size of the mirror image file is greatly reduced. Creation of an image file in the squashfs format may be accomplished using the following commands.
At step S22, a mirror server is added to accelerate the boot speed by load balancing. Due to the addition of the mirror image server, different nodes can be downloaded from different mirror image servers, and therefore the starting speed is increased. The client node and the image server may be configured in a static relationship, i.e. the client node downloads images from a fixed image server, the image servers corresponding to different nodes being different. And also can be configured in a load balancing mode.
In one embodiment, a DNS polling load balancing startup approach may be employed. In this embodiment, step S22 may include: setting a plurality of servers as the same domain name; the domain name is resolved to multiple servers in a round robin fashion. Wherein resolving the domain name to the plurality of servers in a polling manner comprises: when applying for resolving the domain name, the domain name is resolved into the IP addresses of a plurality of servers in sequence.
In another embodiment, an LVS load balancing activation mode may be employed. In this embodiment, the plurality of servers includes a load scheduler and a server pool, and the load scheduler is configured to send the request of the client to the servers in the server pool for execution. Wherein, the step S22 may include: configuring virtual IPs for the load scheduler and the servers in the server pool, wherein the virtual IPs of the load scheduler are visible to the outside, and the virtual IPs of the servers in the server pool are invisible to the outside; the client requests the system image through the virtual IP, and the load scheduler starts service on the server in the server pool to provide the downloaded system image.
That is, DNS polling and LVS 2 load balancing activation methods may be employed. These two load balancing activation methods will be described in detail below.
First, DNS polling mode
In the DNS polling mode, a plurality of mirror image servers are set as the same domain name, and the domain name is respectively resolved to each host in a polling mode, for example, mirror image domain names boot are respectively resolved to 6 hosts with IP addresses of 11.2.25.7-11.2.25.12 and the like. Then, in a configuration file pxelinux.cfg/default of pxelinux.0, the http mirror server is specified in a domain name rather than a fixed IP manner. The specific configuration steps are as follows.
(1) Installing DNS service related rpm (bind)
(2) Modify DNS configuration file/etc/name.conf. Wherein the bold part in the following is mainly modified:
it should be noted that: the/etc/named. conf owner is named, otherwise the service will fail to start because the file cannot be read when starting.
(3) Configuring DNS database files
The DNS database file is specified as named.silicon.sugon, stored under the/var/named directory (/ etc/named/named.silicon.sugon), which can be referred to as follows:
the host names of the starting servers correspond to the IP, the host name of the boot is analyzed into 6 servers (boot 3-boot 8) of 11.2.25.7-11.2.25.12, and the 6 servers are analyzed in a polling mode, namely, the boot host names are sequentially analyzed into 11.2.25.7-11.2.25.12 each time a DNS is applied for analysis.
(4) Starting a DNS service:
check if boot hostname resolution is correct using dig command:
when the dig command is used for analyzing the boot host name each time, the returned IP addresses are different, and whether the DNS polling mode is effective or not can be checked. So far, DNS service configuration has been completed, and modifications in DHCP service and pxelinux.0 configuration are described below.
(5) Modify DHCP service, add DNS configuration (/ etc/DHCP/dhcpd. conf), add bold part of the following:
(6) pxelinux.0 configuration files (pxelinux.cfg/default) are modified to modify the http server IP address to the host name (bold part in the following).
Second, LVS load balancing mode
The LVS is a short for Linux virtual host (Linux virtual server), and is a virtual server cluster system. Which comprises the following steps:
a load balancer/Director, which is a front-end outside the whole cluster, and is responsible for sending the client's request to a group of servers for execution, and the client considers the service to come from an IP address (we can refer to as a virtual IP address); and
the server pool (server pool/Realserver) is a group of servers that really execute the client request, and the executed services generally include WEB, MAIL, FTP, DNS, etc.
The LVS can specifically support NAT, Tunneling and dr (direct routing)3 load balancing modes. In the DR mode, the Realserver can directly send the system image to the client node without passing through an intermediate Director server, so that the Director server is prevented from becoming a bottleneck.
By adopting a DR mode of LVS, 1 server is required to be selected as a Director to serve as a load balancing server, other servers serve as RealServer, http service is started on the RealServer, http requests are actually responded, and downloading images are provided. Both the Director and the RealServer need to configure a Virtual Ip (VIP), and the client requests the system image through the VIP. The Director server VIP is visible to the outside and needs to be bound to the actual network card (e.g., eth0), while the RealServer VIP is invisible to the outside and is typically bound to the virtual network card (e.g., lo).
To use the lvs service, the ipv sadm rpm package needs to be installed and the ip _ vs kernel module loaded. The configuration on the Director and RealServer servers is described below, respectively.
(1) Director server
The following script lvs-director automatically sets the director server.
Starting a service on a director server:
and viewing the service state:
it should be noted that: the use of the above script is affected by the use of the system start ipv sadm, which suggests shutting down the ipv sadm service in use.
(2) RealServer server
By the diskless starting method provided by the invention, the diskless starting time of the high-performance cluster can be greatly reduced. Compressing the image file into the squashfs format reduces the image file size to 1/3 of the original file. Configuring N mirror servers and configuring load balancing can reduce the starting time of the whole cluster to 1/N of 1. If a set of clusters has 50 diskless startup nodes, the size of the image file before compression is 4GB, and the startup of the whole set of clusters needs 1600s, which is about half an hour. By using the diskless starting method, when 5 mirror image servers are configured, the starting time is reduced to 1/15 which is about 106 s.
According to an embodiment of the present invention, there is also provided a computer cluster, including a client and a plurality of servers, wherein the client downloads a compressed root file system image from one of the plurality of servers in a load balancing manner; and the client decompresses the root file system image to complete the boot.
In one embodiment, multiple servers are set to the same domain name; and, each time a domain name resolution is applied, the domain name is sequentially resolved into the IP addresses of a plurality of servers.
In one embodiment, the plurality of servers includes a load scheduler for sending requests of clients to servers in the server pool for execution, and a server pool. The load scheduler and the servers in the server pool have virtual IPs, wherein the virtual IPs of the load scheduler are visible to the outside and the virtual IPs of the servers in the server pool are not visible to the outside. The client requests the system image through the virtual IP, and the load scheduler starts service on the server in the server pool to provide the downloaded system image.
In one embodiment, compressing the root file system image comprises: the root file system image is compressed into the squashfs format.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (10)
1. A diskless boot method of a computer cluster, comprising:
s1, compressing the mirror image of the root file system;
s2, the client downloads the compressed root file system mirror image from one of the servers in a load balancing mode;
s3, the client decompresses the root file system image to complete the boot.
2. The diskless boot method of claim 1, wherein S2 comprises:
setting the plurality of servers to be the same domain name;
resolving the domain name to the plurality of servers in a round robin manner.
3. The diskless boot method of claim 2, wherein resolving the domain name to the plurality of servers in a round robin manner comprises:
and when the domain name is applied and analyzed each time, the domain name is sequentially analyzed into the IP addresses of the plurality of servers.
4. The diskless boot method of claim 1, wherein said plurality of servers comprises a load scheduler and a server pool, said load scheduler being configured to send said client's request to a server in said server pool for execution.
5. The diskless boot method of claim 4, wherein S2 comprises:
configuring virtual IPs for the load scheduler and the servers in the server pool, wherein the virtual IPs of the load scheduler are visible to the outside, and the virtual IPs of the servers in the server pool are invisible to the outside;
the client requests a system image through the virtual IP, and the load scheduler starts service on the server in the server pool to provide a download system image.
6. The diskless boot method of any of claims 1-5, wherein compressing the root file system image comprises:
and compressing the root file system mirror image into a squashfs format.
7. A computer cluster is characterized by comprising a client and a plurality of servers, wherein the client downloads a compressed root file system image from one of the servers in a load balancing mode; and the client decompresses the root file system image to complete the startup.
8. The computer cluster of claim 7, wherein the plurality of servers are set to the same domain name; and when the domain name is requested to be resolved each time, the domain name is resolved into the IP addresses of the plurality of servers in sequence.
9. The computer cluster of claim 7, wherein the plurality of servers comprises a load scheduler and a server pool, the load scheduler configured to send the client's request to a server in the server pool for execution;
the load scheduler and the servers in the server pool are provided with virtual IPs, wherein the virtual IPs of the load scheduler are visible to the outside, and the virtual IPs of the servers in the server pool are invisible to the outside;
the client requests a system image through the virtual IP, and the load scheduler starts service on the server in the server pool to provide a download system image.
10. The computer cluster of any of claims 7-9, wherein compressing the root file system image comprises:
and compressing the root file system mirror image into a squashfs format.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910943120.8A CN110830550A (en) | 2019-09-30 | 2019-09-30 | Computer cluster and diskless starting method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910943120.8A CN110830550A (en) | 2019-09-30 | 2019-09-30 | Computer cluster and diskless starting method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110830550A true CN110830550A (en) | 2020-02-21 |
Family
ID=69548787
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910943120.8A Pending CN110830550A (en) | 2019-09-30 | 2019-09-30 | Computer cluster and diskless starting method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110830550A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016159A (en) * | 2022-12-16 | 2023-04-25 | 北京云端智度科技有限公司 | Tamper-proof installation method, tamper-proof installation system, tamper-proof installation medium and tamper-proof installation equipment for application system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020078188A1 (en) * | 2000-12-18 | 2002-06-20 | Ibm Corporation | Method, apparatus, and program for server based network computer load balancing across multiple boot servers |
| CN101778132A (en) * | 2010-01-22 | 2010-07-14 | 杭州华三通信技术有限公司 | Method, system and device for network installation based on load balance |
| CN103023932A (en) * | 2011-09-21 | 2013-04-03 | 鸿富锦精密工业（深圳）有限公司 | Method and system for load balancing of server |
| CN103905531A (en) * | 2014-03-14 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Method for converting load balancing among nodes in cluster storage system |
-
2019
- 2019-09-30 CN CN201910943120.8A patent/CN110830550A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020078188A1 (en) * | 2000-12-18 | 2002-06-20 | Ibm Corporation | Method, apparatus, and program for server based network computer load balancing across multiple boot servers |
| CN101778132A (en) * | 2010-01-22 | 2010-07-14 | 杭州华三通信技术有限公司 | Method, system and device for network installation based on load balance |
| CN103023932A (en) * | 2011-09-21 | 2013-04-03 | 鸿富锦精密工业（深圳）有限公司 | Method and system for load balancing of server |
| CN103905531A (en) * | 2014-03-14 | 2014-07-02 | 浪潮电子信息产业股份有限公司 | Method for converting load balancing among nodes in cluster storage system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116016159A (en) * | 2022-12-16 | 2023-04-25 | 北京云端智度科技有限公司 | Tamper-proof installation method, tamper-proof installation system, tamper-proof installation medium and tamper-proof installation equipment for application system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2421785C2 (en) | Automated control of device drivers | |
| US9497295B2 (en) | Redistribution of operating environments for the redeployment of grid clients | |
| US20070214348A1 (en) | Method and apparatus for operating system deployment | |
| US7802084B2 (en) | System and method for management and installation of operating system images for computers | |
| US6963981B1 (en) | Method and apparatus for remote installation of an operating system over a network connection | |
| US7558867B2 (en) | Automatic firmware upgrade for a thin client using one or more FTP servers | |
| US6810478B1 (en) | System for remote booting of muntliple operating systems using chained bootstrap mechanism in a network | |
| KR101365904B1 (en) | Booting an operating system in discrete stages | |
| US7761867B2 (en) | Method and system for creation of operating system partition table | |
| US8245022B2 (en) | Method and system to support ISCSI boot through management controllers | |
| US7269721B2 (en) | Method, system, and apparatus for booting with remote configuration data | |
| US10146556B2 (en) | System and method to perform an OS boot using service location protocol and launching OS using a dynamic update of network boot order without a reboot | |
| US8250183B1 (en) | System and method for pre-installing of virtual server files | |
| WO2004025486A2 (en) | Use of off-motherboard resources in a computer system | |
| US20060155838A1 (en) | Program installation system and method using the same | |
| US8346886B2 (en) | System, method, and medium for configuring client computers to operate disconnected from a server computer while using a master instance of the operating system | |
| US20050160150A1 (en) | Apparatus and method for managing and transporting virtual disks over a network to networked stations | |
| US6687820B2 (en) | System includes a selection manager for remotely managing the selection of an operating system for a target computer | |
| US11159367B2 (en) | Apparatuses and methods for zero touch computing node initialization | |
| CN111857956B (en) | Virtual machine starting method and equipment | |
| CN101820387A (en) | Method for rapidly deploying extensible cluster | |
| US10938648B2 (en) | System and method for configuring an information handling system | |
| CN110908753A (en) | Intelligent fusion cloud desktop server, client and system | |
| CN113632064A (en) | Provisioning services (PVS) cloud streaming with read caching | |
| CN115562734A (en) | Data processor configuration method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200221 |