CN111641724B - Application method of LVS load balancer in cloud - Google Patents

Application method of LVS load balancer in cloud Download PDF

Info

Publication number
CN111641724B
CN111641724B CN202010497853.6A CN202010497853A CN111641724B CN 111641724 B CN111641724 B CN 111641724B CN 202010497853 A CN202010497853 A CN 202010497853A CN 111641724 B CN111641724 B CN 111641724B
Authority
CN
China
Prior art keywords
end server
load balancing
lvs
data packet
virtual node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010497853.6A
Other languages
Chinese (zh)
Other versions
CN111641724A (en
Inventor
张信杰
李彦君
胡章丰
任秋峥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Electronic Port Co.,Ltd.
Original Assignee
Shandong Huimao Electronic Port Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Huimao Electronic Port Co Ltd filed Critical Shandong Huimao Electronic Port Co Ltd
Priority to CN202010497853.6A priority Critical patent/CN111641724B/en
Publication of CN111641724A publication Critical patent/CN111641724A/en
Application granted granted Critical
Publication of CN111641724B publication Critical patent/CN111641724B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers

Abstract

The invention discloses an application method of an LVS load balancer under cloud, relating to the field of cloud computing virtual service; aiming at the openstack public cloud platform, load balancing configuration is carried out through the LVS load balancing virtual nodes, data packets are forwarded to a back-end server, an ovs flow table is received through the back-end server, corresponding source IP and destination IP are processed through the ovs flow table, corresponding mac addresses and ports are modified, and correct sending of the data packets is completed; by using the method and the system, the DR mode of the LVS can be realized without modifying any configuration of the back-end server under the public cloud environment, and the load balancing efficiency is improved, so that the operation resources are saved, and the operation cost is reduced.

Description

Application method of LVS load balancer under cloud
Technical Field
The invention discloses an application method, relates to the field of cloud computing virtual services, and particularly relates to an application method of an LVS load balancer in a cloud.
Background
Cloud computing provides a very convenient resource use mode and flexible resource expansion capability for users through virtualization abstraction of various physical resources such as computing, network, storage and the like. Load balancing is a basic function of cloud products, and has higher requirements on the forwarding performance of data packets, and the LVS is convenient to forward at four layers, so that the LVS becomes the first choice for realizing cloud load balancing. The LVS, also called Linux virtual server, is a relatively mature open source project, and functions are implemented by the ipvs module, which mainly works on the INPUT chain of the netfilter. The LVS has multiple working modes such as DR, NAT, FULLNAT and the like, wherein DR data packets are forwarded on a second layer, the performance of the mode is the best, but the DR mode needs to configure the rear-end REALSERVER, for example, VIP information needs to be configured on a rear-end server, and ARP corresponding processing is configured, so that the realization of the DR mode of the LVS network load balancer under an openstack public cloud environment has great limitation.
Disclosure of Invention
The invention provides an application method of an LVS load balancer in a cloud aiming at the problems in the prior art, and provides a method which can be used without additional configuration on a back-end server in a DR mode aiming at the limitation of DR mode configuration of the LVS load balancer under the current openstack public cloud platform.
The specific scheme provided by the invention is as follows:
an application method of an LVS load balancer under a cloud aims at an openstack public cloud platform, load balancing configuration is carried out through LVS load balancing virtual nodes, data packets are forwarded to a back-end server,
and receiving the ovs flow table through the back-end server, processing the corresponding source IP and the target IP through the ovs flow table, modifying the corresponding mac address and port, and finishing the correct sending of the data packet.
The application method of the LVS load balancer in the cloud judges whether an LVS load balancing virtual node exists, if yes, a load balancing creating command is sent to the LVS load balancing virtual node, and if not, the LVS load balancing virtual node is created.
In the application method of the LVS load balancer under the cloud, a load balancing creating command is received through the LVS load balancing virtual node, a load balancing example is created, and the load balancing example is configured on a corresponding network card through the VIP.
According to the application method of the LVS load balancer under the cloud, a back-end server adding command is received through the LVS load balancing virtual node, and the back-end server is added into LVS management.
In the application method of the LVS load balancer under the cloud, the process of ovs flow table processing is as follows:
modifying the VIP into the IP of the back-end server according to the data packet entering the back-end server, wherein the target mac is the mac of the back-end server, and the target port is the service port of the back-end server;
and aiming at the data packet sent by the back-end server, modifying the IP of the back-end server into VIP, the source mac into the mac of the back-end server, and the source port into the service port of the back-end server.
An application system of LVS load balancer in cloud comprises a configuration module and a data packet transmission module,
the configuration module performs load balancing configuration on the openstack public cloud platform through the LVS load balancing virtual node and forwards a data packet to a back-end server,
and the data packet transmission module receives the ovs flow table through the back-end server, processes the corresponding source IP and the target IP through the ovs flow table, modifies the corresponding mac address and port and completes the correct transmission of the data packet.
An application device of an LVS load balancer under cloud comprises: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is used for calling the machine readable program and executing the application method of the LVS load balancer under the cloud.
A computer readable medium having computer instructions stored thereon, which when executed by a processor, cause the processor to perform a method of applying an LVS load balancer in a cloud.
The invention has the advantages that:
the invention provides an application method of an LVS load balancer under a cloud, aiming at an openstack public cloud platform, carrying out load balancing configuration through an LVS load balancing virtual node, forwarding a data packet to a back-end server, receiving an ovs flow table through the back-end server, processing a corresponding source IP and a corresponding destination IP through the ovs flow table, modifying a corresponding mac address and a corresponding port, and finishing correct sending of the data packet; by using the method and the system, the DR mode of the LVS can be realized without modifying any configuration of the back-end server under the public cloud environment, and the load balancing efficiency is improved, so that the operation resources are saved, and the operation cost is reduced.
Drawings
FIG. 1 is a schematic diagram of the application of the process of the present invention;
FIG. 2 is a schematic flow diagram of the process of the present invention.
Detailed Description
The OpenStack cloud computing platform comprises two main modules, namely Nova and Swift, wherein the Nova and Swift are virtual server deployment and service computing modules for NASA development; the distributed cloud storage module developed by the Rackspace can be used together or separately. OpenStack supports almost all types of cloud environments, and the project aims to provide a cloud computing management platform which is simple to implement, can be expanded in a large scale, is rich and has a unified standard.
The present invention is further described below in conjunction with the following figures and specific examples so that those skilled in the art may better understand the present invention and practice it, but the examples are not intended to limit the present invention.
The invention provides an application method of an LVS load balancer under cloud, which aims at an openstack public cloud platform, performs load balancing configuration through an LVS load balancing virtual node, forwards a data packet to a back-end server,
and receiving the ovs flow table through the rear-end server, processing the corresponding source IP and the target IP through the ovs flow table, modifying the corresponding mac address and port, and finishing the correct sending of the data packet.
In the process, aiming at the openstack public cloud platform, a user can create an LVS load balancing virtual node in a VPC space, create a back-end server in the same VPC space, perform load balancing configuration through the LVS load balancing virtual node and add the back-end server into the LVS load balancing configuration, forward a data packet to the back-end server through the LVS load balancing virtual node,
and receiving an ovs flow table issued by the corresponding host through the back-end server, processing the corresponding source IP and destination IP through the ovs flow table, modifying the corresponding mac address and port, and finishing the correct sending of the data packet.
By using the method, the DR mode of the LVS can be realized without modifying any configuration of the back-end server under the public cloud environment, and the load balancing efficiency is improved, so that the operation resources are saved, and the operation cost is reduced.
In a specific embodiment of the present invention, it is described that, when a user issues a command for creating LVS load balancing through a console of an openstack public cloud platform by using the method of the present invention, the user determines whether a LVS load balancing virtual node exists in a VPC space of the user, if yes, the user sends the command for creating load balancing to the LVS load balancing virtual node, otherwise, the user creates the LVS load balancing virtual node,
the LVS load balancing virtual node receives a command for creating load balancing, at the same time, the load balancing node creates a load balancing example and receives a configuration request for user load balancing, actual configuration of load balancing is carried out on the load balancing node, such as basic operations of adding and deleting the load balancing example, adding and deleting a monitor, binding and unbinding a back-end server and the like, and the basic operations are configured on a corresponding network card through the VIP,
the LVS load balancing virtual node receives a back end server adding command issued by the console, adds a back end server into LVS management,
meanwhile, the console can send a flow table issuing instruction to the neutron, the back-end server receives the ovs flow table, the corresponding source IP and the corresponding destination IP are processed through the ovs flow table, the corresponding mac address and the corresponding port are modified, and correct sending of the data packet is completed.
The above embodiment illustrates a detailed process of the method implementation in the openstack public cloud platform.
In another embodiment of the present invention, the console sends an instruction for issuing a flow table to the neutron, adds a flow table corresponding to the back-end server, and the ovs flow table processing process:
aiming at a data packet entering a back-end server, the rule is that a target IP is VIP, a target port is a service port of the back-end server, a target mac is a mac of the back-end server, and the target IP is modified to be the IP of the back-end server;
aiming at a data packet sent by a back-end server, the rule is that a source IP is the back-end server IP, a source port is a service port of the back-end server, a source mac is the mac of the back-end server, and the source IP is modified into VIP.
Then Neutron receives the flow table command issued by the console, finds the host machine corresponding to the back-end server, adds the corresponding flow table,
before the real data packet of the user reaches the back-end server and the data packet sent by the back-end server is processed by the flow table, the corresponding IP can be modified, so that the data packet can correctly reach the destination.
The above embodiment specifically describes the process of processing the IP, mac address, and port by the flow table, so that the forwarding of the packet can correctly reach the destination, and the DR mode application of the LVS load balancer is implemented.
The invention also provides an application system of the LVS load balancer under the cloud, which comprises a configuration module and a data packet transmission module,
the configuration module performs load balancing configuration through the LVS load balancing virtual node aiming at the openstack public cloud platform, and forwards a data packet to a back-end server,
and the data packet transmission module receives the ovs flow table through the back-end server, processes the corresponding source IP and the target IP through the ovs flow table, modifies the corresponding mac address and port and completes the correct transmission of the data packet.
In the process, aiming at the openstack public cloud platform, a user can create an LVS load balancing virtual node by using a configuration module of the system in a VPC space, create a back-end server in the same VPC space, perform load balancing configuration through the LVS load balancing virtual node, add the back-end server into LVS load balancer configuration, forward a data packet to the back-end server through the LVS load balancing virtual node,
and enabling the back-end server to receive an ovs flow table issued by the corresponding host through the data packet transmission module, processing the corresponding source IP and destination IP through the ovs flow table, modifying the corresponding mac address and port, and finishing the correct transmission of the data packet.
The system can realize the DR mode of the LVS without modifying any configuration of a back-end server under a public cloud environment, and improves the efficiency of load balancing, thereby saving operation resources and reducing operation cost.
In a specific embodiment of the system of the present invention, it is described that, when the system of the present invention is used in an openstack public cloud platform, a user can issue a command for creating a LVS load balancing to a configuration module through a console of the platform, and determine whether a LVS load balancing virtual node exists in a VPC space of the user, if so, send the command for creating the load balancing to the LVS load balancing virtual node, otherwise, create the LVS load balancing virtual node,
the LVS load balancing virtual node created by the configuration module receives a command for creating load balancing, creates a load balancing example at the load balancing node and receives a configuration request for user load balancing, performs actual configuration of load balancing on the load balancing node, such as basic operations of adding and deleting the load balancing example, adding and deleting a listener, binding and unbinding a back-end server and the like, and configures the virtual node to a corresponding network card through the VIP,
the LVS load balancing virtual node receives a back end server adding command issued by the console, adds a back end server into LVS management,
meanwhile, the console can send a flow table issuing instruction to the neutron, the back-end server receives the ovs flow table, the corresponding source IP and the corresponding destination IP are processed through the ovs flow table, the corresponding mac address and the corresponding port are modified, and correct sending of the data packet is completed.
The above embodiment illustrates a detailed process implemented by the system of the present invention on the openstack public cloud platform.
In another embodiment of the system of the present invention, the console sends an instruction for issuing a flow table to the neutron, a flow table corresponding to the back-end server is added, and the data packet transmission module processes the flow table through the ovs:
aiming at a data packet entering a back-end server, the rule is that a target IP is VIP, a target port is a service port of the back-end server, a target mac is a mac of the back-end server, and the target IP is modified to be the IP of the back-end server;
aiming at a data packet sent by a back-end server, the rule is that a source IP is the back-end server IP, a source port is a service port of the back-end server, a source mac is the mac of the back-end server, and the source IP is modified into VIP.
Then Neutron receives the flow table command issued by the console, finds the host machine corresponding to the back-end server, adds the corresponding flow table,
before the real data packet of the user reaches the back-end server and the data packet sent by the back-end server is processed by the flow table, the corresponding IP can be modified, so that the data packet can correctly reach the destination.
The above embodiment specifically describes the process of processing the IP, mac address, and port by the flow table, so that the forwarding of the packet can correctly reach the destination, and the DR mode application of the LVS load balancer is implemented.
The invention also provides an application device of the LVS load balancer under the cloud, which comprises the following components: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is used for calling the machine readable program and executing the application method of the LVS load balancer under the cloud.
And a computer readable medium having stored thereon computer instructions, which when executed by a processor, cause the processor to perform a method for applying an LVS load balancer under a cloud.
Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the embodiments described above are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.
In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.
Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.
Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then a CPU or the like mounted on the expansion board or the expansion unit is caused to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the embodiments described above.
It should be noted that not all steps and modules in the above flows and system structure diagrams are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.
In the above embodiments, the hardware unit may be implemented mechanically or electrically. For example, a hardware element may comprise permanently dedicated circuitry or logic (such as a dedicated processor, FPGA or ASIC) to perform the corresponding operations. The hardware elements may also comprise programmable logic or circuitry, such as a general purpose processor or other programmable processor, that may be temporarily configured by software to perform the corresponding operations. The specific implementation (mechanical, or dedicated permanent, or temporarily set) may be determined based on cost and time considerations.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.

Claims (4)

1. An application method of an LVS load balancer under cloud is characterized in that whether an LVS load balancing virtual node exists or not is judged for an openstack public cloud platform, if yes, a load balancing creating command is sent to the LVS load balancing virtual node, otherwise, the LVS load balancing virtual node is created, the load balancing creating command is received through the LVS load balancing virtual node, a load balancing example is created, the LVS load balancing virtual node is configured on a corresponding network card through VIP,
the load balancing configuration is carried out through the LVS load balancing virtual node, the LVS load balancing virtual node receives a back-end server adding command, adds a back-end server into LVS management and forwards a data packet to the back-end server,
receiving an ovs flow table through a host corresponding to a back-end server, processing a corresponding source IP and a corresponding destination IP through the ovs flow table, modifying a corresponding mac address and port, and finishing correct sending of a data packet, wherein the ovs flow table processing process comprises the following steps: aiming at the data packet entering the back-end server, the rule is that the target IP is VIP, the target port is a service port of the back-end server, the target mac is the mac of the back-end server, and the target IP is modified to be the IP of the back-end server;
aiming at the data packet sent by the back-end server, the rule is that the source IP is the back-end server IP, the source port is the service port of the back-end server, the source mac is the mac of the back-end server, and the source IP is modified to be the VIP.
2. An application system of LVS load balancer in cloud is characterized in that the system comprises a configuration module and a data packet transmission module,
the configuration module judges whether an LVS load balancing virtual node exists or not aiming at the openstack public cloud platform, if so, a load balancing creating command is sent to the LVS load balancing virtual node, otherwise, the LVS load balancing virtual node is created, the load balancing creating command is received through the LVS load balancing virtual node, a load balancing example is created, and the load balancing example is configured on a corresponding network card through the VIP,
the configuration module performs load balancing configuration through the LVS load balancing virtual node, receives a back-end server adding command through the LVS load balancing virtual node, adds the back-end server into LVS management, and forwards a data packet to the back-end server,
the data packet transmission module receives the ovs flow table through the back-end server, processes the corresponding source IP and the target IP through the ovs flow table, modifies the corresponding mac address and port, and completes the correct transmission of the data packet, wherein the data packet transmission module utilizes the ovs flow table processing process: aiming at a data packet entering a back-end server, the rule is that a target IP is VIP, a target port is a service port of the back-end server, a target mac is a mac of the back-end server, and the target IP is modified to be the IP of the back-end server;
aiming at the data packet sent by the back-end server, the rule is that the source IP is the back-end server IP, the source port is the service port of the back-end server, the source mac is the mac of the back-end server, and the source IP is modified to be the VIP.
3. An LVS load balancer application device under cloud is characterized by comprising: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine readable program to execute the method of claim 1 for applying the LVS load balancer to the cloud.
4. Computer readable media characterized in that said computer readable media has stored thereon computer instructions which, when executed by a processor, cause said processor to execute a method for cloud application of an LVS load balancer as claimed in claim 1.
CN202010497853.6A 2020-06-04 2020-06-04 Application method of LVS load balancer in cloud Active CN111641724B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010497853.6A CN111641724B (en) 2020-06-04 2020-06-04 Application method of LVS load balancer in cloud

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010497853.6A CN111641724B (en) 2020-06-04 2020-06-04 Application method of LVS load balancer in cloud

Publications (2)

Publication Number Publication Date
CN111641724A CN111641724A (en) 2020-09-08
CN111641724B true CN111641724B (en) 2023-02-21

Family

ID=72331239

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010497853.6A Active CN111641724B (en) 2020-06-04 2020-06-04 Application method of LVS load balancer in cloud

Country Status (1)

Country Link
CN (1) CN111641724B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104780115A (en) * 2014-01-14 2015-07-15 上海盛大网络发展有限公司 Load balancing method and load balancing system in cloud computing environment
CN106302225A (en) * 2016-10-18 2017-01-04 上海优刻得信息科技有限公司 A kind of method and apparatus of server load balancing
CN106686085A (en) * 2016-12-29 2017-05-17 华为技术有限公司 Load balancing method, apparatus and system

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102523302B (en) * 2011-12-26 2015-08-19 华为数字技术(成都)有限公司 The load-balancing method of cluster virtual machine, server and system
CN106533942B (en) * 2016-11-30 2019-06-14 北京邮电大学 A kind of data processing method and device based on SDN
CN107846365A (en) * 2017-10-24 2018-03-27 赞同科技股份有限公司 A kind of load balancing based on SDN realizes system and method
CN110708393B (en) * 2019-10-21 2023-11-21 北京百度网讯科技有限公司 Method, device and system for transmitting data

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104780115A (en) * 2014-01-14 2015-07-15 上海盛大网络发展有限公司 Load balancing method and load balancing system in cloud computing environment
CN106302225A (en) * 2016-10-18 2017-01-04 上海优刻得信息科技有限公司 A kind of method and apparatus of server load balancing
CN106686085A (en) * 2016-12-29 2017-05-17 华为技术有限公司 Load balancing method, apparatus and system

Also Published As

Publication number Publication date
CN111641724A (en) 2020-09-08

Similar Documents

Publication Publication Date Title
CN107947961B (en) SDN-based Kubernetes network management system and method
US9749145B2 (en) Interoperability for distributed overlay virtual environment
CN111866064B (en) Load balancing method, device and system
US8363656B2 (en) Multiple virtual machines sharing a single IP address
US7111303B2 (en) Virtual machine operating system LAN
US10491464B2 (en) Network topology assisted device provisioning
US7899047B2 (en) Virtual network with adaptive dispatcher
US20090063706A1 (en) Combined Layer 2 Virtual MAC Address with Layer 3 IP Address Routing
US20120297384A1 (en) Virtual Managed Network
US20120290703A1 (en) Distributed Policy Service
WO2014139383A1 (en) Virtual gateways and implicit routing in distributed overlay virtual environments
JP2003208365A (en) Virtual network with adaptive dispatcher
CN111641719B (en) Intranet type load balancing implementation method based on Openstack and storage medium
KR101960239B1 (en) Isolation of virtual machine i/o in multi-disk hosts
CN106657180B (en) Information transmission method and device for cloud service, terminal equipment and system
WO2013097484A1 (en) Method, server and system for balancing loads of virtual machine cluster
CN107517129B (en) Method and device for configuring uplink interface of equipment based on OpenStack
CN105657078A (en) Data transmission method, data transmission device and multi-layer network manager
US20160241643A1 (en) Protocol independent storage discovery and enablement
CN111641724B (en) Application method of LVS load balancer in cloud
CN108353017B (en) Computing system and method for operating multiple gateways on a multi-gateway virtual machine
CN114025370B (en) Data message transmission method, medium, system and computing equipment
US11165681B2 (en) Inter-autonomous system trace route message
CN111404811A (en) Message forwarding method and device
US20240022513A1 (en) Communication Method and Apparatus

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230406

Address after: Energy Building 1-2201, No. 10777 Jingshi Road, Jinan Area, Jinan Free Trade Pilot Zone, Shandong Province, 250000

Patentee after: Shandong Electronic Port Co.,Ltd.

Address before: Floor S06, Inspur Science Park, No. 1036, Inspur Road, hi tech Zone, Jinan City, Shandong Province

Patentee before: SHANDONG HUIMAO ELECTRONIC PORT Co.,Ltd.