CN110717853B - Optical image processing system based on embedded GPU - Google Patents

Optical image processing system based on embedded GPU Download PDF

Info

Publication number
CN110717853B
CN110717853B CN201911270535.XA CN201911270535A CN110717853B CN 110717853 B CN110717853 B CN 110717853B CN 201911270535 A CN201911270535 A CN 201911270535A CN 110717853 B CN110717853 B CN 110717853B
Authority
CN
China
Prior art keywords
module
embedded gpu
image processing
interface
fpga
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911270535.XA
Other languages
Chinese (zh)
Other versions
CN110717853A (en
Inventor
汪成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Jingce Electronic Group Co Ltd
Wuhan Jingli Electronic Technology Co Ltd
Original Assignee
Wuhan Jingce Electronic Group Co Ltd
Wuhan Jingli Electronic Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Jingce Electronic Group Co Ltd, Wuhan Jingli Electronic Technology Co Ltd filed Critical Wuhan Jingce Electronic Group Co Ltd
Priority to CN201911270535.XA priority Critical patent/CN110717853B/en
Publication of CN110717853A publication Critical patent/CN110717853A/en
Application granted granted Critical
Publication of CN110717853B publication Critical patent/CN110717853B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G06F15/163—Interprocessor communication
    • G06F15/17—Interprocessor communication using an input/output type connection, e.g. channel, I/O port

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

The invention discloses an optical image processing system based on an embedded GPU. The system comprises a master embedded GPU module, a slave embedded GPU module, an FPGA module and a storage module; the storage module is connected with the main embedded GPU module, and the storage module stores an operating system and is used for installing the operating system for the main embedded GPU module; the FPGA module is used for receiving the configuration parameters and the control commands sent by the master embedded GPU module, sending the acquired image data to the slave embedded GPU module for image data processing, receiving the processing result of the slave embedded GPU module and outputting the processing result to the master embedded GPU module. The invention can realize the simplification, integration, miniaturization, high flexibility and low cost of the system structure.

Description

Optical image processing system based on embedded GPU
Technical Field
The invention belongs to the technical field of electronics, and particularly relates to an optical image processing system based on an embedded GPU.
Background
Display screens such as LCDs are widely used in the computer, communication, consumer electronics, and other industries. The manufacturing process of the display screen is complicated, so that the display screen may have defects, and therefore, automatic Optical inspection, namely aoi (automated Optical inspection), is required. An important part of an AOI system is the image processing system. The image processing system of AOI may need to perform a large amount of image calculation processing work such as deep learning. With the development of the display industry of LCD and OLED, the complexity and requirements of image processing systems for AOI have also increased exponentially, and the algorithms involved in image processing are heavy.
The conventional AOI optical image processing system usually adopts a server + GPU structure, that is, a server of an X86 or ARM architecture is often adopted, a professional GPU standard card is mounted to form a GPU server, the server is used as a main processor, and the GPU is only used for performing deep learning calculation of image processing. However, this structure has the following disadvantages:
(1) the system has limited computing processing capacity, slow response speed, high overall cost and large system volume;
(2) the system structure is complex, and generally a plurality of functional modules are required to be combined, for example, different image capturing units are required to be configured for screens with different resolutions;
(3) and the system architecture is formed by separating, and the flexibility of use and functions is low.
Disclosure of Invention
In view of at least one of the drawbacks or needs for improvement of the related art, the present invention provides an embedded GPU-based optical image processing system that can achieve simplification of system structure, integration, miniaturization, high flexibility, and low cost.
To achieve the above object, according to one aspect of the present invention, there is provided an optical image processing system based on an embedded GPU, including a master embedded GPU module, a slave embedded GPU module, an FPGA module, and a storage module;
the storage module is connected with the main embedded GPU module, and the storage module stores an operating system and is used for installing the operating system for the main embedded GPU module;
the FPGA module is used for receiving the configuration parameters and the control commands sent by the master embedded GPU module, sending the acquired image data to the slave embedded GPU module for image data processing, receiving the processing result of the slave embedded GPU module, and sending the processing result to the master GPU module for output.
Preferably, the FPGA module is used as an FPGA core backplane and includes a plurality of high-speed connectors, the FPGA core backplane is connected to a plurality of daughter cards through the plurality of high-speed connectors, and a hot plug circuit is provided between the high-speed connectors and the daughter cards.
Preferably, the daughter card comprises a Camera Link interface daughter card, the Camera Link interface daughter card is used for being connected with a CL interface industrial Camera with a high pixel to acquire image data, and the Camera Link interface daughter card comprises a sub FPGA module, and the sub FPGA module is connected with the FPGA core backplane.
Preferably, the daughter card comprises a G-ETH interface daughter card, the G-ETH interface daughter card is used for connecting with a low-pixel portal interface industrial camera to acquire image data, and the G-ETH interface daughter card comprises a sub-FPGA module, and the sub-FPGA module is connected with the FPGA core backplane.
Preferably, the daughter card includes a QSFP + optical fiber interface daughter card and/or an IO daughter card, the QSFP + optical fiber interface daughter card is used to connect a device supporting an optical fiber interface, and the IO daughter card is used for development and debugging of a system.
Preferably, the number of slave embedded GPU modules is increased or decreased depending on the application scenario of the image processing system.
Preferably, the slave embedded GPU module supports hot-plugging.
Preferably, the main embedded GPU module is connected to the FPGA module through a PCIe interface.
Preferably, the main embedded GPU module is connected to the storage module through a SATA interface or a PCIe interface.
Preferably, the master GPU module is connected to a display device and/or an input device, the display device is used to provide a human-computer interaction interface or display an image processing result, and the input device is used to receive an instruction input by a user.
Generally, the above technical solutions contemplated by the present invention have the following advantages compared with the prior art:
(1) an external storage module is used for installing an operating system for a main embedded GPU, the main embedded GPU is used as a main processor, the parallel processing advantage of an FPGA is used for forming a small-sized computing system, the system is simple in structure, highly integrated, small-sized, wide in applicability and low in cost, and an independent PC or a server is not needed any more as a processor;
(2) the whole image processing system consists of a backboard and a daughter card, is in modular design, can be flexibly configured according to actual application scenes, and has high functional flexibility;
(3) the modules can support hot plugging, and the installation and the use are convenient.
Drawings
FIG. 1 is a schematic structural diagram of an embedded GPU-based optical image processing system according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an FPGA module provided in the embodiment of the present invention;
fig. 3 is a schematic structural diagram of a Camera Link interface daughter card according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a daughter card of the G-ETH interface provided in the embodiment of the present invention;
FIG. 5 is a schematic diagram of a QSFP + fibre optic interface daughter card architecture according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an IO interface daughter card according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The optical image processing system based on the embedded GPU comprises a master embedded GPU module, a slave embedded GPU module, an FPGA module and a storage module. The storage module is connected with the main embedded GPU module, and the storage module stores an operating system which is used for installing the operating system for the main embedded GPU module. The FPGA module is used for receiving the configuration parameters and the control commands sent by the master embedded GPU module to realize control management of the system, and on the other hand, the FPGA module is used for sending the acquired image data to the slave embedded GPU module to perform data processing, receiving the processing result of the slave embedded GPU module and sending the processing result to the master GPU module to be output. Therefore, the invention utilizes the external storage module as the main embedded GPU to install the operating system, and utilizes the parallel processing advantage of the FPGA to form a small, convenient and applicable image processing computing system.
Fig. 1 is a schematic diagram of an architecture of an embedded GPU-based optical image processing system according to an embodiment of the present invention. In the hardware platform system, the embedded GPU module comprises a main embedded GPU module (GPU 1) and a plurality of auxiliary embedded GPU modules (GPU 2-GPUn), wherein n is an integer larger than 2. The main embedded GPU module is connected with a storage module which stores a Linux operating system, and the storage module can be a mechanical hard disk or SSD or Pcei _ SSD and the like and is used for storing the operating system and other related data. The FPGA module is used as a control module, expands external multiple PCIe signals, is used for being interconnected with the main embedded GPU module and the auxiliary embedded GPU module, and is mainly used for controlling the system and forwarding the acquired image data.
The GPU1 is used as a host processor. The GPU1 may be configured to receive configuration parameters and control commands and send the configuration parameters and control commands to the FPGA module through human-computer interaction, and also be configured to receive and output a processing result of the FPGA module. GPU1 may support a simplified version of the Linux operating system. The memory controller inside the GPU can be used for externally hanging a memory with certain capacity, the form of the memory can be a memory bank form, and the memory controller can also be a memory granule form, and the memory controller is mainly used for caching data. And externally connecting a storage device through a related storage bus interface, and installing a Linux operating system. The storage bus interface may be a SATA or PCIe interface. The other GPU modules (GPU 2 to GPUn) are used as slave devices, mainly for calculation processing of image processing data. The number of GPU modules can be flexibly configured according to actual application scenes and hardware resources, expansion or deletion is carried out, and the flexibility of use and the flexibility of functions are greatly improved through a modularized design.
Alternatively, the embedded GPU module may include only a master embedded GPU module that, in addition to functioning as a master processor, may also be used to perform computational processing, such as image processing data, in place of the slave embedded GPU module described below.
The GPU1 may be connected to a display device that is used to provide a human-computer interface or to display image processing results. For example, the GPU1 is connected to a display through a video interface as an operation interface for human-computer interface interaction. The video interface may be a DP interface. The GPU1 may be coupled to an input device for receiving user input commands. For example, the GPU1 connects a mouse and a keyboard through a USB interface. Thus, a complete miniaturized computer system can be formed by the highly integrated GPU1 module and peripheral devices, and input and output of image data are calculated, analyzed and managed.
The PCIe signal of the embedded GPU module can be used as both a root port and an endpoint. In this example, PCIe of the GPU1 is mainly used for a root port, one group of X4 Lane signals is used for externally connecting a PCIe-SSD to expand the storage module, the other group of X8 Lane signals is used for connecting an FPGA for high-speed data transmission, and other slave GPU devices can be used as a root port or an endpoint, which is mainly based on different application scenarios.
Based on the embedded GPU module, the image processing system has strong computing power and small volume, realizes the embedded system with multiple GPU modules, and the GPU modules support plugging and unplugging, so that the number of the GPU modules can be flexibly configured according to actual application scenes.
The FPGA module is mainly used for the control function of the system, commands from the operating system of the GPU1, and is mainly used for the control and management of the system. Distributing data to be processed to different GPU processing units and adopting a distributed processing architecture; meanwhile, the advantages of parallel processing of the FPGA are fully utilized, and data exchange and coordination processing of image data between the image collector and the GPU module are realized.
Optionally, the FPGA module is used as an FPGA core backplane, and includes a plurality of high-speed connectors, the FPGA core backplane is connected to a plurality of daughter cards through the plurality of high-speed connectors, and a hot plug circuit is provided between the high-speed connectors and the daughter cards. The hot plug circuit comprises hot plug protection of a power supply and hot plug protection of important signals. As shown in fig. 2, the FPGA module is used as an external interface of the core backplane and adopts a consistent high-speed connector, and the pin definitions of the FPGA module are also kept completely consistent or compatible with each other, so as to ensure that different types of daughter cards except the IO daughter card can be hot-plugged without distinction.
Optionally, the daughter card includes a Camera Link interface daughter card, as shown in fig. 3, 4 Camera Link interfaces are extended through interconnection between the SERDES signal pulled out from the backplane and the FPGA of the daughter card, and are used for connecting to the CL interface industrial Camera with a high pixel to acquire image data. And data acquired by the high-pixel industrial camera is transmitted to the FPGA backboard through the high-speed connector, and then the data is distributed to each slave GPU for data processing under the control of the master GPU.
Optionally, the daughter card includes a G-ETH interface daughter card, as shown in fig. 4, the SERDES signals pulled out from the backplane are interconnected with the FPGA of the daughter card, so as to expand 4 POE-supported gigabit network port interfaces, which are used for connecting the low-pixel network port interface industrial camera to acquire image data. Similarly, data collected by the low-pixel industrial camera is transmitted to the FPGA backboard through the high-speed connector, and then the data are distributed to each slave GPU for data processing under the control of the master GPU.
Optionally, the daughter card includes a QSFP + fibre interface daughter card, as shown in fig. 5, the SERDES signal pulled out from the backplane is connected to the daughter card, and two fibre interfaces of QSFP + are output for connecting devices supporting the fibre interfaces, so as to extend other interface signals. The interface is needed when interaction with other devices is required.
Optionally, the daughter card includes an IO interface daughter card, as shown in fig. 6, the IO signal required by the system is pulled out from the backplane, such as Uart/JTAG/GPIO/LAN interface, which is mainly used for development debugging and debug of the system.
All daughter cards and backplane interfaces can be designed to support hot plug, which is convenient for debugging and on-site operation and maintenance.
The interface definition and size of the daughter cards may be consistent. The daughter card can select the daughter cards with different interface types according to actual application requirements, wherein the IO interface card can be fixed externally, and the IO daughter card is installed at the position of any end. The other interface sub-cards except the IO card can support any hot plug without difference, and each type of sub-card has a specific ID number or FRU information to be uniquely identified by the main controller.
Therefore, the whole optical image processing system consists of the backboard and the daughter cards, is designed in a modularized mode, and can be flexibly configured according to practical application scenes, for example, different numbers and different types of cameras can be configured for automatic optical detection of screens with different resolutions.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (10)

1. An optical image processing system based on an embedded GPU is characterized by comprising a master embedded GPU module, a slave embedded GPU module, an FPGA module and a storage module;
the main embedded GPU module is used as a main processor and sends configuration parameters and control commands to the FPG A module;
the storage module is connected with the main embedded GPU module, and the storage module stores an operating system and is used for installing the operating system for the main embedded GPU module;
the FPGA module is used for receiving the configuration parameters and the control commands sent by the master embedded GPU module, sending the acquired image data to the slave embedded GPU module for image data processing, receiving the processing result of the slave embedded GPU module, and sending the processing result to the master embedded GPU module for output;
the slave embedded GPU module is used for image data processing.
2. The embedded GPU-based optical image processing system of claim 1, wherein the FPGA module is used as an FPGA core backplane and comprises a plurality of high-speed connectors, the FPGA core backplane is connected with a plurality of daughter cards through the plurality of high-speed connectors, and hot plug circuits are arranged between the high-speed connectors and the daughter cards.
3. The embedded GPU-based optical image processing system of claim 2, wherein said daughter card comprises a Camera Link interface daughter card, said Camera Link interface daughter card is used to connect with a high pixel CL interface industrial Camera for collecting image data, said Camera Link interface daughter card comprises sub FPGA modules, and said sub FPGA modules are connected with said FPGA core backplane.
4. The embedded GPU-based optical image processing system of claim 2, wherein the daughter card comprises a G-ETH interface daughter card, the G-ETH interface daughter card is used for connecting with a low-pixel network port interface industrial camera to acquire image data, and the G-ETH interface daughter card comprises a sub FPGA module, and the sub FPGA module is connected with the FPGA core backplane.
5. The embedded GPU-based optical image processing system of claim 2, wherein the sub-cards comprise a QSFP + fiber interface sub-card and/or an IO sub-card, the QSFP + fiber interface sub-card is used for connecting devices supporting fiber interfaces, and the IO sub-card is used for development and debugging of the system.
6. An embedded GPU-based optical image processing system as in any one of claims 1-5, characterized in that the number of slave embedded GPU modules is increased or decreased according to the application scenario of the image processing system.
7. An embedded GPU-based optical image processing system as claimed in any one of claims 1 to 5, wherein the slave embedded GPU module supports hot-plug.
8. An embedded GPU-based optical image processing system as claimed in any one of claims 1 to 5, wherein the master embedded GPU module is connected to the FPGA module via a PCIe interface.
9. An embedded GPU-based optical image processing system as claimed in any one of claims 1 to 5, characterized in that the master embedded GPU module is connected to the storage module via a SATA interface or a PCIe interface.
10. An embedded GPU-based optical image processing system as claimed in any one of claims 1 to 5, wherein the main embedded GPU module is connected with a display device and/or an input device, the display device is used for providing a human-computer interaction interface or displaying image processing results, and the input device is used for receiving instructions input by a user.
CN201911270535.XA 2019-12-12 2019-12-12 Optical image processing system based on embedded GPU Active CN110717853B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911270535.XA CN110717853B (en) 2019-12-12 2019-12-12 Optical image processing system based on embedded GPU

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911270535.XA CN110717853B (en) 2019-12-12 2019-12-12 Optical image processing system based on embedded GPU

Publications (2)

Publication Number Publication Date
CN110717853A CN110717853A (en) 2020-01-21
CN110717853B true CN110717853B (en) 2020-05-22

Family

ID=69216655

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911270535.XA Active CN110717853B (en) 2019-12-12 2019-12-12 Optical image processing system based on embedded GPU

Country Status (1)

Country Link
CN (1) CN110717853B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI792024B (en) 2020-07-22 2023-02-11 瑞昱半導體股份有限公司 Graphics processing device
CN114064557A (en) * 2020-07-29 2022-02-18 瑞昱半导体股份有限公司 Image processing apparatus
CN112328532B (en) * 2020-11-02 2024-02-09 长沙景嘉微电子股份有限公司 Method and device for multi-GPU communication, storage medium and electronic device
CN112822129A (en) * 2021-02-05 2021-05-18 井芯微电子技术(天津)有限公司 PCIe switching equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109032018A (en) * 2018-08-02 2018-12-18 清华大学 Unmanned plane general signal processing device based on embedded gpu
CN109142379A (en) * 2018-09-19 2019-01-04 武汉意普科技有限责任公司 SOC embedded machine vision equipment based on FPGA

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102541640B (en) * 2011-12-28 2014-10-29 厦门市美亚柏科信息股份有限公司 Cluster GPU (graphic processing unit) resource scheduling system and method
US20140210833A1 (en) * 2013-01-29 2014-07-31 Kabushiki Kaisha Toshiba Information processing apparatus and control method for information processing apparatus
CN103869292B (en) * 2014-04-02 2014-12-24 清华大学 General purpose radar imaging processing system based on embedded GPU
CN110096367A (en) * 2019-05-14 2019-08-06 宁夏融媒科技有限公司 A kind of panorama real-time video method for stream processing based on more GPU

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109032018A (en) * 2018-08-02 2018-12-18 清华大学 Unmanned plane general signal processing device based on embedded gpu
CN109142379A (en) * 2018-09-19 2019-01-04 武汉意普科技有限责任公司 SOC embedded machine vision equipment based on FPGA

Also Published As

Publication number Publication date
CN110717853A (en) 2020-01-21

Similar Documents

Publication Publication Date Title
CN110717853B (en) Optical image processing system based on embedded GPU
CN107423169B (en) Method and system for testing high speed peripheral device interconnection equipment
US9934187B2 (en) Hot-pluggable computing system
CN112285530A (en) Universal testing device for VPX high-speed signal board
US10614022B2 (en) PCIe fabric connectivity expansion card
US20200287955A1 (en) VIDEO REDIRECTION ACROSS MULTIPLE INFORMATION HANDLING SYSTEMS (IHSs) USING A GRAPHICS CORE AND A BUS BRIDGE INTEGRATED INTO AN ENCLOSURE CONTROLLER (EC)
CN110766600B (en) Image processing system with distributed architecture
JP5859663B2 (en) System for performing connection, IO connector assembly and operation method
TWI570566B (en) Implementing io expansion cards
US11308002B2 (en) Systems and methods for detecting expected user intervention across multiple blades during a keyboard, video, and mouse (KVM) session
CN115981971A (en) Lighting method of server hard disk and server
CN211239961U (en) Card insertion type video processing apparatus and display system
CN100476794C (en) Mainboard of four-path server
JP2013069269A (en) Electric apparatus provided with a plurality of data connection ports
CN117349212A (en) Server main board and solid state disk insertion detection method thereof
CN210324191U (en) Computer module and mainboard
CN111930647A (en) USB interface working mode selection device and method and android device
CN115718529A (en) Reinforced computer mainboard system based on loongson CPU COME module
CN206363303U (en) A kind of CPU module based on VPX structures
CN213365381U (en) Main board
CN115543246A (en) BMC and server
US8612548B2 (en) Computer server system and computer server for a computer server system
US20200117631A1 (en) Automated device discovery system
CN211454416U (en) VPX 3U computer mainboard based on explain 121 treater
CN112783814A (en) Clock circuit, electronic device and method for multi-mode PCIE (peripheral component interface express) spread spectrum

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant