CN115150531A - Industrial intelligent camera and working method thereof - Google Patents

Abstract

The invention discloses an industrial intelligent camera and a working method thereof, belonging to the field of industrial digital image sensing, and realizing image preprocessing and real-time optical axis control in an FPGA system; in a GPU + ARM system, the GPU realizes image wavefront extraction and image identification; the ARM controls the light beam quality, and performs grid configuration on the FPGA system to realize the management functions of input and output display, data storage and the like. An open type Linux system is embedded in the ARM, and a control algorithm is completely opened to a user, so that the intelligent industrial control based on image recognition can be realized besides the self-adaptive optical control as long as different user codes are embedded. The invention realizes the integrated camera of image acquisition, image recognition and high-speed real-time control by integrating the front-end FPGA preprocessing and optical axis control system and the rear-end light beam quality control system into a whole camera.

Description

Industrial intelligent camera and working method thereof

Technical Field

The invention relates to the field of industrial digital image sensing, in particular to an industrial intelligent camera and a working method thereof.

Background

The industrial intelligent camera is an industrial digital image sensing device which integrates a series of functions such as high-speed image acquisition, intelligent image recognition and some intelligent control algorithms. Laser beam quality control and photostability control are achieved by means of adaptive optics techniques. The invention provides an industrial intelligent camera special for adaptive optics, which fills the blank.

Disclosure of Invention

The invention aims to overcome the defects of the prior art, provides an industrial intelligent camera and a working method thereof, and fills the industrial blank.

The purpose of the invention is realized by the following scheme:

an industrial intelligent camera comprises an image acquisition system, an image preprocessing system and a core processing system; the image acquisition system is used for transmitting the acquired image to the image preprocessing system; the image preprocessing system is provided with an image light spot mass center coordinate extracting module for transmitting the extracted image light spot mass center coordinate and the original image to the core processing system, and is also provided with an optical axis control module for calculating an optical axis adjusting quantity and then controlling an optical axis through control output; the core processing system is of a dual-processor architecture, wherein the first processor is used for image wavefront extraction and image identification, and the second processor is used for performing light beam quality control and configuring the image preprocessing system; the image acquisition system, the image preprocessing system and the core processing system are all integrated in one camera.

The system further comprises a far-field image acquisition system, wherein the far-field image acquisition system is provided with an aperture lens, and the aperture lens is used for acquiring a target recognition far-field image; for the original image transmitted from the far-field image acquisition system to the image preprocessing system, the image preprocessing system does not perform any calculation, and only transmits the original image to the core processing system.

Furthermore, the near field image acquisition system comprises a photosensitive surface and an interface circuit substrate, wherein the photosensitive surface is connected with the interface circuit substrate, and the interface circuit substrate comprises a high-speed interface board based on a differential conversion circuit and used for transmitting photosensitive electric signals in a parallel mode.

Furthermore, the second processor is embedded with an open type Linux system, and control algorithms are all opened to users, and the users can realize different industrial control capabilities based on image recognition by embedding different user codes.

Further, the image preprocessing system comprises an image reading circuit, an FPGA processor, an optical axis control RS422 port driving circuit and a PCIEx4 transmission interface drive;

the image reading circuit is used for transferring the differential signal transmitted by the image acquisition system into an original image buffer in the FPGA processor in real time, and comprises a differential parallel port image reading circuit;

the FPGA processor extracts a centroid coordinate through an image, adds a control algorithm to the centroid coordinate, calculates an optical axis adjustment amount by using the control algorithm, and then controls an RS422 port driving circuit to drive an RS422 port to output through an optical axis so as to control the optical axis;

and the PCIEx4 transmission interface driver is used for packaging and transmitting the centroid coordinate array and the original image preprocessed by the FPGA processor to the PCIEx4 interface.

Furthermore, the core processing system comprises a circuit system interface substrate, a solid state disk and a first network interface driving circuit, the first processor is a GPU processor, the second processor is an ARM processor, and the GPU processor and the ARM processor are integrated into a whole chip to form a GPU + ARM core board;

the circuit system interface substrate is used for converting and driving various interfaces, and the various interfaces comprise a PCIEx4 interface from the FPGA system to the GPU + ARM core board, a PCIE interface from the GPU + ARM core board to the solid state disk, and an external network port, a USB interface, an HDMI interface, a CAN interface and an RS422 interface of the GPU + ARM core board;

the solid state disk is used for installing an embedded linux operating system and installing a user program APP and data file storage;

the first network port driving circuit is used for outputting the control quantity calculated by the GPU + ARM core board through a control algorithm through a network port so as to control the deformable mirror and control the quality of the light beam; the network port comprises a high-speed gigabit network port.

Furthermore, the image preprocessing system comprises a second network port driving circuit and a high-speed driving optical fiber interface circuit which are both used for controlling the function switch.

Further, the core processing system comprises a USB and HDMI interface driving circuit, a CAN and RS422 interface driving circuit;

the USB and HDMI interface driving circuit is used for driving the USB and HDMI interfaces; the USB interface comprises a MinUSB interface and a USB-Type-C interface which are used for inputting or outputting external data; the HDMI provides a display interface for the outside, and is a digital high-definition interface for monitoring;

the CAN and RS422 interface driver is used for driving the CAN interface and the RS422 interface and providing a multifunctional output interface channel for the outside; the multifunctional output interface channel comprises an optical axis control output channel based on an RS422 interface and a camera cascade control channel based on a CAN.

An operating method of an industrial intelligent camera comprises the following steps:

after the industrial intelligent camera is started, starting a linux system in a GPU + ARM core module, and starting a user APP; a user APP receives grid coordinate configuration parameters sent by an upper computer and configuration parameters related to an optical system through a network port, and simultaneously the APP sends the grid configuration parameters to the FPGA system through a PCIEx4 interface for preprocessing of the FPGA system;

storing an original image read out from the photosensitive surface by an image reading circuit of the FPGA system in an image buffer; after receiving the original image, the FPGA system performs the following processing in real time:

s1, dividing regions according to configured grid coordinate parameters, and calculating array centroid coordinates including large-region grid coordinates of a control optical axis and a centroid coordinate array of a two-dimensional grid region for wave front reconstruction;

s2, adding the centroid coordinates extracted from the large area into PID control calculation, and sending the centroid coordinates out through an RS422 port to control the stability of an optical axis;

s3, packaging the centroid coordinate array extracted from the two-dimensional grid area and the original image, and transmitting the centroid coordinate array and the original image to a GPU + ARM core board through a PCIEx4 port;

after receiving the data, the GPU + ARM core board performs the following processing:

s4, in the GPU processor, using the centroid coordinate and the grid coordinate configuration parameters to reconstruct the wave front, and using the configured response matrix to calculate the voltage; in a GPU processor, deep learning calculation is carried out by using an original image, a wave front mode coefficient is trained, a wave front is reconstructed, and voltage is solved by using a configured response matrix;

s5, comparing the calculated wave front with the voltage in an ARM processor, preferably performing PID control calculation to obtain a final control voltage, and sending the control voltage out of a control two-dimensional deformable mirror through a gigabit network port to control the quality of the light beam; in the ARM processor, the wave front, the control voltage and other states are displayed through an HDMI port; thus, light stability control and beam quality control are achieved.

A method of operation of an industrial smart camera, including an industrial smart camera as described above, comprising the steps of:

after an original image from a photosensitive surface to an image preprocessing system, an FPGA of the image preprocessing system does not perform any calculation, and only the original image is sent to a GPU + ARM core board through a PCIEx4 port;

in the GPU + ARM core board, a GPU processor learns the original image by using an algorithm, trains and identifies various target characteristics;

and step three, the ARM processor sends the identified target result to a display through an HDMI interface for displaying, and simultaneously sends the identification structure out through an RS422 port and a CAN port to control different industrial targets.

The beneficial effects of the invention include:

the invention provides an industrial intelligent camera special for adaptive optics and a working method thereof, and fills up the industrial blank. The high-speed image sensing and adaptive optical technology are highly integrated into an intelligent camera whole, and the intelligent camera is a universal industrial measurement and control application platform from sensing to control, and a very efficient industrial digital image sensing technical scheme is provided.

The embodiment of the invention is suitable for optical axis control and beam quality control of the adaptive optical system. Image preprocessing and real-time optical axis control are realized in an FPGA system; in a GPU + ARM system, the GPU realizes image wavefront extraction and image identification; the ARM controls the light beam quality, and performs grid configuration on the FPGA system to realize management functions such as input and output display, data storage and the like. Meanwhile, because an open Linux system is embedded in the ARM and all control algorithms are opened to users, the invention can realize some intelligent industrial control based on image recognition, such as face recognition, process control and the like, besides adaptive optical control as long as different user codes are embedded.

The embodiment of the invention integrates the front-end FPGA preprocessing and optical axis control system and the rear-end light beam quality control system into a whole camera, thereby realizing the integrated camera system with image acquisition, image recognition and high-speed real-time control.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic diagram of the circuitry of an industrial smart camera;

FIG. 2 is a perspective design view of an industrial smart camera;

fig. 3 is a three-dimensional design diagram of an industrial intelligent camera with an aperture lens.

Detailed Description

The invention is further described below with reference to the following figures and examples. All features disclosed in all embodiments in this specification, or all methods or process steps implicitly disclosed, may be combined and/or expanded, or substituted, in any way, except for mutually exclusive features and/or steps.

The invention aims to solve the problem that an intelligent camera realizes light beam quality control and light stability control, and provides an industrial intelligent camera and a working method thereof. In a specific embodiment, a circuit system of the industrial smart camera is shown in fig. 1, and fig. 2 is a perspective design diagram thereof.

In the embodiment, the three systems can be integrated into one camera in a circuit structure by adopting a mode of three systems of near field image acquisition, FPGA image preprocessing and GPU + ARM image recognition and control algorithm. The method comprises the following steps: the original image acquisition system (comprises a photosurface and a photosurface substrate). An FPGA system (comprising an FPGA processor, a differential parallel port image reading circuit, an optical axis control RS422 port driving circuit, a network port driving circuit, a high-speed driving optical fiber interface circuit and a PCIEx4 transmission interface drive). GPU + ARM system (including GPU + ARM core module, circuitry interface substrate, solid state drive, net gape drive circuit, USB and HDMI interface drive, CAN and RS422 interface drive), the detail is as follows:

the original image acquisition system is an input information source system of an original image and consists of a smooth surface and a photosensitive surface substrate.

The photosensitive surface can adopt a processing chip NOIP1FN5000A, the resolution is 2592x2048, and the full frame frequency reaches 85HZ.

The photosensitive surface substrate adopts a high-speed interface board based on a differential conversion circuit to transmit photosensitive electric signals in a parallel mode.

The FPGA system is used as a preprocessing system of the front end, and meanwhile PID control of the optical axis is achieved. The hardware comprises an FPGA processor, a differential parallel port image reading circuit, an optical axis control RS422 port driving circuit, a network port driving circuit, a high-speed driving optical fiber interface circuit and a PCIEx4 transmission interface drive.

The chip of the FPGA processor can adopt XC7K325T or other models with equivalent functions.

The parallel port image reading circuit is used for storing the differential signal transmitted by the front-end image acquisition system in an original image buffer in the FPGA in real time.

The optical axis control RS422 port drive circuit, FPGA extracts the barycentric coordinate through the image, adds the control algorithm such as PID to the barycentric coordinate, calculates the optical axis adjustment quantity, then outputs through the RS422 port to achieve the purpose of controlling the optical axis.

The network port driving circuit and the high-speed driving optical fiber interface circuit are used for controlling some function switches influencing targets, such as a switch attenuator, a voice coil motor and the like.

And the PCIEx4 transmission interface driver is used for packaging and transmitting the centroid coordinate array and the original image preprocessed by the FPGA to the PCIEx4 interface.

The GPU + ARM system is a core algorithm processing center of the embodiment of the invention and comprises a GPU + ARM core module, a circuit system interface substrate, a solid state disk, a network port driving circuit, a USB and HDMI interface driver, and a CAN and RS422 interface driver.

The GPU and the ARM are integrated in a whole chip, the circuit board is based on an embedded GPU and ARM chip, and a Jetson Xavier NX module is adopted.

The circuit system interface substrate bears conversion and driving of various interfaces, and comprises a PCIEx4 interface from the FPGA system to the GPU + ARM core board, a PCIE interface from the GPU + ARM core board to the solid state disk, and interface driving of an external network port, a USB, an HDMI, a CAN and an RS422 of the GPU + ARM core board.

A solid state disk for installing an embedded linux operating system, and to install a user program APP, data file storage, and the like, wherein the hard disk is larger than 64GB.

And the network port driving circuit outputs the control quantity calculated by the GPU + ARM through a complex algorithm through a network port so as to control the deformable mirror and control the quality of the light beam. The network port is a high-speed gigabit network port.

USB and HDMI interface drive, the USB interface provides two kinds, minUSB and USB-Type-C, supplies external data input or output to use. The HDMI provides a display interface for the outside, and is a digital high-definition interface for monitoring.

The CAN and RS422 interface drives are used for providing a multifunctional output interface channel for the outside, such as optical axis control output (another channel for optical axis control) based on the RS422 interface, camera cascade control based on the CAN and the like.

In a specific embodiment, when implemented as an operating method for optical axis stabilization control and beam quality control in adaptive optics, the method comprises the following steps:

and starting a camera system, starting a linux system in the GPU + ARM core module, and starting APP software developed by a user. The APP receives grid coordinate configuration parameters sent by the upper computer and configuration parameters related to the optical system through the network port, and simultaneously the APP sends the grid configuration parameters to the FPGA system through the PCIEx4 interface for preprocessing of the FPGA system.

The original image read out from the photosensitive surface by the reading circuit of the FPGA system is stored in an image buffer. After receiving the original image, the FPGA system performs the following three tasks in real time:

1) And dividing regions according to the configured grid coordinate parameters, and calculating the array centroid coordinates including the large-region grid coordinates of the control optical axis and the centroid coordinate array of the two-dimensional grid region for reconstructing the wavefront.

2) And adding the centroid coordinates extracted from the large area into PID control calculation, and sending out through an RS422 port to control the stability of the optical axis.

3) And packaging the centroid coordinate array extracted from the two-dimensional grid area and the original image, and transmitting the packaged centroid coordinate array and the original image to the GPU + ARM core module through a PCIEx4 port.

After receiving the data, the GPU + ARM core module performs the following processing, including the steps:

1) In the GPU, wave fronts are reconstructed by using the column centroid coordinates and a configuration matrix issued in front, and the voltage is solved by using the configured response matrix.

2) Meanwhile, in the GPU, deep learning calculation is carried out by using an original image, a mode coefficient of the wavefront is trained and tested, the wavefront is reconstructed, and the voltage is solved by using a configured response matrix.

3) In ARM, comparing the wave front and the voltage resolved in 1) and 2), preferably using one of them to make PID control calculation to obtain the final control voltage, and sending the control voltage out through kilomega network port to control the two-dimensional deformable mirror to control the light beam quality.

4) Meanwhile, in the ARM, the wave front, the control voltage and other states are displayed through the HDMI. Thus, light stability control and beam quality control are achieved.

In a specific embodiment, the invention is used as an intelligent recognition system, and an aperture lens is added at the front end, so as to form a single-product solution integrating detection, acquisition, deep learning algorithm inference and target recognition, as shown in fig. 3, when the intelligent recognition system is implemented as an intelligent recognition working method, the intelligent recognition system comprises the following steps:

the target recognition uses a far-field image, after the far-field image is transmitted to an original image of the FPGA system from the photosurface, the FPGA does not perform any calculation, and only transmits the original image to the GPU + ARM core module through a PCIEx4 port.

In a GPU + ARM core module, the GPU performs algorithms such as deep learning and strength learning on an original image, trains and identifies various target characteristics.

ARM sends the target result of discernment to the display through the HDMI interface and shows, passes through RS422 mouth with the identification structure simultaneously, and CAN mouth is sent out in order to control different industry targets.

Example 1: an industrial intelligent camera comprises an image acquisition system, an image preprocessing system and a core processing system; the image acquisition system is used for transmitting the acquired images to the image preprocessing system; the image preprocessing system is provided with an image light spot mass center coordinate extracting module for transmitting the extracted image light spot mass center coordinate and the original image to the core processing system, and is also provided with an optical axis control module for calculating an optical axis adjusting quantity and then controlling an optical axis through control output; the core processing system is of a dual-processor architecture, wherein the first processor is used for image wavefront extraction and image identification, and the second processor is used for performing light beam quality control and configuring the image preprocessing system; the image acquisition system, the image preprocessing system and the core processing system are all integrated in one camera.

Example 2: on the basis of the embodiment 1, the system comprises a far-field image acquisition system, wherein the far-field image acquisition system is provided with an aperture lens, and the aperture lens is used for acquiring a target recognition far-field image; for the original image transmitted from the far-field image acquisition system to the image preprocessing system, the image preprocessing system does not perform any calculation, and only transmits the original image to the core processing system.

Example 3: on the basis of embodiment 1, the near-field image acquisition system includes a photosensitive surface and an interface circuit substrate, the photosensitive surface is connected with the interface circuit substrate, and the interface circuit substrate includes a high-speed interface board based on a differential conversion circuit, and is used for transmitting photosensitive electrical signals in a parallel manner.

Example 4: on the basis of the embodiment 1, the second processor is embedded with an open Linux system, and control algorithms are all opened to users, and the users can realize different industrial control capabilities based on image recognition by embedding different user codes.

Example 5: on the basis of the embodiment 1, the image preprocessing system comprises an image reading circuit, an FPGA processor, an optical axis control RS422 port driving circuit and PCIEx4 transmission interface driving;

the image reading circuit is used for storing the differential signals transmitted by the image acquisition system into an original image buffer in the FPGA processor in real time and comprises a differential parallel port image reading circuit;

the FPGA processor extracts a centroid coordinate through an image, adds a control algorithm to the centroid coordinate, calculates an optical axis adjustment amount by using the control algorithm, and then controls an RS422 port driving circuit to drive an RS422 port to output through an optical axis so as to control the optical axis;

and the PCIEx4 transmission interface driver is used for packaging and transmitting the centroid coordinate array and the original image preprocessed by the FPGA processor to the PCIEx4 interface.

Example 6: on the basis of the embodiment 1, the core processing system comprises a circuit system interface substrate, a solid state disk and a first network interface driving circuit, wherein a first processor is a GPU (graphics processing Unit) processor, a second processor is an ARM (advanced RISC machine) processor, and the GPU processor and the ARM processor are integrated into a whole chip to form a GPU + ARM core board;

the circuit system interface substrate is used for converting and driving various interfaces, and the various interfaces comprise a PCIEx4 interface from the FPGA system to the GPU + ARM core board, a PCIE interface from the GPU + ARM core board to the solid state disk, and an external network port, a USB interface, an HDMI interface, a CAN interface and an RS422 interface of the GPU + ARM core board;

the solid state disk is used for installing an embedded linux operating system and installing a user program APP and data file storage;

the first network port driving circuit is used for outputting the control quantity calculated by the GPU + ARM core board through a control algorithm through a network port so as to control the deformable mirror and control the quality of the light beam; the network port comprises a high-speed gigabit network port.

Example 7: on the basis of the implementation 5, the image preprocessing system comprises a second network port driving circuit and a high-speed driving optical fiber interface circuit, which are both used for controlling a function switch.

Example 8: on the basis of embodiment 6, the core processing system includes a USB and HDMI interface driving circuit, a CAN and RS422 interface driving circuit;

the USB and HDMI interface driving circuit is used for driving the USB and HDMI interfaces; the USB interface comprises a MinUSB interface and a USB-Type-C interface which are used for inputting or outputting external data; the HDMI provides a display interface for the outside, and is a digital high-definition interface for monitoring;

the CAN and RS422 interface driver is used for driving the CAN interface and the RS422 interface and providing a multifunctional output interface channel for the outside; the multifunctional output interface channel comprises an optical axis control output channel based on an RS422 interface and a camera cascade control channel based on a CAN.

Example 9: an operating method of an industrial intelligent camera comprises the following steps:

after the industrial intelligent camera in any one of the embodiment 1, the embodiment 3 to the embodiment 8 is started, starting a linux system in a GPU + ARM core module, and starting a user APP; a user APP receives grid coordinate configuration parameters sent by an upper computer and configuration parameters related to an optical system through a network port, and simultaneously the APP sends the grid configuration parameters to the FPGA system through a PCIEx4 interface for preprocessing of the FPGA system;

storing an original image read out from the photosensitive surface by an image reading circuit of the FPGA system in an image buffer; after receiving the original image, the FPGA system performs the following processing in real time:

s1, dividing regions according to configured grid coordinate parameters, and calculating array centroid coordinates including large-region grid coordinates of a control optical axis and a centroid coordinate array of a two-dimensional grid region for reconstructing a wavefront;

s2, adding the centroid coordinates extracted from the large area into PID control calculation, and sending out the centroid coordinates through an RS422 port to control the stability of an optical axis;

s3, packaging the centroid coordinate array extracted from the two-dimensional grid area and the original image and transmitting the centroid coordinate array and the original image to the GPU + ARM core board through a PCIEx4 port;

after receiving the data, the GPU + ARM core board performs the following processing:

s4, in the GPU processor, using the centroid coordinate and the grid coordinate configuration parameters to reconstruct the wave front, and using the configured response matrix to calculate the voltage; in a GPU processor, deep learning calculation is carried out by using an original image, a wave front mode coefficient is trained, a wave front is reconstructed, and voltage is solved by using a configured response matrix;

s5, comparing the calculated wavefront with the voltage in an ARM processor, preferentially calculating by using one PID control to obtain a final control voltage, and sending the control voltage out of a control two-dimensional deformable mirror through a gigabit network port to control the quality of a light beam; in the ARM processor, displaying the wave front, the control voltage and other states through an HDMI port; thus, light stability control and beam quality control are achieved.

Example 10: an operating method of an industrial smart camera including the industrial smart camera of embodiment 2, comprising the steps of:

after an original image from a photosensitive surface to an image preprocessing system is processed, an FPGA of the image preprocessing system does not perform any calculation, and the original image is only sent to a GPU + ARM core board through a PCIEx4 port;

in the GPU + ARM core board, a GPU processor learns the original image by using an algorithm, trains and identifies various target characteristics;

and step three, the ARM processor sends the identified target result to a display through an HDMI interface for displaying, and simultaneously sends the identification structure out through an RS422 port and a CAN port to control different industrial targets.

The parts not involved in the present invention are the same as or can be implemented using the prior art.

Other embodiments than the above examples may be devised by those skilled in the art based on the foregoing disclosure, or by adapting and using knowledge or techniques of the relevant art, and features of various embodiments may be interchanged or substituted and such modifications and variations that may be made by those skilled in the art without departing from the spirit and scope of the present invention are intended to be within the scope of the following claims.

Claims (10)

1. An industrial intelligent camera is characterized by comprising an image acquisition system, an image preprocessing system and a core processing system; the image acquisition system is used for transmitting the acquired image to the image preprocessing system; the image preprocessing system is provided with an image light spot mass center coordinate extracting module for transmitting the extracted image light spot mass center coordinate and the original image to the core processing system, and is also provided with an optical axis control module for calculating an optical axis adjusting quantity and then controlling an optical axis through control output; the core processing system is of a dual-processor architecture, wherein the first processor is used for image wavefront extraction and image identification, and the second processor is used for performing light beam quality control and configuring the image preprocessing system; the image acquisition system, the image preprocessing system and the core processing system are all integrated in one camera.

2. The industrial intelligent camera as claimed in claim 1, which comprises a far-field image acquisition system, wherein the far-field image acquisition system is provided with an aperture lens, and an object identification far-field image is acquired by using the aperture lens; for the original image transmitted from the far-field image acquisition system to the image preprocessing system, the image preprocessing system does not perform any calculation, and only transmits the original image to the core processing system.

3. The industrial smart camera as claimed in claim 1 wherein the near field image capture system comprises a photosensitive surface and an interface circuit substrate, the photosensitive surface is connected with the interface circuit substrate, the interface circuit substrate comprises a high speed interface board based on a differential conversion circuit for transmitting photosensitive electrical signals in parallel.

4. The industrial smart camera as claimed in claim 1, wherein the second processor is embedded with an open Linux system and opens all control algorithms to users, and users can realize different industrial control capabilities based on image recognition by embedding different user codes.

5. The industrial intelligent camera as claimed in claim 1, wherein the image preprocessing system comprises an image readout circuit, an FPGA processor, an optical axis control RS422 port driving circuit and a pcie x4 transmission interface drive;

the image reading circuit is used for transferring the differential signal transmitted by the image acquisition system into an original image buffer in the FPGA processor in real time, and comprises a differential parallel port image reading circuit;

the FPGA processor extracts a centroid coordinate through the image, adds a control algorithm to the centroid coordinate, calculates an optical axis adjustment amount by using the control algorithm, and then controls an RS422 port driving circuit to drive an RS422 port to output through an optical axis so as to control the optical axis;

and the PCIEx4 transmission interface driver is used for packaging and transmitting the centroid coordinate array and the original image preprocessed by the FPGA processor to the PCIEx4 interface.

6. The industrial smart camera as claimed in claim 1, wherein the core processing system comprises a circuitry interface substrate, a solid state disk and a first network interface driving circuit, the first processor is a GPU processor, the second processor is an ARM processor, and the GPU processor and the ARM processor are integrated into a single chip to form a GPU + ARM core board;

the circuit system interface substrate is used for converting and driving various interfaces, and the various interfaces comprise a PCIEx4 interface from the FPGA system to the GPU + ARM core board, a PCIE interface from the GPU + ARM core board to the solid state disk, and an external network port, a USB interface, an HDMI interface, a CAN interface and an RS422 interface of the GPU + ARM core board;

the solid state disk is used for installing an embedded linux operating system and installing a user program APP and data file storage;

the first network port driving circuit is used for outputting the control quantity calculated by the GPU + ARM core board through a control algorithm through a network port so as to control the deformable mirror and control the quality of the light beam; the network port comprises a high-speed gigabit network port.

7. The industrial smart camera as claimed in claim 5, wherein the image preprocessing system comprises a second portal driver circuit and a high speed drive fiber interface circuit, both for controlling the function switch.

8. The industrial smart camera of claim 6, wherein the core processing system comprises USB and HDMI interface driver circuits, CAN and RS422 interface driver circuits;

the USB and HDMI interface driving circuit is used for driving the USB and HDMI interfaces; the USB interface comprises a MinUSB interface and a USB-Type-C interface which are used for inputting or outputting external data; the HDMI provides a display interface for the outside, and is a digital high-definition interface for monitoring;

the CAN and RS422 interface driver is used for driving the CAN interface and the RS422 interface and providing a multifunctional output interface channel for the outside; the multifunctional output interface channel comprises an optical axis control output channel based on an RS422 interface and a camera cascade control channel based on a CAN.

9. An operating method of an industrial intelligent camera is characterized by comprising the following steps:

after the industrial intelligent camera as claimed in any one of claims 1 and 3 to 8 is started, starting a linux system in a GPU + ARM core module, and starting a user APP; a user APP receives grid coordinate configuration parameters sent by an upper computer and configuration parameters related to an optical system through a network port, and simultaneously the APP sends the grid configuration parameters to the FPGA system through a PCIEx4 interface for preprocessing of the FPGA system;

storing an original image read out from the photosensitive surface by an image reading circuit of the FPGA system in an image buffer; after receiving the original image, the FPGA system performs the following processing in real time:

s1, dividing regions according to configured grid coordinate parameters, and calculating array centroid coordinates including large-region grid coordinates of a control optical axis and a centroid coordinate array of a two-dimensional grid region for reconstructing a wavefront;

s2, adding the centroid coordinates extracted from the large area into PID control calculation, and sending out the centroid coordinates through an RS422 port to control the stability of an optical axis;

s3, packaging the centroid coordinate array extracted from the two-dimensional grid area and the original image, and transmitting the centroid coordinate array and the original image to a GPU + ARM core board through a PCIEx4 port;

after receiving the data, the GPU + ARM core board performs the following processing:

s4, in the GPU processor, using the centroid coordinate and the grid coordinate configuration parameters to reconstruct the wave front, and using the configured response matrix to calculate the voltage; in a GPU processor, deep learning calculation is carried out by using an original image, a wave front mode coefficient is trained, a wave front is reconstructed, and voltage is solved by using a configured response matrix;

s5, comparing the calculated wave front with the voltage in an ARM processor, preferably performing PID control calculation to obtain a final control voltage, and sending the control voltage out of a control two-dimensional deformable mirror through a gigabit network port to control the quality of the light beam; in the ARM processor, the wave front, the control voltage and other states are displayed through an HDMI port; thus, light stability control and beam quality control are achieved.

10. A method of operating an industrial smart camera, comprising the industrial smart camera of claim 2, comprising the steps of:

after an original image from a photosensitive surface to an image preprocessing system is processed, an FPGA of the image preprocessing system does not perform any calculation, and the original image is only sent to a GPU + ARM core board through a PCIEx4 port;

in the GPU + ARM core board, a GPU processor learns the original image by using an algorithm, trains and identifies various target characteristics;

and thirdly, the ARM processor sends the recognized target result to a display through an HDMI interface for displaying, and meanwhile, the recognition structure is sent out through an RS422 port and a CAN port so as to control different industrial targets.

Images