CN113163081A - Industrial camera - Google Patents
Industrial camera Download PDFInfo
- Publication number
- CN113163081A CN113163081A CN202110271317.9A CN202110271317A CN113163081A CN 113163081 A CN113163081 A CN 113163081A CN 202110271317 A CN202110271317 A CN 202110271317A CN 113163081 A CN113163081 A CN 113163081A
- Authority
- CN
- China
- Prior art keywords
- neural network
- image data
- target image
- industrial camera
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 claims abstract description 109
- 238000003062 neural network model Methods 0.000 claims abstract description 88
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 56
- 238000013528 artificial neural network Methods 0.000 claims abstract description 46
- 230000015654 memory Effects 0.000 claims abstract description 35
- 230000007547 defect Effects 0.000 claims description 41
- 239000000872 buffer Substances 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 18
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 3
- 230000002950 deficient Effects 0.000 claims description 2
- 230000002093 peripheral effect Effects 0.000 claims description 2
- 238000001514 detection method Methods 0.000 abstract description 13
- 230000009471 action Effects 0.000 description 7
- 239000011521 glass Substances 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 239000007788 liquid Substances 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000000034 method Methods 0.000 description 4
- 238000004140 cleaning Methods 0.000 description 3
- 239000000428 dust Substances 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000005507 spraying Methods 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 102100026816 DNA-dependent metalloprotease SPRTN Human genes 0.000 description 1
- 101710175461 DNA-dependent metalloprotease SPRTN Proteins 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000003749 cleanliness Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the application discloses an industrial camera, which can realize that one side of the industrial camera has logic judgment capability and target detection capability, and further meets the real-time processing requirement of an application scene on image data. The industrial camera of the present application includes: the system comprises an image sensor, a programmable gate array, a first memory and an artificial intelligence chip; the image sensor transmits the acquired image data to the programmable gate array; the programmable gate array is used for carrying out preset processing on the image data to obtain target image data and sending the target image data to the artificial intelligent chip; a first memory for storing a neural network model; the artificial intelligence chip comprises a neural network processor, and is used for sending a processing instruction to the neural network processor when target image data are received, so that the neural network processor calls a pre-stored neural network model according to the processing instruction to identify the target image data, and sends an identification result and/or the target image data to the external equipment.
Description
Technical Field
The embodiment of the application relates to the technical field of intelligent cameras, in particular to an industrial camera.
Background
Industrial cameras are a key component in machine vision systems, and the essential function of industrial cameras is to convert the collected light signals into ordered electrical signals. The selection of an appropriate industrial camera is an important link in the design of a machine vision system, and the selection of the industrial camera directly determines the resolution, the image quality and the like of the acquired image.
An industrial camera in the prior art generally only includes an image acquisition module and an image processing unit, where the image acquisition module is configured to acquire image data, and the image processing unit is configured to pre-process the image data acquired by the image acquisition module, so that the industrial camera obtains target image data meeting preset requirements. When a large-scale repeated production and detection scene is faced, the industrial camera in the prior art can only provide the image data acquisition and processing functions of the scene, and needs to be connected with other external equipment in a matching manner to analyze and logically judge the image data, and the transmission of the image data between the external equipment and the industrial camera needs to take a certain transmission time, so that the real-time processing requirement of the application scene on the image data cannot be met.
Disclosure of Invention
The embodiment of the application provides an industrial camera, which can realize that one side of the industrial camera has logic judgment capability and target detection capability, and further meets the real-time processing requirement of an application scene on image data.
An industrial camera of the present application includes: the system comprises an image sensor, a programmable gate array, a first memory and an artificial intelligence chip;
the image sensor is in electric signal connection with the programmable gate array and transmits acquired image data to the programmable gate array;
the programmable gate array is in electric signal connection with the artificial intelligence chip and is used for performing preset processing on the image data to obtain target image data and sending the target image data to the artificial intelligence chip;
the first memory is used for storing a neural network model;
the artificial intelligence chip comprises a neural network processor, and is used for sending a processing instruction to the neural network processor when the target image data is received, so that the neural network processor calls a pre-stored neural network model according to the processing instruction to identify the target image data, and sends an identification result and/or the target image data to an external device.
Optionally, the first memory stores a plurality of neural network models; the different types of neural network models correspond to different processing scenes, and the different types of neural network models obtain different recognition results.
Optionally, the system further comprises an input/output interface;
the input/output interface is used for receiving the processing instruction, wherein the processing instruction carries a processing scene identifier and sends the processing instruction to the artificial intelligence chip;
the artificial intelligence chip is specifically configured to send the processing instruction to the neural network processor, so that the neural network processor invokes a neural network model corresponding to the processing scene identifier to identify the target image data.
Optionally, the input/output interface includes a high-speed network interface;
the high-speed network interface is arranged on the programmable gate array and/or the artificial intelligence chip and is used for communicating with the external equipment.
Optionally, the external device includes: a neural network model training server, wherein the neural network model training server stores the neural network model in advance;
the high-speed network interface is also used for receiving a new neural network model and sending the new neural network model to the artificial intelligence chip, the new neural network model calls and receives the recognition result and the target image data uploaded by the industrial camera for analysis and labeling for the neural network training server to obtain labeled target image data, and then the labeled target image data is used for continuously training the neural network model to form a new neural network model;
the artificial intelligence chip is also used for updating the new neural network model to the first memory.
Optionally, the processing scenario includes: identifying a first processing scenario of a defective object;
the neural network processor is specifically configured to invoke a neural network model corresponding to the first processing scene to identify whether the target image data includes the defect object, obtain an identification result including the defect object or an identification result not including the defect object, and send the identification result to the external device.
Optionally, the processing scenario includes: a second processing scenario for defect location identification;
the neural network processor is specifically configured to invoke a neural network model corresponding to the second processing scenario to identify a defect position of a defect included in the target image data, obtain an identification result of the defect position, and send the target image data and the identification result to the external device.
Optionally, the processing scenario includes: a third processing scenario for defect type identification;
the neural network processor is specifically configured to invoke a neural network model corresponding to the third processing scenario to identify a defect type included in the target image data, so as to obtain an identification result of the defect type.
Optionally, the programmable gate array is further configured to send the target image data and/or the recognition result to the external device when a call request of the external device is received or when the confidence of the recognition result is lower than a preset threshold; or the like, or, alternatively,
the artificial intelligence chip is further used for sending the target image data and/or the recognition result to the external equipment when a calling request of the external equipment is received or when the confidence degree of the recognition result is lower than a preset threshold value.
Optionally, the method further includes: a first buffer and/or a second buffer;
the first buffer is connected with the artificial intelligence chip;
the second buffer is connected with the programmable gate array.
According to the technical scheme, the embodiment of the application has the following advantages:
the industrial camera provided by the embodiment of the application is provided with an image sensor, a programmable gate array, a first memory and an artificial intelligence chip; the image sensor transmits acquired image data to the programmable gate array, the programmable gate array is used for conducting preset processing on the image data to obtain target image data, then the target image data are sent to the artificial intelligence chip, the artificial intelligence chip is provided with the neural network processor, when the artificial intelligence chip receives the target image data, a processing instruction is sent to the neural network processor, the neural network processor enables the neural network processor to call the neural network model pre-stored in the first storage to identify the target image data according to the processing instruction, and an identification result and/or the target image data are sent to the external equipment. Therefore, an artificial intelligent chip is added in the industrial camera, and a neural network processor in the artificial intelligent chip can call a neural network model pre-stored in a first memory to perform logic judgment or detection on target image data, so that the logic judgment capability and the target detection capability of one side of the industrial camera are realized, and the industrial camera can meet the real-time processing requirement of an application scene on the image data.
Drawings
FIG. 1 is a schematic structural diagram of an embodiment of an industrial camera according to the present application;
fig. 2 is a schematic structural diagram of another embodiment of an industrial camera according to the present application.
Detailed Description
The embodiment of the application provides an industrial camera, which can realize that one side of the industrial camera has logic judgment capability and target detection capability, and further meets the real-time processing requirement of an application scene on image data.
An image sensor (image sensor) in the embodiments of the present application, also called a photosensitive element, an image capturing module, etc., refers to a device that converts an optical image into an electronic signal, and is widely used in cameras and other electronic optical devices for capturing image data. The field-programmable gate array (FPGA) of the embodiment of the present application refers to a program-driven logic device, a control program of which is stored in a memory connected to the FPGA, and after being powered on, the program is automatically loaded to a chip for execution, and is used for collecting image data and preprocessing an image, and can transmit the image data and a control command with an artificial intelligence chip. The programmable gate array of the present application includes, but is not limited to, Xilinx Virtex series, Spartan series, Virtex series, Kintex series, or Intel corporation Cyclone series, Stratix series, Arria series, etc. In practical application, the industrial camera in the embodiment of the application can be selected according to self requirements and comprehensive consideration of logic quantity, power consumption, cost and the like of different series of programmable gate arrays. The Artificial Intelligence chip in the embodiment of the present application refers to an Artificial Intelligence chip, which at least should be integrated with a neural network processor, such as a 5AIoT chip published by yunnan inspirational flying company tmeepeye 1000, haisi Hi3559 of haisi corporation, Myriad X chip of Intel corporation, TX2 chip of Nvidia corporation, and the like, and different Artificial Intelligence chips have differences in computational power, power consumption, visual reasoning capability, and the like.
Referring to fig. 1, an embodiment of an industrial camera according to the present application includes: an image sensor 110, a programmable gate array 120, a first memory 130, and an artificial intelligence chip 140. The image sensor 110 is electrically connected with the programmable gate array 120, and the image sensor 110 transmits the acquired image data to the programmable gate array 120; the programmable gate array 120 is electrically connected to the artificial intelligence chip 140, and the programmable gate array 120 is configured to perform preset processing on image data received from the image sensor 110 to obtain target image data, and send the target image data to the artificial intelligence chip 140, where the preset processing refers to processing such as image color correction, image shape distortion correction, image exposure compensation on the image data, so as to obtain relatively real target image data and reduce the influence on the formed image data due to defects of the image sensor 110; the first memory 130 of the industrial camera according to the embodiment of the present disclosure stores a neural network model, where the neural network model is a pre-trained neural network model with a specific function; the artificial intelligence chip 140 includes a neural network processor 141, and when the artificial intelligence chip 140 receives target image data sent by the programmable gate array 120, a processing instruction is sent to the neural network processor 141, so that the neural network processor 141 calls a neural network model pre-stored in the first memory 130 according to the processing instruction to perform processing such as identification on the target image data, and sends an identification result and/or the target image data to an external device, where the external device generally refers to a device for performing action feedback on an application scene of image data currently formed by an industrial camera in time. Therefore, the artificial intelligent chip 140 is added in the industrial camera according to the embodiment of the application, and the neural network processor 141 in the artificial intelligent chip 140 can call the neural network model pre-stored in the first memory 130 to perform logic judgment on the target image data, so that the logic judgment capability and the target detection capability of one side of the industrial camera are realized, and the industrial camera with the logic judgment capability and the target detection capability can meet the real-time processing requirement of an application scene on the image data.
Further, the first memory 130 of the embodiment of the present application may store a plurality of neural network models, where different types of neural network models correspond to different processing scenarios, and different types of neural network models obtain different recognition results. For example, the first memory 130 may store therein a neural network model of a deep learning classification network: one or more of a VGG network (visual geometry group), a residual network (net), and a mobile net. The memory 130 may also store a neural network model that enables target detection: one or more of a Yolo (young only look) series model, RFCN (Region-based full volumetric network), SSD (single shot multi-box detector), and the like. In practical application, different neural network models can be stored in the memory of the industrial camera according to different scene requirements so as to deal with different processing scenes, and generally different types of neural network models have different recognition results.
Further, the industrial camera according to the embodiment of the present application may further include an input/output interface, which may be the second input/output interface 121 disposed on the programmable gate array 120, and/or the first input/output interface 142 on the artificial intelligence chip 140; the input/output interface may be configured to receive a processing instruction, where the processing instruction carries a processing scene identifier, and send the processing instruction to the artificial intelligence chip 140. The artificial intelligence chip 140 is specifically configured to send a processing instruction to the neural network processor 141, so that the neural network processor 141 invokes a neural network model corresponding to the processing scene identifier to identify the target image data. For example, the input/output interface may include a toggle switch of a physical switch, and when the toggle switch is toggled to a different preset position, a processing instruction carrying a different processing scene identifier is invoked, and the processing instruction causes the neural network processor 141 to invoke a neural network model corresponding to the processing scene identifier to recognize the target image data. It can be understood that the input/output interface may further include a high-speed network interface, for example, the high-speed network interface may be a Universal Serial Bus (USB), an optical fiber interface, a wireless communication module, and the like, and the industrial camera according to the embodiment of the present application receives a processing instruction carrying different processing scene identifiers through the high-speed network interface, where the processing instruction causes the neural network processor 141 to invoke a neural network model corresponding to the processing scene identifiers to identify the target image data. It should be noted that, one embodiment of the communication between the high-speed network interface and the external device in the embodiment of the present application may be: when the industrial camera sends the recognition result and/or the target image data to the external equipment through the high-speed network interface, the industrial camera can also select to send the recognition result and/or the target image data to the external equipment through the high-speed network interface. Specifically, components with a large number of high-speed network interfaces can be selected from the components of the industrial camera in the embodiment of the present application, so as to enrich the number of interfaces of the industrial camera, and facilitate the industrial camera to have more choices in dealing with various application scenarios, for example, please refer to fig. 2, at least one high-speed network interface is provided on the selected programmable gate array as the second input/output interface 121, and at least one high-speed network interface is provided on the artificial intelligence chip 140 as the first input/output interface 142, so that different high-speed network interfaces can be used in different processing scenarios to achieve the purpose of energy saving and high efficiency.
Specifically, in the process of transmitting the acquired image data to the programmable gate array 120 by the image sensor 110 of the industrial camera according to the embodiment of the present invention, the transmission may be realized through a low-voltage differential signaling (LVDS) interface, a Mobile Industry Processor Interface (MIPI), a bt.656/601 interface, a bt.1120 interface, or a dc (digital camera) interface, which interface is specifically used may be selected according to actual needs, and is not further limited herein. The data transmission between the programmable gate array 120 and the artificial intelligence chip 140 can be realized through a Serial Peripheral Interface (SPI), a Mobile Industry Processor Interface (MIPI), a general input/output (GPIO), or an interface conforming to a high-speed serial computer extended bus standard (PCIE), which interface is specifically used can be selected according to actual needs, and no further limitation is made here.
Specifically, referring to fig. 2, the industrial camera according to the embodiment of the present application may further include a first buffer 150 and a second buffer 160, where the first buffer 150 and the second buffer 160 may respectively have a plurality of buffers, where the second buffer 160 is communicatively connected to the programmable gate array 120, and the first memory 150 is communicatively connected to the artificial intelligence chip 140. The first buffer 150 is used for buffering various image data and various calculation results related to the artificial intelligence chip 140, and the second buffer 160 is used for buffering various image data and various calculation results related to the programmable gate array 120. First buffer 150 and second buffer 160 may each comprise, but are not limited to, DDR or SDRAM, and other types of buffers are also possible.
Further, in the artificial intelligence chip 140 of the industrial camera according to the embodiment of the present application, on the basis of integrating a neural-Network Processing Unit (NPU), it is preferable to further integrate modules such as a Digital Signal Processor (DSP), an acceleration operator ACC, and a video decoder, so as to assist in increasing the processing speed of the neural network processor 141 of the artificial intelligence chip 140 on the image data. The design architecture of the artificial intelligence chip 140 is selected to be relatively redundant, for example, the digital signal processor and the acceleration operator ACC belong to enhancement features, and the image data clipping function can be realized according to the application scene. Of course, artificial intelligence chips with different functional characteristics can be selected according to actual requirements.
Further, the industrial camera according to the embodiment of the present application may further have more memories attached thereto, the memory may be one or a combination of multiple types, such as a Solid State Disk (SSD), an SPI Nor Flash, an SPI NAND Flash, an eMMC, and a secure digital card (SD), and the memories of different types and capacities meet the requirements of the industrial camera for different processing scenarios. For example, referring to fig. 2, the programmable gate array 120 of the industrial camera according to the embodiment of the present application is connected to a second memory 170, and the second memory 170 is used for providing a larger memory space for the programmable gate array to meet the requirements of different processing scenarios of the industrial camera.
It is understood that the industrial camera implemented by the present application further requires a power supply module for providing a suitable operating voltage to the programmable gate array 120, the artificial intelligence chip 140, and the like.
The external device of the embodiment of the application can be an execution mechanism of a certain action, a personal computer, an edge device, a server, a cloud network and the like. The execution mechanism can make preset actions according to information such as a logic judgment result output by the industrial camera, and the personal computer, the edge device, the server, the cloud network and the like can store or further process the information such as the logic judgment result and/or the target image data output by the industrial camera. For example, when the external device is a neural network model training server, and the neural network model training server stores a neural network model as in the first memory of the industrial camera in advance, the neural network model training server may send a call request to the industrial camera, where the call request is used to call and receive the recognition result and the target image data uploaded by the industrial camera for analysis, and label the target image data according to the recognition result to obtain labeled target image data, and then may use the labeled target image data to train the corresponding neural network model continuously to obtain a new neural network model with better logic judgment capability and target detection capability, and send the new neural network model to the industrial camera of the embodiment of the present application, so that the industrial camera updates and stores the new neural network model into the first memory, and then the industrial camera can call a new neural network model for recognition and judgment. When the confidence coefficient of the recognition result of the target image data by the industrial camera using the neural network model is lower than the preset threshold, the target image data and the recognition result can also be actively uploaded to an external device (such as a neural network model training server).
Therefore, the embodiment of the application provides an intelligent industrial camera, which can realize that one side of the industrial camera has logic judgment capability and target detection capability, further meets the real-time processing requirement of an application scene on image data, can be matched with equipment such as a neural network model training server and the like, can directly send target image data which cannot be judged and detected for new defects to the neural network model training server, and the neural network model training server finishes training a new neural network model which can identify the new defects and then sends the new neural network model to the industrial camera of the embodiment of the application, thereby achieving the purpose of remotely updating the neural network model by matching a high-speed network interface with a network, realizing on-line function upgrade of the industrial camera of the embodiment of the application, and constantly improving the logic judgment capability and the target detection capability according to the actual condition of a deployment field, the industrial camera is simple in structure and low in cost, and is beneficial to solving the problem of high price of large-scale deployment of the industrial camera.
The industrial camera of the embodiment of the application can at least meet the logic judgment of the following processing scenes, and obtain the corresponding processing result.
When the industrial camera in the embodiment of the present application faces processing scenarios: when the first processing scene of the defect object is identified, the industrial camera may determine the current processing scene as a classified scene through the input interface, the image sensor 110 of the industrial camera acquires image data about the first processing scene through a lens of the industrial camera, the image sensor 110 transmits the image data to the programmable gate array 120, the programmable gate array 120 is configured to perform preset processing on the image data received from the image sensor 110 to obtain target image data, and transmit the target image data to the artificial intelligence chip 140, and when the artificial intelligence chip 140 receives the target image data transmitted by the programmable gate array 120, transmit a processing instruction to the neural network processor 141, so that the neural network processor 141 invokes a neural network model, such as a vgg (virtual geometry group) network, for image classification stored in the first memory 130 in advance, the neural network processor 141 calls a neural network model corresponding to the first processing scene to identify whether the target image data contains a defect object, and then sends the identification result to the external device. For example: when the industrial camera is used for checking the surface cleanliness of a glass plane on a production line, the external equipment is an execution mechanism for spraying cleaning liquid, the execution mechanism can perform preset actions according to the recognition result output by the industrial camera, specifically, the execution mechanism performs preset actions for starting spraying the cleaning liquid on the recognition result of one type containing a defect object (dust exists on the glass plane), and the execution mechanism performs preset actions for not starting spraying the cleaning liquid on the recognition result of the other type not containing the defect object (dust does not exist on the glass plane). It can be understood that the execution mechanism as the external device may set different actions according to different requirements of the processing scenario, and is not further limited herein. Therefore, the industrial camera in the first processing scene only needs to feed back the identification result rather than the image data without feeding back the image data to the external device, and the identification result occupies a much smaller transmission data volume than the image data, so that the industrial camera in the embodiment of the application is more suitable for large-scale deployment, and the real-time processing requirement of the application scene on the image data is met more quickly and conveniently. It can be understood that, on the basis of the first processing scenario, the industrial camera may also send the target image data and the corresponding recognition result to the external device, for example, when a call request of the external device is received or when the confidence of the recognition result is lower than a preset threshold, the neural network model training server may call the recognition result and the target image data uploaded by the industrial camera to perform analysis and labeling to obtain labeled target image data, and then perform continuous training on the corresponding neural network model using the labeled target image data and other data to obtain a new neural network model with better logic judgment capability and target detection capability, and send the new neural network model to the industrial camera of the embodiment of the present application, so that the industrial camera of the embodiment of the present application updates and stores the new neural network model into the first memory, updating of the neural network model in the first memory is effected. It is worth noting that the industrial camera in the embodiment of the application can upload the target image and/or the recognition result to the external device through the programmable gate array or the artificial intelligence chip.
When the industrial camera of the embodiment of the present application faces processing scenarios: in the second processing scenario of the defect location identification, the industrial camera may also determine, through the input interface, that the current processing scenario is a defect location identification scenario in which the defect location is detected and identified, the image sensor 110 of the industrial camera acquires image data about the second processing scenario through a lens of the industrial camera, the image sensor 110 transmits the image data to the programmable gate array 120, the programmable gate array 120 is configured to perform preset processing on the image data received from the image sensor 110 to obtain target image data, and transmit the target image data to the artificial intelligence chip 140, and when the artificial intelligence chip 140 receives the target image data transmitted by the programmable gate array 120, transmit a processing instruction to the neural network processor 141 so that the neural network processor 141 calls a neural network model that is stored in the first memory 130 in advance and can detect the defect location, for example, RFCN (Region-based full volumetric network), the neural network processor calls a neural network model corresponding to the second processing scene to identify the defect position of the defect included in the target image data, so as to obtain an identification result of the defect position, and the target image data and the identification result are sent to the external device. For example: when the industrial camera is used for identifying the pollution position of the glass plane on the production line, the external device can be a personal computer, an edge computing device or a server, and the like, specifically, a neural network model in the industrial camera identifies the coordinates of the pollution position on the glass plane in the target image data to obtain the identification result of the defect position (namely, the coordinates of the pollution position in the target image data), and then the identification result and the target image data are both sent to the external device for storage or further processing. For example, when the industrial camera receives a call request of the external device or when the confidence of the recognition result is lower than a preset threshold, the recognition result and the target image data may be used to train the neural network model, which is not described herein again.
When the industrial camera of the embodiment of the present application faces processing scenarios: in a third processing scenario of defect type identification, the industrial camera may also determine, through the input interface, that a current processing scenario is a defect type identification scenario in which a defect type is detected and identified, an image sensor 110 of the industrial camera acquires image data about the third processing scenario through a lens of the industrial camera, the image sensor 110 transmits the image data to the programmable gate array 120, the programmable gate array 120 is configured to perform preset processing on the image data received from the image sensor 110 to obtain target image data, and send the target image data to the artificial intelligence chip 140, and when the artificial intelligence chip 140 receives the target image data sent by the programmable gate array 120, send a processing instruction to the neural network processor 141, so that the neural network processor 141 calls a neural network model that is stored in the first memory 130 in advance and can classify the image, for example, the VGG network model, the neural network processor 141 calls the neural network model corresponding to the third processing scenario to identify the defect type included in the target image data, obtains an identification result of the defect type, and sends the target image data and the identification result to the external device. For example: when the industrial camera is used for identifying the pollution type of the glass plane on the production line, the external device can be a personal computer, an edge computing device or a server, and the like, specifically, the identification result that the pollution type on the glass plane is the defect type such as solid dust pollution, liquid stain pollution and the like is identified by a neural network model in the industrial camera, and the industrial camera can send target image data and the corresponding identification result to the external device. For example, when the industrial camera receives a call request of the external device or when the confidence of the recognition result is lower than a preset threshold, the recognition result and the target image data may be used to train the neural network model, which is not described herein again.
The above description of the present application with reference to specific embodiments is not intended to limit the present application to these embodiments. For those skilled in the art to which the present application pertains, several changes and substitutions may be made without departing from the spirit of the present application, and these changes and substitutions should be considered to fall within the scope of the present application.
Claims (10)
1. An industrial camera, comprising: the system comprises an image sensor, a programmable gate array, a first memory and an artificial intelligence chip;
the image sensor is in electric signal connection with the programmable gate array and transmits acquired image data to the programmable gate array;
the programmable gate array is in electric signal connection with the artificial intelligence chip and is used for performing preset processing on the image data to obtain target image data and sending the target image data to the artificial intelligence chip;
the first memory is used for storing a neural network model;
the artificial intelligence chip comprises a neural network processor, and is used for sending a processing instruction to the neural network processor when the target image data is received, so that the neural network processor calls a pre-stored neural network model according to the processing instruction to identify the target image data, and sends an identification result and/or the target image data to an external device.
2. The industrial camera of claim 1, wherein the first memory stores a plurality of neural network models; the different types of neural network models correspond to different processing scenes, and the different types of neural network models obtain different recognition results.
3. The industrial camera of claim 2, further comprising an input-output interface;
the input/output interface is used for receiving the processing instruction, wherein the processing instruction carries a processing scene identifier and sends the processing instruction to the artificial intelligence chip;
the artificial intelligence chip is specifically configured to send the processing instruction to the neural network processor, so that the neural network processor invokes a neural network model corresponding to the processing scene identifier to identify the target image data.
4. The industrial camera of claim 3, wherein the input-output interface comprises a high-speed network interface;
the high-speed network interface is arranged on the programmable gate array and/or the artificial intelligence chip and is used for communicating with the external equipment.
5. The industrial camera of claim 4, wherein the peripheral device comprises: a neural network model training server, wherein the neural network model training server stores the neural network model in advance;
the high-speed network interface is also used for receiving a new neural network model and sending the new neural network model to the artificial intelligence chip, the new neural network model calls and receives the recognition result and the target image data uploaded by the industrial camera for analysis and labeling for the neural network training server to obtain labeled target image data, and then the labeled target image data is used for continuously training the neural network model to form a new neural network model;
the artificial intelligence chip is also used for updating the new neural network model to the first memory.
6. The industrial camera of claim 4, wherein the processing scenario comprises: identifying a first processing scenario of a defective object;
the neural network processor is specifically configured to invoke a neural network model corresponding to the first processing scene to identify whether the target image data includes the defect object, obtain an identification result including the defect object or an identification result not including the defect object, and send the identification result to the external device.
7. The industrial camera of claim 4, wherein the processing scenario comprises: a second processing scenario for defect location identification;
the neural network processor is specifically configured to invoke a neural network model corresponding to the second processing scenario to identify a defect position of a defect included in the target image data, obtain an identification result of the defect position, and send the target image data and the identification result to the external device.
8. The industrial camera of claim 4, wherein the processing scenario comprises: a third processing scenario for defect type identification;
the neural network processor is specifically configured to invoke a neural network model corresponding to the third processing scenario to identify a defect type included in the target image data, obtain an identification result of the defect type, and send the target image data and the identification result to the external device.
9. The industrial camera as claimed in any one of claims 6 to 8, wherein the programmable gate array is further configured to send the target image data and/or the recognition result to the external device when a call request of the external device is received or when a confidence of the recognition result is lower than a preset threshold;
or the like, or, alternatively,
the artificial intelligence chip is further used for sending the target image data and/or the recognition result to the external equipment when a calling request of the external equipment is received or when the confidence degree of the recognition result is lower than a preset threshold value.
10. The industrial camera of claim 1, further comprising: a first buffer and/or a second buffer;
the first buffer is connected with the artificial intelligence chip;
the second buffer is connected with the programmable gate array.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110271317.9A CN113163081A (en) | 2021-03-12 | 2021-03-12 | Industrial camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110271317.9A CN113163081A (en) | 2021-03-12 | 2021-03-12 | Industrial camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113163081A true CN113163081A (en) | 2021-07-23 |
Family
ID=76887016
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110271317.9A Pending CN113163081A (en) | 2021-03-12 | 2021-03-12 | Industrial camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113163081A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN207897064U (en) * | 2017-11-07 | 2018-09-21 | 北京大恒图像视觉有限公司 | A kind of 100M/1G/2.5G/5G/10G interface adaptives industrial camera |
| CN208890933U (en) * | 2018-08-10 | 2019-05-21 | 杭州言曼科技有限公司 | Industrial camera |
| CN212391799U (en) * | 2020-06-05 | 2021-01-22 | 北京智芯微电子科技有限公司 | Power transmission line video monitoring device and system |
-
2021
- 2021-03-12 CN CN202110271317.9A patent/CN113163081A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN207897064U (en) * | 2017-11-07 | 2018-09-21 | 北京大恒图像视觉有限公司 | A kind of 100M/1G/2.5G/5G/10G interface adaptives industrial camera |
| CN208890933U (en) * | 2018-08-10 | 2019-05-21 | 杭州言曼科技有限公司 | Industrial camera |
| CN212391799U (en) * | 2020-06-05 | 2021-01-22 | 北京智芯微电子科技有限公司 | Power transmission line video monitoring device and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102494946B1 (en) | Battery detection method and device (BATTERY DETECTION METHOD AND DEVICE) | |
| JP6867555B2 (en) | Automated optical inspection system based on CPU + GPU + FPGA architecture | |
| CN110712202B (en) | Special-shaped component grabbing method, device and system, control device and storage medium | |
| US11508163B2 (en) | Method and apparatus for training lane line identifying model, device, and storage medium | |
| EP3879443A2 (en) | Method and apparatus for recognizing wearing state of safety belt, electronic device, and storage medium | |
| US20230007831A1 (en) | Method for warehouse storage-location monitoring, computer device, and non-volatile storage medium | |
| CN115699082A (en) | Defect detection method and device, storage medium and electronic equipment | |
| CN111488821A (en) | Method and device for identifying traffic signal lamp countdown information | |
| CN115641517A (en) | Machine vision defect identification method and system, edge side device and storage medium | |
| CN108270958B (en) | Camera module self-adaptive system and self-adaptive method thereof | |
| CN209156466U (en) | A kind of SSD hard disk discharging device | |
| CN111783674A (en) | Face recognition method and system based on AR glasses | |
| CN211906310U (en) | Zynq-based machine vision detection system | |
| CN113163081A (en) | Industrial camera | |
| CN104102601A (en) | FPGA (field programmable gate array) based image data acquisition and storage system | |
| CN112180353A (en) | Target object confirmation method and system and storage medium | |
| CN102866165A (en) | Surface defect detecting device of hardware | |
| CN211956492U (en) | Image processing system for visual navigation robot | |
| CN110632420B (en) | RTC (real time clock) module testing method and device and multi-module automatic testing method and device for Android equipment production | |
| CN113305840A (en) | Intelligent electric power overhaul robot visual identification system based on raspberry group | |
| CN114511753A (en) | Target detection model updating method, device, equipment and storage medium | |
| CN114143458A (en) | Image acquisition control method and device based on machine vision | |
| CN117078620B (en) | PCB welding spot defect detection method and device, electronic equipment and storage medium | |
| CN202903688U (en) | Hardware surface defect detection device | |
| CN106326862A (en) | Multi-face pickup device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210723 |
|
| RJ01 | Rejection of invention patent application after publication |