CN112415002A - Multimode sensing device based on image sensor - Google Patents
Multimode sensing device based on image sensor Download PDFInfo
- Publication number
- CN112415002A CN112415002A CN202011257142.8A CN202011257142A CN112415002A CN 112415002 A CN112415002 A CN 112415002A CN 202011257142 A CN202011257142 A CN 202011257142A CN 112415002 A CN112415002 A CN 112415002A
- Authority
- CN
- China
- Prior art keywords
- image sensor
- signal
- sensing unit
- sensing device
- optical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/75—Systems in which material is subjected to a chemical reaction, the progress or the result of the reaction being investigated
- G01N21/77—Systems in which material is subjected to a chemical reaction, the progress or the result of the reaction being investigated by observing the effect on a chemical indicator
- G01N21/78—Systems in which material is subjected to a chemical reaction, the progress or the result of the reaction being investigated by observing the effect on a chemical indicator producing a change of colour
- G01N21/783—Systems in which material is subjected to a chemical reaction, the progress or the result of the reaction being investigated by observing the effect on a chemical indicator producing a change of colour for analysing gases
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/01—Arrangements or apparatus for facilitating the optical investigation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Chemical & Material Sciences (AREA)
- Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Chemical Kinetics & Catalysis (AREA)
- Plasma & Fusion (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
- Investigating Or Analysing Materials By The Use Of Chemical Reactions (AREA)
Abstract
The invention discloses a multi-modal sensing device based on an image sensor, which comprises the following structures: the image sensor, the optical sensing unit and the lens module; the image sensor part surface is modified with optical sensing units capable of generating optical signal response for detecting sensing signals. Meanwhile, the part of the image sensor not modifying the optical sensing unit can detect external image information by combining with the lens module. The invention uses the image sensor compatible with the semiconductor process and the integrated circuit process, has small volume and high integration level, realizes the simultaneous and parallel detection of various sensing signals and image signals on the same device, and improves the acquisition capability of intelligent equipment to external information.
Description
Technical Field
The invention relates to physical sensing, chemical sensing and image sensing, and belongs to the field of intelligent sensing.
Background
In the internet of things and the 5G era, the intelligent sensor has wide application prospects in robots, health medical treatment and intelligent manufacturing. In the process of processing complex tasks, intelligent systems such as biology, robots and the like need a series of sensing systems to work cooperatively. For example, mosquitoes initially sense the presence of their surrounding host through the carbon dioxide sensor during the process of seeking a host, and this signal stimulates the mosquito to begin seeking a host and fly against a concentration gradient of carbon dioxide, which requires the cooperation of the airflow sensor. During the flight process, a visual perception system of the mosquito continuously searches for a suspected host target, after the mosquito approaches the suspected host target, the mosquito judges and confirms the host target through the joint work of temperature, humidity and a lactate sensor, and the blood sucking action is finished after the mosquito is judged to contact the skin of the host through a tactile sensor. The whole process involves signal perception in multiple dimensions. Similarly, the multi-modal sensing technology can greatly improve the perception capability of the intelligent device to the environment and objects and the capability of completing complex tasks.
Traditional intelligent equipment adopts discrete sensing devices to respectively acquire information of different dimensions. For example, the bionic robot can acquire image information by using a camera, acquire a touch signal by using a mechanical sensor on a robot hand, and detect odor by using an array formed by metal oxide and other types of gas sensors. However, the technical route using discrete devices occupies a large volume, requires many data interfaces for signal transmission, and requires a long time, so that it is difficult to meet the requirements of development of intelligent devices in the direction of miniaturization, low energy consumption, fast response, and the like. The development of the integrated sensing device capable of multi-modal perception has important application value in the field of intelligent equipment.
Due to the popularization of smart devices such as smart phones, the performance of image sensors is greatly improved while the cost is reduced in recent years. The digital technologies based on image acquisition and analysis, such as photographing, face recognition, mobile payment and the like, also realize wide application landing. At present, image sensors mainly include a photosensitive coupling element type and a complementary metal oxide semiconductor type, and each image sensor integrates a large number of photoelectric conversion units, an image signal processing system and a signal interface. This makes the image sensor very suitable for signal detection of an optical sensor array. In the conventional method, an image sensor and a lens module are used for acquiring an image outside a certain distance of an optical sensor array, but the size of the whole detection system is large, and the size of a sensing unit generally needs to be in a millimeter level to obtain a clear image.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provide a multi-mode sensing device based on an image sensor.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows: an image sensor based multimodal sensing device comprising the following structure: the image sensor, the optical sensing unit and the lens module; the image sensor part surface is modified with optical sensing units capable of generating optical signal response for detecting sensing signals. Meanwhile, the part of the image sensor not modifying the optical sensing unit can detect external image information by using the lens module.
The device directly decorates the optical sensing unit on the surface of the image sensor, and a clear image of the micron-sized sensing unit is directly obtained without focusing light rays by a lens module. This reduces the volume of the sensing unit and the detection system considerably. In addition, due to the reduction in size of the sensing unit, the image sensor has a large number of photoelectric conversion units left uncovered after the surface is decorated with an optical sensor array for physical or chemical signal detection. The photoelectric conversion units can acquire clear external image signals by matching with the lens module. The multi-mode sensor is high in integration level and multiple in information dimension, and contributes to miniaturization of intelligent equipment and improvement of perception capability.
Further, the image sensor comprises an array type photoelectric conversion unit and a signal transmission module.
Furthermore, the image sensor of the invention also comprises a signal processing module.
Furthermore, the optical sensing unit is a physical signal sensing unit and/or a chemical signal sensing unit.
Further, the physical signal sensing unit of the present invention can generate optical signal changes caused by changes including color, light intensity, shape, displacement, etc. when there is a detectable physical signal.
Further, the chemical signal sensing unit of the present invention can generate optical signal changes caused by changes including color, light intensity, shape, refractive index, fluorescence signal, etc. when a detectable chemical signal is present.
Furthermore, the multi-modal sensing device based on the image sensor further comprises a shell, the lens module is fixed on the shell, the shell is arranged on the image sensor, and one or more gas circulation structures are arranged on the shell.
Further, the multi-modal image sensor device based on image sensor of the present invention further comprises one or more light sources for providing illumination to the optical sensing unit or as an excitation light source for fluorescence signals.
Furthermore, the multi-modal sensing device based on the image sensor further comprises one or more light path control modules, and the light path control modules are used for blocking the light source and the airflow from interfering with external image information received by the image sensor.
The invention also provides a specific application device based on the multi-modal sensor device, namely a visual sense and olfaction cooperative sensing device, which comprises the following structures:
an image sensor for acquiring optical signals;
the gas sensing unit is directly decorated on the surface of the image sensor and is used for detecting a gas sensing signal generating an optical signal response; when the gas is detectable around, the gas sensing unit can generate the optical signal change which can be detected by the image sensor;
and the lens module is arranged above the surface of the image sensor and is used for acquiring an external image signal.
The visual sense and smell sense cooperative sensing device can simultaneously realize the end side detection and analysis of visual sense and smell sense information.
Compared with the prior art, the invention has the following beneficial effects:
(1) structurally, the multi-mode signal detection is realized on the same device by fixing the physical signal sensing unit and the chemical signal sensing unit which can generate optical signal response on the surface of the image sensor part. Compared with the traditional discrete device, the device has small volume and simple structure, and is more suitable for integrated application on intelligent equipment sensitive to size, such as micro robots, intelligent mobile phones and the like.
(2) The image detected by the multi-mode sensing device based on the image sensor comprises visual, physical and chemical information, and various types of signals can be analyzed simultaneously during one-time image signal processing, so that the efficiency of information acquisition and analysis of intelligent equipment is obviously improved, and the time cost and the calculation cost are reduced.
(3) The image sensor adopted in the invention is compatible with the semiconductor and integrated circuit process, so that the functions of storage and calculation can be integrated in the image sensor, the end-side detection and analysis of multi-dimensional information can be realized, and the integration level and the calculation capability of an intelligent system can be further improved.
(4) The multi-mode sensing device based on the image sensor has the advantages of simple structure, low cost, good repeatability, convenience for mass production and manufacture, and good market application prospect.
Drawings
Fig. 1a and 1b are a schematic overall structure diagram and a schematic partial disassembly diagram of a multi-mode sensing device, respectively;
FIG. 2 is a schematic diagram of an image signal of a multi-modal sensing device;
fig. 3a, 3b and 3c are a schematic view of a structure of a lens module, a schematic view of a split multi-modal sensor device, and a cross-sectional view of the multi-modal sensor device, respectively.
FIGS. 4a and 4b are schematic diagrams of the physical signal sensing unit before and after response, respectively;
fig. 5a and 5b are schematic views of images before and after the response of the chemical signal sensing unit, respectively.
Detailed Description
The invention is described in detail below with reference to the following figures and specific embodiments, but the invention is not limited thereto.
As shown in fig. 1, the multi-modal sensing device 1 based on an image sensor of the present invention includes an image sensor 11, optical sensing units (a physical signal sensing unit 12, a chemical signal sensing unit 13), and a lens module. The physical signal sensing unit 12 and the chemical signal sensing unit 13 may be selectively combined and decorated on the image sensor 11 as needed. As shown in fig. 2, since the physical signal sensing unit 12 and the chemical signal sensing unit 13 are on the surface of the image sensor 11, the image sensor 11 can directly obtain the image 21 of the physical signal sensing unit 12 and the image 22 of the chemical signal sensing unit 13 without a lens module. Meanwhile, a portion of the surface of the image sensor 11 not covered by the physical signal sensing unit 12 and the chemical signal sensing unit 13 may acquire an image 23 of an external object, such as a person included in the image shown in fig. 2, by a focusing function of the lens module. Preferably, as shown in fig. 1, the lens module is fixed on a housing 14, and the image sensor and the optical sensing unit decorated on the surface of the image sensor are located inside the housing, so that the contamination of dust and the like on the image sensor 11 can be prevented, and the detection accuracy can be influenced.
As shown in fig. 3, in order to obtain better detection effect, a light source 142, a first optical path control module 143 and a second optical path control module 111 may be added to the multi-modal sensing device 1 based on an image sensor. As shown in fig. 3a and 3c, a light source 142 is positioned at the top inside the housing 14 to provide light to the physical signal sensing unit 12 and the chemical signal sensing unit 13. The first optical path control module 143 is disposed between the image sensor 11 and the light source 142, and blocks light generated by the light source 142, so as to prevent the light source 142 from interfering with external image information received by the image sensor 11. As shown in fig. 3b and 3c, the second optical path control module 111 is disposed between the optical sensing unit and the gas flow structure 141, and can block external light entering from the gas flow structure 141, so as to reduce interference of the external light on the image information of the physical signal sensing unit 12 and the chemical signal sensing unit 13 received by the image sensor 11. The light source, the light path control module and the gas circulation structure are all adaptively arranged according to the optical sensing unit, for example, if the sensing unit is related to wind direction, wind speed and acceleration, the light path control module is not arranged so as not to influence detection. The physical signal sensing unit 12 and the chemical signal sensing unit 13 can be processed and decorated on the surface of the image sensor 11 by printing, spraying, spinning, printing, stamp transferring and other technologies. The physical signal sensing unit 12 and the chemical signal sensing unit 13 have a size that is at least the size of a single photoelectric conversion unit of the image sensor 11 and at most the size of the entire image sensitive area of the image sensor 11.
The physical quantity detectable by the physical signal sensing unit 12 includes, but is not limited to, temperature, humidity, wind direction, wind speed, acceleration, and the like. These changes in the physical signal may cause the physical signal sensing unit 12 to produce a change in the optical signal. As shown in fig. 4, the temperature sensing unit 311 at a low temperature and the temperature sensing unit 312 at a high temperature are different in color, and the temperature sensing units may be organic, inorganic, or liquid crystal thermochromic materials; the humidity sensing unit 321 with low humidity and the humidity sensing unit 322 with high humidity can present different colors, and humidity sensitive materials adopted by the humidity sensing unit include cobalt salt and the like; some elastic structures can change different shapes under the action of wind according to the difference of wind direction and wind speed, for example, the airflow sensing unit 331 under no wind becomes the deformed airflow sensing unit 332 when the left side blows, and the dotted line represents the shape under no deformation; some elastic structures may be displaced by acceleration, for example, the acceleration sensing unit 341 in a static state may become the acceleration sensing unit 342 that is displaced when acceleration is applied to the right. These signals can be detected by the image sensor 11.
The gas sample contacts the chemical sensing unit 13 through the gas flow structure 141, and the optical signal changes due to the change of parameters including, but not limited to, light intensity, color, shape, and refractive index. Reaction mechanisms that generate these optical signals include, but are not limited to: redox reaction, pH color change, Schiff's reaction, complexation reaction, metal dyeing, lyotropic color change, solvent adsorption, pore adsorption and the like. As shown in fig. 5, the less colored chemical sensing unit 411 before reaction is changed into a darker colored chemical sensing unit 412 after reaction with the gas sample, for example, the dark brown product is generated by the reaction of light brown N, N-dimethyl-1-naphthalene with nitrogen dioxide; the pre-reaction color-changeable chemical sensing unit 421 becomes a chemical sensing unit 422 with a different color after reacting with the gas sample, for example, blue m-cresol purple may become yellow after reacting with carbon dioxide; the pre-reaction expandable chemical sensing unit 431 becomes a chemical sensing unit 432 with a larger volume after reacting with the gas sample, for example, the volume of metal palladium expands after absorbing hydrogen; the chemical sensing unit 441 with the changeable refractive index before reaction changes into a chemical sensing unit 442 with a different light spot position after reacting with the gas sample, for example, acrylic resin can change the refractive index after combining with benzene gas; the strip-shaped chemical sensing unit 451 before reaction becomes the chemical sensing unit 452 bent after reaction with the gas sample, and for example, the sensing unit formed of a layer of paper coated with polyethylene expands and lengthens when it contacts hexane gas, while the paper does not change, so that the sensing unit bends toward one side of the paper.
The device can be widely applied to the field of detection and identification, and the application of the device in analyzing and identifying substances is described below by taking a visual-olfactory cooperative sensing device (an optical sensing unit only adopts a gas sensing unit) as a specific embodiment. In the embodiment, the gas sensing unit is composed of serum albumin modified by 6-propionyl-2-dansyl chloride, oxidized porphyrinogen and lyotropic chromotropic dye, and a gas response database is established according to the known smell and the response signal of the gas sensing unit. When the unknown object is identified, the method specifically comprises the following steps:
the method comprises the following steps: the visual sense-smell cooperative sensing device is arranged near two transparent containers filled with objects, and image signals containing gas sensing information and image information of the objects to be detected are respectively collected from the two transparent containers by utilizing the visual sense-smell cooperative sensing device.
Step two: the image signal is transmitted to a signal processing module inside or outside the image sensor 11 through a signal transmission module of the image sensor 11. The signal processing module compares the response signal of the gas sensing unit 12 with the established gas response database, and judges that the gaseous volatile matter in the first transparent container contains alcohol and grape odor components, and the gaseous volatile matter in the second transparent container contains grape odor components. And the signal processing module compares the external image information with the established image database to judge that the two transparent containers contain purple liquid.
Step three: the signal processing module integrates the response signal of the gas sensing unit 12 and the external image information, and can judge that the first transparent container is wine and the second transparent container is grape juice.
The invention can be used for detecting and identifying solid, liquid and gas. In addition to the comprehensive utilization of the response signal of the gas sensing unit 12 and the external image information, the qualitative and quantitative analysis of the substance may be performed by using the response signal of the gas sensing unit 12 and the external image information alone. The invention simulates the method of utilizing multidimensional sensory information when a living being identifies a substance, and greatly improves the identification and detection capability of the traditional machine olfaction and machine vision system.
It should be noted that the above examples are only for clearly illustrating the embodiments, and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should all embodiments be exhaustive. And obvious variations or modifications of the invention may be made without departing from the scope of the invention.
Claims (10)
1. An image sensor based multimodal sensing device comprising the following structure:
and the image sensor is used for acquiring optical signals.
And the optical sensing unit is directly decorated on the surface of the image sensor and used for detecting the sensing signal generating the optical signal response.
And the lens module is arranged above the surface of the image sensor and is used for acquiring an external image signal.
2. The image sensor-based multimodal sensing device of claim 1, wherein: the image sensor comprises an array type photoelectric conversion unit and a signal transmission module.
3. The image sensor-based multimodal sensing device of claim 2, wherein: the image sensor also has a signal processing module.
4. The image sensor-based multimodal sensing device of claim 1, wherein: the optical sensing unit is a physical signal sensing unit and/or a chemical signal sensing unit.
5. The image sensor-based multimodal sensing device of claim 4, wherein: the physical signal sensing unit can generate optical signal changes caused by changes including color, light intensity, shape, displacement and the like when a detectable physical signal exists.
6. The image sensor-based multimodal sensing device of claim 4, wherein: the chemical signal sensing unit can generate optical signal change caused by change of color, light intensity, shape, refractive index, fluorescence signal and the like when a detectable chemical signal exists.
7. The image sensor-based multimodal sensing device of claim 1, wherein: the camera lens module is fixed on the shell, the image sensor and the optical sensing unit decorated on the surface of the image sensor are positioned inside the shell, and one or more gas circulation structures are arranged on the shell.
8. The image sensor-based multimodal sensing device of claim 6, wherein: one or more light sources are also included for providing illumination to the optical sensing unit or as an excitation light source for the fluorescent signal.
9. The image sensor-based multimodal sensing device of claim 1, wherein: the system also comprises one or more light path control modules for blocking the light source and the airflow from interfering the external image information received by the image sensor.
10. A visual sense and smell sense cooperative sensing device based on an image sensor is characterized by comprising the following structures:
an image sensor for acquiring an optical signal;
the gas sensing unit is directly decorated on the surface of the image sensor and is used for detecting a gas sensing signal generating an optical signal response;
and the lens module is arranged above the surface of the unmodified image sensor and used for acquiring an external image signal.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011257142.8A CN112415002B (en) | 2020-11-10 | 2020-11-10 | Multimode sensing device based on image sensor |
| PCT/CN2020/142001 WO2022099904A1 (en) | 2020-11-10 | 2020-12-31 | Multi-modal sensing device based on image sensor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011257142.8A CN112415002B (en) | 2020-11-10 | 2020-11-10 | Multimode sensing device based on image sensor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112415002A true CN112415002A (en) | 2021-02-26 |
| CN112415002B CN112415002B (en) | 2023-03-14 |
Family
ID=74781132
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011257142.8A Active CN112415002B (en) | 2020-11-10 | 2020-11-10 | Multimode sensing device based on image sensor |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112415002B (en) |
| WO (1) | WO2022099904A1 (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113607782A (en) * | 2021-07-28 | 2021-11-05 | 浙江工业大学 | Visual perception type water quality early warning system and method enhanced by olfactory signal |
| WO2022099904A1 (en) * | 2020-11-10 | 2022-05-19 | 之江实验室 | Multi-modal sensing device based on image sensor |
| CN114531555A (en) * | 2022-04-24 | 2022-05-24 | 之江实验室 | Bionic visual sensor with enhanced end profile |
| CN114821576A (en) * | 2022-06-30 | 2022-07-29 | 之江实验室 | Target identification method and device based on visual sense and olfactory sense device |
Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2958458B1 (en) * | 1998-08-26 | 1999-10-06 | 防衛庁技術研究本部長 | Multi-view image sensor |
| CN101680839A (en) * | 2007-05-16 | 2010-03-24 | （株）赛丽康 | biochip |
| US20110024744A1 (en) * | 2009-07-31 | 2011-02-03 | E2V Semiconductors | Connection pad structure for an electronic component |
| US20120188365A1 (en) * | 2009-07-20 | 2012-07-26 | Precitec Kg | Laser processing head and method for compensating for the change in focus position in a laser processing head |
| CN103325804A (en) * | 2013-06-28 | 2013-09-25 | 上海宏力半导体制造有限公司 | Cmos image sensor structure |
| CN104641218A (en) * | 2012-07-20 | 2015-05-20 | 德尔格安全股份两合公司 | Gas measurement system |
| US20150198797A1 (en) * | 2012-07-10 | 2015-07-16 | Aïmago S.A. | Perfusion assessment multi-modality optical medical device |
| US20150358567A1 (en) * | 2014-06-05 | 2015-12-10 | Edward Hartley Sargent | Sensors and systems for the capture of scenes and events in space and time |
| CN105738292A (en) * | 2014-12-12 | 2016-07-06 | 中国科学院大连化学物理研究所 | Cell phone imaging method with colorimetric sensor array |
| CN105784840A (en) * | 2016-04-27 | 2016-07-20 | 浙江大学 | Blood viscosity measuring device and method based on small blood samples |
| US20160238538A1 (en) * | 2013-09-23 | 2016-08-18 | Siemens Healthcare Diagnostics Inc. | Diagnostic apparatus for capturing medical specimen image |
| CN105898122A (en) * | 2016-05-09 | 2016-08-24 | 中国兵器工业计算机应用技术研究所 | Photoelectric sensing system |
| KR20170006758A (en) * | 2015-07-09 | 2017-01-18 | (주)옵토레인 | Apparatus for sensing a harmful gas and method therefore |
| US20180073989A1 (en) * | 2016-09-09 | 2018-03-15 | Medtronic, Inc. | Fluid sensor card |
| CN108229255A (en) * | 2016-12-21 | 2018-06-29 | 江苏迪康姆生物技术有限公司 | A kind of palmmprint acquires all-in-one machine |
| CN109475328A (en) * | 2016-06-09 | 2019-03-15 | 因赛特系统公司 | For detecting the integrated light-emitting display and sensor of biological nature |
| CN109801934A (en) * | 2018-12-13 | 2019-05-24 | 深圳市灵明光子科技有限公司 | A kind of image sensing cell and preparation method thereof, imaging sensor |
| CN110155364A (en) * | 2019-05-08 | 2019-08-23 | 之江实验室 | A kind of unmanned plane electron speed regulator and multi-rotor unmanned aerial vehicle with vibration detection function |
| JP2019148801A (en) * | 2019-03-20 | 2019-09-05 | ザ ボード オブ トラスティーズ オブ ザ レランド スタンフォード ジュニア ユニバーシティー | Method for using epi-illumination fluorescence microscope, method for using imaging device, and epi-illumination fluorescence microscope |
| CN110392214A (en) * | 2019-08-05 | 2019-10-29 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN110662957A (en) * | 2017-05-24 | 2020-01-07 | 格拉斯哥大学校董会 | Metabolite detection device and corresponding detection method |
| CN110799830A (en) * | 2017-04-27 | 2020-02-14 | 多元生物学公司 | Orthogonal multi-biometric sensing and imaging system |
| CN111145190A (en) * | 2019-12-27 | 2020-05-12 | 之江实验室 | Single organ interaction method based on medical image processing and visualization |
| CN111436209A (en) * | 2018-11-12 | 2020-07-21 | 深圳市汇顶科技股份有限公司 | Optical sensing device and terminal |
| CN111617515A (en) * | 2020-05-15 | 2020-09-04 | 浙江大学 | Gas-liquid-solid three-phase separation device and separation method based on array sensor |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6998613B2 (en) * | 2003-04-22 | 2006-02-14 | Raytheon Company | Integrated spectroscopic microbolometer with microfilter arrays |
| CN101514956B (en) * | 2008-08-18 | 2011-06-22 | 北京九州泰康生物科技有限责任公司 | Molecule detecting system |
| US8358419B2 (en) * | 2011-04-05 | 2013-01-22 | Integrated Plasmonics Corporation | Integrated plasmonic sensing device and apparatus |
| CN106918626B (en) * | 2017-04-14 | 2019-06-25 | 中国石油化工股份有限公司 | Dangerous atmosphere comprehensive state fingerprint identification method |
| CN107402244A (en) * | 2017-06-30 | 2017-11-28 | 江苏大学 | A kind of bimodal interaction sensor array chip manufacturing and information acquisition method |
| CN109447130B (en) * | 2018-09-30 | 2021-10-12 | 江苏大学 | Ha-bai preserved meat detection device and method based on visual gas-sensitive array |
| CN111539234A (en) * | 2020-05-28 | 2020-08-14 | 厦门大学 | Molecular barcode detection method based on modular super-structure surface and CMOS image sensor |
| CN112415002B (en) * | 2020-11-10 | 2023-03-14 | 之江实验室 | Multimode sensing device based on image sensor |
-
2020
- 2020-11-10 CN CN202011257142.8A patent/CN112415002B/en active Active
- 2020-12-31 WO PCT/CN2020/142001 patent/WO2022099904A1/en active Application Filing
Patent Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2958458B1 (en) * | 1998-08-26 | 1999-10-06 | 防衛庁技術研究本部長 | Multi-view image sensor |
| CN101680839A (en) * | 2007-05-16 | 2010-03-24 | （株）赛丽康 | biochip |
| US20120188365A1 (en) * | 2009-07-20 | 2012-07-26 | Precitec Kg | Laser processing head and method for compensating for the change in focus position in a laser processing head |
| US20110024744A1 (en) * | 2009-07-31 | 2011-02-03 | E2V Semiconductors | Connection pad structure for an electronic component |
| US20150198797A1 (en) * | 2012-07-10 | 2015-07-16 | Aïmago S.A. | Perfusion assessment multi-modality optical medical device |
| CN104641218A (en) * | 2012-07-20 | 2015-05-20 | 德尔格安全股份两合公司 | Gas measurement system |
| CN103325804A (en) * | 2013-06-28 | 2013-09-25 | 上海宏力半导体制造有限公司 | Cmos image sensor structure |
| US20160238538A1 (en) * | 2013-09-23 | 2016-08-18 | Siemens Healthcare Diagnostics Inc. | Diagnostic apparatus for capturing medical specimen image |
| US20150358567A1 (en) * | 2014-06-05 | 2015-12-10 | Edward Hartley Sargent | Sensors and systems for the capture of scenes and events in space and time |
| CN105738292A (en) * | 2014-12-12 | 2016-07-06 | 中国科学院大连化学物理研究所 | Cell phone imaging method with colorimetric sensor array |
| KR20170006758A (en) * | 2015-07-09 | 2017-01-18 | (주)옵토레인 | Apparatus for sensing a harmful gas and method therefore |
| CN105784840A (en) * | 2016-04-27 | 2016-07-20 | 浙江大学 | Blood viscosity measuring device and method based on small blood samples |
| CN105898122A (en) * | 2016-05-09 | 2016-08-24 | 中国兵器工业计算机应用技术研究所 | Photoelectric sensing system |
| CN109475328A (en) * | 2016-06-09 | 2019-03-15 | 因赛特系统公司 | For detecting the integrated light-emitting display and sensor of biological nature |
| US20180073989A1 (en) * | 2016-09-09 | 2018-03-15 | Medtronic, Inc. | Fluid sensor card |
| CN108229255A (en) * | 2016-12-21 | 2018-06-29 | 江苏迪康姆生物技术有限公司 | A kind of palmmprint acquires all-in-one machine |
| CN110799830A (en) * | 2017-04-27 | 2020-02-14 | 多元生物学公司 | Orthogonal multi-biometric sensing and imaging system |
| CN110662957A (en) * | 2017-05-24 | 2020-01-07 | 格拉斯哥大学校董会 | Metabolite detection device and corresponding detection method |
| CN111436209A (en) * | 2018-11-12 | 2020-07-21 | 深圳市汇顶科技股份有限公司 | Optical sensing device and terminal |
| CN109801934A (en) * | 2018-12-13 | 2019-05-24 | 深圳市灵明光子科技有限公司 | A kind of image sensing cell and preparation method thereof, imaging sensor |
| JP2019148801A (en) * | 2019-03-20 | 2019-09-05 | ザ ボード オブ トラスティーズ オブ ザ レランド スタンフォード ジュニア ユニバーシティー | Method for using epi-illumination fluorescence microscope, method for using imaging device, and epi-illumination fluorescence microscope |
| CN110155364A (en) * | 2019-05-08 | 2019-08-23 | 之江实验室 | A kind of unmanned plane electron speed regulator and multi-rotor unmanned aerial vehicle with vibration detection function |
| CN110392214A (en) * | 2019-08-05 | 2019-10-29 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN111145190A (en) * | 2019-12-27 | 2020-05-12 | 之江实验室 | Single organ interaction method based on medical image processing and visualization |
| CN111617515A (en) * | 2020-05-15 | 2020-09-04 | 浙江大学 | Gas-liquid-solid three-phase separation device and separation method based on array sensor |
Non-Patent Citations (3)
| Title |
|---|
| HUANG XIAO-WEI ET，: "Colorimetric sensor arrays based on chemo-responsive dyes for food odor visualization", 《TRENDS IN FOOD SCIENCE & TECHNOLOGY》 * |
| 吕学勤 等，: "图像处理在焊缝熔池控制中的应用", 《上海电力学院学报》 * |
| 张超凡 等，: "基于视觉形态特征检测的烟梗切丝质量分析", 《烟草科技》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022099904A1 (en) * | 2020-11-10 | 2022-05-19 | 之江实验室 | Multi-modal sensing device based on image sensor |
| CN113607782A (en) * | 2021-07-28 | 2021-11-05 | 浙江工业大学 | Visual perception type water quality early warning system and method enhanced by olfactory signal |
| CN114531555A (en) * | 2022-04-24 | 2022-05-24 | 之江实验室 | Bionic visual sensor with enhanced end profile |
| CN114821576A (en) * | 2022-06-30 | 2022-07-29 | 之江实验室 | Target identification method and device based on visual sense and olfactory sense device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112415002B (en) | 2023-03-14 |
| WO2022099904A1 (en) | 2022-05-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112415002B (en) | Multimode sensing device based on image sensor | |
| Geng et al. | Recent progress in optical biosensors based on smartphone platforms | |
| Lopez-Ruiz et al. | Smartphone-based simultaneous pH and nitrite colorimetric determination for paper microfluidic devices | |
| Huang et al. | Smartphone-based analytical biosensors | |
| Quesada-González et al. | Mobile phone-based biosensing: An emerging “diagnostic and communication” technology | |
| CN103649731B (en) | Proving installation | |
| Askim et al. | Hand-held reader for colorimetric sensor arrays | |
| US9983139B2 (en) | Modular illumination and sensor chamber | |
| CA2802318C (en) | Device for detecting an analyte in a bodily fluid | |
| CN101769925A (en) | Method and system for intelligently identifying and reading immunity-chromatography test strip and application thereof | |
| Ning et al. | Rapid segmentation and sensitive analysis of CRP with paper-based microfluidic device using machine learning | |
| Kassal et al. | Wireless and mobile optical chemical sensors and biosensors | |
| CN107219221A (en) | Drugs Checkout Gear and detection method | |
| US20230097619A1 (en) | Automatic test card for multi-blood group system and test method | |
| CN105319210B (en) | Urinalysis method | |
| Tania et al. | Assay type detection using advanced machine learning algorithms | |
| Wang et al. | A standalone and portable microfluidic imaging detection system with embedded computing for point-of-care diagnostics | |
| CN105277680A (en) | Intelligentized immunochromatography and spectral analysis integrated device and analysis method therefor | |
| CN104400785B (en) | Interactive intelligent household service robot | |
| Shi et al. | Multi-parameter photoelectric data fitting for microfluidic sweat colorimetric analysis | |
| CN108387542A (en) | Fluorine ion colorimetric sensor and its application in a kind of water based on smart mobile phone | |
| Bui et al. | Smartphone-based portable bio-chemical sensors: exploring recent advancements | |
| CN208818644U (en) | A kind of multi-functional quick analytic instrument system of hand-held intelligent | |
| US20190017987A1 (en) | Performing one or more analyses on a thin layer of biologic fluid using optically responsive chemical sensors | |
| CN104297230A (en) | Portable type biochip detection device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |