CN111366252A - Method for stabilizing screening result of infrared human body surface temperature rapid screening instrument - Google Patents
Method for stabilizing screening result of infrared human body surface temperature rapid screening instrument Download PDFInfo
- Publication number
- CN111366252A CN111366252A CN202010429268.2A CN202010429268A CN111366252A CN 111366252 A CN111366252 A CN 111366252A CN 202010429268 A CN202010429268 A CN 202010429268A CN 111366252 A CN111366252 A CN 111366252A
- Authority
- CN
- China
- Prior art keywords
- information
- human body
- infrared
- target person
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000012216 screening Methods 0.000 title claims description 33
- 230000000087 stabilizing effect Effects 0.000 title claims description 6
- 230000005540 biological transmission Effects 0.000 claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000009529 body temperature measurement Methods 0.000 claims abstract description 14
- 238000001514 detection method Methods 0.000 claims abstract description 6
- 230000036760 body temperature Effects 0.000 claims description 51
- 238000004891 communication Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 6
- 210000000689 upper leg Anatomy 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 3
- 238000012856 packing Methods 0.000 claims description 3
- 238000004806 packaging method and process Methods 0.000 claims description 2
- 230000003993 interaction Effects 0.000 abstract 1
- 238000010223 real-time analysis Methods 0.000 abstract 1
- 238000001931 thermography Methods 0.000 abstract 1
- 230000002159 abnormal effect Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 206010037660 Pyrexia Diseases 0.000 description 1
- 241000700605 Viruses Species 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/0022—Radiation pyrometry, e.g. infrared or optical thermometry for sensing the radiation of moving bodies
- G01J5/0025—Living bodies
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/80—Calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01K—MEASURING TEMPERATURE; MEASURING QUANTITY OF HEAT; THERMALLY-SENSITIVE ELEMENTS NOT OTHERWISE PROVIDED FOR
- G01K1/00—Details of thermometers not specially adapted for particular types of thermometer
- G01K1/02—Means for indicating or recording specially adapted for thermometers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01K—MEASURING TEMPERATURE; MEASURING QUANTITY OF HEAT; THERMALLY-SENSITIVE ELEMENTS NOT OTHERWISE PROVIDED FOR
- G01K13/00—Thermometers specially adapted for specific purposes
- G01K13/20—Clinical contact thermometers for use with humans or animals
- G01K13/223—Infrared clinical thermometers, e.g. tympanic
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J2005/0077—Imaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Spectroscopy & Molecular Physics (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Alarm Systems (AREA)
Abstract
The invention relates to the field of video processing, in particular to a method for processing a video file with temperature information; when the temperature detection early warning system records the infrared thermal imaging picture, the invention synchronously stores the image picture, all full-screen temperature data and temperature measurement rule information in the recording time period. When the infrared video file is opened by the offline temperature analysis software, determining whether full-screen temperature data and a temperature measuring area exist or not and opening the full-screen temperature data and the temperature measuring area; when the infrared video file is played, the full-screen temperature data and the current data of the temperature measurement rule are positioned through the currently played time point, and the current temperature result of the temperature measurement rule is calculated and displayed on the played video picture through the real-time analysis of the recorded full-screen temperature data. The invention also relates to a remote data transmission process, namely, transmitting the full data to the central server for the processing of the supervisory personnel, transmitting the component data to the user server for the checking of the appointed user, and determining the accuracy of the data through the interaction of the central server and the user server; through setting the storage area and the storage data in the user server, the safety of the data is ensured, and the data is effectively prevented from being tampered.
Description
Technical Field
The invention belongs to the technical field of image and data processing, and relates to a method for stabilizing a screening result of an infrared human body surface temperature rapid screening instrument.
Background
The infrared human body surface temperature rapid screening instrument is mainly applied to rapid channels with more pedestrian flow and used for screening places carrying viruses to cause fever of human bodies. The body temperature screening of the human body by an infrared temperature measurement mode is an important technical and preventive measure of epidemic prevention work.
Research of practical application of the infrared human body screening instrument shows that when the infrared human body surface temperature rapid screening instrument is used for measuring the body surface temperature of the exposed skin of the head and face of a human body to screen, the temperature of the exposed skin of the head and face of the human body is influenced by various factors, so that the measurement result can fluctuate obviously, the marked human body temperature fluctuates continuously in the high-frame-rate image video, and correct reading of the human body temperature value by screening workers is influenced. In addition, the infrared human body screening instrument applied in the market at present can mark a human face recognition frame and a body temperature value for the human body in the screening area, and alarm sound is given out when one or a plurality of human body targets exceed a preset body temperature alarm threshold (such as 37.3 ℃) in the screening area, but the alarm mode which is only judged according to the alarm threshold can cause the continuous generation of the alarm sound, and the alarm can be released until all the heating targets leave the screening range. Not only does this influence the screening result, but the continuous alarm sound becomes noise pollution and also causes very bad influence to the health of screening staff.
Moreover, when the existing technology finds the abnormity, the abnormal personnel needs to be artificially notified; when the number of people is large, people are easily lost, and the tracking and positioning cannot be accurately carried out. When data is processed, the data is usually directly reported to a server cluster of a data center in a periodic or real-time reporting manner after being processed locally, and at this time, the load of each server in the server cluster and the state of an access point are not considered, so that the stability and reliability of data transmission cannot be guaranteed, and data loss is easily caused.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a method for stabilizing the screening result of a rapid infrared human body surface temperature screening instrument.
The technical scheme adopted by the invention for solving the technical problems is as follows:
according to the method, firstly, human body numbering is carried out through a face recognition and tracking technology, secondly, the multiframe temperature measurement values are subjected to arithmetic averaging through the human body numbering to serve as a human body temperature value with smaller fluctuation and more stability, and finally, the human body temperature with the number and a preset body temperature alarm threshold value are applied to a control algorithm of alarm sound together, so that the sound alarm mode of the infrared screening instrument can be obviously optimized, the user experience is improved, and the screening efficiency of workers is effectively improved.
In order to achieve the purpose, the invention adopts the following technical scheme that the processing steps are as follows:
(1) the visible light image acquisition module acquires a visible light image video stream;
(2) the infrared data acquisition and processing module acquires an infrared data stream, and performs infrared data correction and imaging processing to obtain an infrared temperature measurement data stream and an infrared image video stream;
(3) the face recognition and tracking module consists of a face recognition submodule and a face tracking submodule. The human face recognition sub-module obtains all human face positions, sizes and characteristics in each frame of image in the visible light image video stream, and the human face tracking sub-module continues to capture all human face positions, sizes and characteristics in the subsequent frames on the premise that the human face is recognized by the human face recognition sub-module and carries out human body numbering according to the human face recognition result;
(4) the infrared and visible light image registration module performs image registration on the infrared image and the visible light image, and positions of the infrared target and the visible light target are aligned one by one;
(5) the infrared temperature measurement module calculates and provides temperature values of all pixels in the infrared image;
(6) the human body temperature calculation and overtemperature judgment module calculates the arithmetic mean value of continuous multiframe human body temperatures as the body temperature value of each numbered human body according to the human body number, and controls sound alarm and display alarm according to the human body number and the overtemperature judgment result;
(7) the sound alarm module gives out sound alarm.
(8) And the display alarm module carries out alarm marking on the corresponding overtemperature human body.
After the human body numbering is carried out according to the face recognition result, user terminal information of a target person and corresponding emergency contact information are obtained according to the face recognition result, and a corresponding relation table of a face image, the human body number, the terminal information and the emergency contact information is established; and when the sound alarm is generated or the alarm is displayed, the face image, the human body number and the body temperature value are sent to a terminal corresponding to the terminal information and/or the emergency contact information.
Wherein, send face image, human body number and body temperature value to the terminal information and/or the terminal that urgent contact information corresponds includes: when the body temperature value exceeds an alarm threshold value and the continuous occurrence frequency of the human body number i is less than the upper limit N times, further identifying the health code information and the age information of a target person, when the health code of the target person is submitted for another person, firstly sending the face image, the human body number and the body temperature value to a terminal corresponding to terminal information and emergency contact information, then automatically calling the target person and a terminal corresponding to the emergency contact by using a calling robot, and informing the target person and the emergency contact in a voice calling mode;
when the health code of the target person is submitted by the user and the age is larger than or equal to the upper limit threshold value, firstly, the face image, the human body number and the body temperature value are sent to a terminal of the target person corresponding to the terminal information, then, the terminal of the target person is automatically called by using the calling robot, and the target person is informed in a voice calling mode;
when the health code of the target person is submitted by the user, the age is smaller than an upper limit threshold value and larger than a lower limit threshold value, the human face image, the human body number and the body temperature value are packaged into an early warning data packet, an identifier is arranged on a packet head and then is sent to a terminal of the target person corresponding to terminal information, wherein the identifier is used for indicating the read state of the data packet, namely when the early warning data packet is checked by the target person, the read feedback is automatically carried out, and if the read feedback is not received within specific time, the terminal of the target person is automatically called by a calling robot, and the target person is informed in a voice calling mode;
when the age is less than or equal to the lower limit threshold, packing the face image, the human body number and the body temperature value into an early warning data packet, setting an identifier on a packet header, and then sending the early warning data packet to a terminal of an emergency contact of a target person corresponding to terminal information, wherein the identifier is used for indicating the read state of the data packet, and when the read feedback is not received within a specific time, automatically calling the terminal of the emergency contact of the target person by using a calling robot, and informing the target person in a voice calling mode;
the method comprises the steps that terminal information, emergency contact information, health codes and track codes of corresponding users are obtained according to face recognition results, the information, face images, human body numbers and body temperature values are packaged into an early warning data packet, and the early warning data packet is uploaded to a data center; the method specifically comprises the following steps:
determining network state information between the local equipment and the plurality of access points, and network quality information between the plurality of access points and servers in a server cluster of the data center, selecting a plurality of links according to the network state information and the network quality information, and transmitting the early warning data packet to the data center.
Selecting a plurality of links according to the network state information and the network quality information, and transmitting the early warning data packet to a data center specifically comprises:
1) acquiring network state information between local equipment and a plurality of access points, determining the transmission rate of a data packet and the packet loss rate to determine the transmission reliability according to the network state information, and selecting two access points (two are taken as an example, or 3 or 4) with the highest reliability as transit access points;
2) after receiving the early warning data packets, the two transit access points respectively calculate network quality information between the two transit access points and each server and transit quality information, and determine a data sending path according to the network quality information and the transit quality information so as to directly send the early warning data packets to the corresponding servers or send the early warning data packets to the corresponding servers through adjacent access points; the relay quality information is determined according to the reliability of a communication link between a relay access point and an adjacent access point (determined according to a transmission rate and a packet loss rate) and the network quality information between the adjacent access point and each server; the network quality information is determined according to the current accessed amount of the server, the access permission amount, the overload probability estimated according to the historical information and the reliability of the communication link.
The method has the beneficial effects that firstly, the face positioning and the body numbering are carried out through the face recognition and tracking technology, secondly, the arithmetic mean of a plurality of body temperature values of the same body number in continuous multi-frame images is taken as the final measurement result of the body temperature of the numbered human body, and finally, the comprehensive judgment is carried out according to the body number and the overtemperature condition of the body temperature measurement result, and the correct and appropriate alarm processing is carried out, so that the body temperature value of the person marked in the image is prevented from jumping up and down quickly to influence the reading, and the continuous alarm noise pollution is prevented from being generated when the overtemperature is abnormal to influence the correctness of the screening result and the screening efficiency. Meanwhile, a notification mode aiming at different crowds is set, so that the tracking and positioning of personnel are effectively improved, and meanwhile, the transmission mode of the data packet is set, so that the stability of data transmission is ensured.
Drawings
The invention is further illustrated with reference to the following figures and examples.
FIG. 1 is a block diagram of an image processing structure of an infrared screening apparatus according to the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.
The method comprises the steps of firstly carrying out human body numbering through a face recognition and tracking technology, secondly carrying out arithmetic averaging on a plurality of frames of temperature measurement values through the human body numbering to obtain a human body temperature value with smaller fluctuation and more stability, and finally applying the human body temperature with the number and a preset body temperature alarm threshold value to an alarm sound control algorithm together, so that the sound alarm mode of the infrared screening instrument can be obviously optimized, the user experience is improved, and the screening efficiency of workers is effectively improved.
As shown in fig. 1, a processing module of the method for stabilizing a screening result of an infrared human body surface temperature rapid screening instrument of the present invention includes: 1. a visible light image acquisition module; 2. the infrared data acquisition and processing module; 3. a face recognition and tracking module; 4. an infrared and visible image registration module; 5. an infrared temperature measurement module; 6. a human body temperature calculation and overtemperature judgment module; 7. a sound alarm module; 8. a display alarm module; the processing module may be located in the local device, or may be connected to the local device, and transmits corresponding data to the local device.
(1) The visible light image acquisition module acquires a frame of visible light image represented as I (x1, y1), where x1 is the abscissa of the visible light image data and y1 is the ordinate of the visible light image data.
(2) The infrared data acquisition and processing module firstly acquires a frame of infrared data represented as AD1 (x 2, y 2). The infrared data AD1 (x 2, y 2) is subjected to infrared correction processing to obtain corrected infrared data AD2 (x 2, y 2), and the corrected infrared data AD2 (x 2, y 2) is subjected to imaging processing to obtain infrared image video data IR (x 2, y 2), wherein x2 is an abscissa of the infrared data, and y2 is an ordinate of the infrared data.
(3) In the face recognition and tracking module, firstly, a face recognition submodule detects and recognizes a face target in each frame of acquired visible light image I (x1, y1), and a face recognition result is recorded as face [ I ] [ j ], which represents the ith face detection result in the jth frame, wherein face [ I ] [ j ]. x (the horizontal coordinate of a frame), face [ I ] [ j ]. y (the vertical coordinate of the frame), face [ I ] [ j ]. width (the width of the frame), face [ I ] [ j ]. height (the height of the frame) four parameters frame out the face position, and face [ I ] [ j ]. encoding represents face feature vector coding. The human face tracking sub-module generates a human body coding table according to the human face detection result faces [ ] [ j ] of the j frame, and when the human face of the j +1 frame is detected to obtain the detection result faces [ ] [ j +1], the human face tracking sub-module performs comparison matching in the known human face coding table, increases or deletes the human face codes, and updates the human body coding table. The number of consecutive occurrences of the human i before j frames is recorded as faces [ i ] [ j ]. n.
(4) And the infrared and visible light image registration module calculates the position of the infrared image coordinates (x 2, y 2) in the visible light image coordinates as (x 2+ Δ x, y2+ Δ y) and records the Δ x and Δ y values when the infrared image coordinates and the visible light image are completely fused through fusion superposition.
(5) And the infrared temperature measurement module obtains infrared image full-pixel temperature data T (x 2, y 2) by calibrating and calculating the corrected infrared data AD2 (x 2, y 2).
(6) In the human body temperature calculation and overtemperature judgment module, processing is carried out according to the following steps:
(a) according to the visible light face detection result faces and the infrared and visible light image registration information delta x and delta y, infrared face frame position information is obtained, and the calculation method comprises the following steps:
ir_faces[i][j].x = faces[i][j].x + Δx;
ir_faces[i][j].y = faces[i][j].y + Δy;
ir_faces[i][j].width = faces[i][j].width;
ir_faces[i][j].height = faces[i][j].height;
as an optional embodiment, the first face frame may be a center coordinate of the face frame, or may be a certain point coordinate, or may be implemented in other optional manners.
(b) According to the infrared face frame position information ir _ faces [ i ] [ j ] x, ir _ faces [ i ] [ j ] y, ir _ faces [ i ] [ j ] width, ir _ faces [ i ] [ j ] height and the infrared image full pixel temperature T (x 2, y 2), counting the highest temperature value in the ith infrared face frame as the ith frame personal temperature value, and recording the ith frame personal temperature value as the irs _ faces [ i ] [ j ] temperature, namely the jobs [ i ] [ j ] temperature.
(c) Suppose that in several consecutive frames before the current j frame, faces [ i ] appear continuously in the human body number i][j]N times (n times), the average temperature value faces [ i ]][j].ave_temperature =
And recording the average temperature value as the body temperature value of the ith human body of the jth frame.
(d) Setting an overtemperature alarm threshold value as Thigh, wherein the upper limit of the continuous occurrence times of the human body number i is N, and when any one of the human bodies numbered in the j frame meets the following conditions:
and if the faces [ i ] [ j ]. ave _ temperature > Thigh and the faces [ i ] [ j ]. N < N, driving the sound alarm, otherwise, closing the sound alarm.
(e) When the ith human body in all the numbered human bodies in the j frame meets the following condition:
if the faces [ i ] [ j ]. ave _ temperature > Thigh, the ith human body overtemperature display alarm is given.
(7) And driving the sound alarm according to the sound alarm driving information.
(8) And driving display alarm according to the display alarm driving information.
After the human body numbering is carried out according to the face recognition result, user terminal information of a target person and corresponding emergency contact information are obtained according to the face recognition result, and a corresponding relation table of a face image, the human body number, the terminal information and the emergency contact information is established; and when the sound alarm is generated or the alarm is displayed, the face image, the human body number and the body temperature value are sent to a terminal corresponding to the terminal information and/or the emergency contact information. When the receiver of the notification information finds that the person corresponding to the face image is not related to the person, the receiver can directly feed back specific information to inform the local equipment, and timely error correction is carried out to reduce the probability of error occurrence.
At present, due to the fact that people in China are mixed, people related to a detected target are various, and therefore, related information needs to be timely notified to target people and related people when the people give an alarm in time, and therefore attention of people is improved. However, target people in China at present relate to various people, such as the old, children, middle-aged people, young people and other people in various age groups, and different people may not obtain corresponding notifications in time due to insufficient familiarity of the different people with intelligent devices (such as mobile phones), so that potential danger is brought; therefore, it is also necessary for a particular target person to inform their guardians or emergency contacts of relevant notifications so that anomalies are discovered in a timely manner. At present, a relatively unified mode in China is to register each person, and the person is basically provided with a health code, wherein the health code is filled by the person and replaced by others; therefore, in order to improve the notification efficiency of various types of information and prevent the statistical deficiency of suspected abnormal persons, various types of notification modes can be set respectively as follows:
the terminal for sending the face image, the human body number and the body temperature value to the terminal information and/or the emergency contact information comprises: when the body temperature value exceeds the alarm threshold value and the number of times of continuous occurrence of the human body number i is less than the upper limit N times, further identifying the health code information and the age information of the target person;
when the health code of the target person is submitted for others, firstly, the face image, the human body number and the body temperature value are sent to terminals corresponding to the target person and the emergency contact person, then the calling robot is used for automatically calling the target person and the terminal corresponding to the emergency contact person, and the target person and the emergency contact person are informed in a voice calling mode;
when the health code of the target person is submitted by the user and the age is larger than or equal to the upper limit threshold value, firstly, the face image, the human body number and the body temperature value are sent to a terminal of the target person corresponding to the terminal information, then, the terminal of the target person is automatically called by using the calling robot, and the target person is informed in a voice calling mode;
when the health code of the target person is submitted by the user, the age is smaller than an upper limit threshold value and larger than a lower limit threshold value, the human face image, the human body number and the body temperature value are packaged into an early warning data packet, an identifier is arranged on a packet head and then is sent to a terminal of the target person corresponding to terminal information, wherein the identifier is used for indicating the read state of the data packet, namely when the early warning data packet is checked by the target person, the read feedback is automatically carried out, and if the read feedback is not received within specific time, the terminal of the target person is automatically called by a calling robot, and the target person is informed in a voice calling mode;
when the age is less than or equal to the lower limit threshold, packing the face image, the human body number and the body temperature value into an early warning data packet, setting an identifier on a packet header, and then sending the early warning data packet to a terminal of an emergency contact of a target person corresponding to terminal information, wherein the identifier is used for indicating the read state of the data packet, and when the read feedback is not received within a specific time, automatically calling the terminal of the emergency contact of the target person by using a calling robot, and informing the target person in a voice calling mode;
the upper and lower thresholds mentioned above may be set according to experience in the art.
In the process, different communication modes are creatively set for different people, and whether the automatic calling robot is used or not is set according to the actual personnel characteristics, so that the manual operation is reduced, and the notification efficiency is improved; meanwhile, various notifications are effectively notified to the target personnel, and the prevention efficiency is improved.
After obtaining relevant data locally, the data is generally required to be uploaded to a data center, and a plurality of servers are arranged in a common data center; that is, a server cluster of a data center may generally be a distributed cluster, where servers may be in the same place or in different places; therefore, a plurality of access points connected with the server exist, and in order to improve the data transmission efficiency, a link needs to be selected based on communication information between the local device and the access point, and between the access point and the server; however, in the prior art, only one link is usually selected; and, so as to be transmitted directly to the server through the access point, thereby causing data loss if the link between the access point and the server is suddenly abnormal. Therefore, the invention creatively provides a data transmission strategy, which comprises the following steps:
acquiring terminal information, emergency contact information, health codes and track codes of corresponding users according to face recognition results, packaging the information, face images, human body numbers and body temperature values into an early warning data packet, and uploading the early warning data packet to a data center; the method specifically comprises the following steps:
determining network state information between the local equipment and the plurality of access points, and network quality information between the plurality of access points and servers in a server cluster of the data center, selecting a plurality of links according to the network state information and the network quality information, and transmitting the early warning data packet to the data center.
Selecting a plurality of links according to the network state information and the network quality information, and transmitting the early warning data packet to a data center specifically comprises:
1) acquiring network state information between local equipment and a plurality of access points, determining the transmission rate of a data packet and the packet loss rate to determine the transmission reliability according to the network state information, and selecting two access points with the highest reliability as transit access points; the determination of the reliability of the transmission is not limited to depending on the transmission rate and the packet loss rate, and may also include interference noise, signal strength, and the like; and the reliability can be quantitatively expressed in a weighted value mode; the obtained values are transmitted to each server of the data center through independent links; and adding the weight value corresponding to the transit access point into an additional data packet header.
2) After receiving the early warning data packets, the two transit access points respectively calculate network quality information between the two transit access points and each server and transit quality information, and determine a data sending path according to the network quality information and the transit quality information so as to directly send the early warning data packets to the corresponding servers or send the early warning data packets to the corresponding servers through adjacent access points; the relay quality information is determined according to the reliability of a communication link between a relay access point and an adjacent access point (determined according to a transmission rate and a packet loss rate) and the network quality information between the adjacent access point and each server; the network quality information is determined according to the current accessed amount of the server, the access permission amount, the overload probability estimated according to the historical information and the reliability of the communication link. The reliability of the communication link can be calculated in the same manner as the reliability of transmission, and can also be calculated in different manners to obtain the value of the reliability index; the network quality information and the transfer quality information can also be quantitatively expressed in a numerical mode; after the values are obtained, the values are transmitted to each server of the data center through independent links; the independent transmission link referred to above is a link that is different from the redundant link used in the transmission of the early warning packet. And adding the transit quality information value corresponding to the access point participating in forwarding to the extra header.
In an optional embodiment, after receiving the value corresponding to each access point, the data center determines the relay quality information values corresponding to the relay access point and the forwarding access point according to a selection strategy of a link; after the server of the data center receives the early warning data packet, verifying the data packet according to the value; to determine the accuracy of the data packet; because at least two servers in the data center receive the early warning data packets, the two servers respectively verify the accuracy of the data packets and then compare the received data packets again to verify the integrity of the data in the data packets. When the verification is inaccurate or incomplete, the local equipment is informed to retransmit; the redundant link used in the retransmission process is completely different from the original redundant link, that is, no intersection exists.
In the prior art, each server usually sets the corresponding access proportion according to the access amount, thereby improving the processing efficiency of various access requests; but when the servers are not distributed in the same area, it is difficult to ensure load balancing among the servers. Therefore, when the link is selected, the condition is considered preferentially, the condition that the link is possibly overloaded is estimated according to the access permission quantity of each server and the historical information thereof, namely, the current accessed quantity, the access permission quantity, the overload probability estimated according to the historical information, the reliability of the communication link and other information of the server are considered when the network quality information is set, so that the load information of the server can be estimated according to the historical information, and the load balance among the servers in different areas is effectively improved.
The method has the beneficial effects that firstly, the face positioning and the body numbering are carried out through the face recognition and tracking technology, secondly, the arithmetic mean of a plurality of body temperature values of the same body number in continuous multi-frame images is taken as the final measurement result of the body temperature of the numbered human body, and finally, the comprehensive judgment is carried out according to the body number and the overtemperature condition of the body temperature measurement result, and the correct and appropriate alarm processing is carried out, so that the body temperature value of the person marked in the image is prevented from jumping up and down quickly to influence the reading, and the continuous alarm noise pollution is prevented from being generated when the overtemperature is abnormal to influence the correctness of the screening result and the screening efficiency. Meanwhile, a notification mode aiming at different crowds is set, so that the tracking and positioning of personnel are effectively improved, and meanwhile, the transmission mode of the data packet is set, so that the stability of data transmission is ensured.
In light of the foregoing description of the preferred embodiment of the present invention, many modifications and variations will be apparent to those skilled in the art without departing from the spirit and scope of the invention. The technical scope of the present invention is not limited to the content of the specification, and must be determined according to the scope of the claims.
Claims (5)
1. A method for stabilizing screening results of an infrared human body surface temperature rapid screening instrument is characterized by comprising the following steps:
(1) the visible light image acquisition module acquires a visible light image video stream;
(2) the infrared data acquisition and processing module acquires an infrared data stream, and performs infrared data stream correction and imaging processing to obtain an infrared temperature measurement data stream and an infrared image video stream;
(3) the human face recognition sub-module obtains all human face positions, sizes and characteristics in each frame of image in a visible light image video stream, and the human face tracking sub-module continues to capture all human face positions, sizes and characteristics in subsequent frames on the premise that the human face recognition sub-module recognizes a human face, and carries out human body numbering according to a human face recognition result;
(4) the infrared and visible light image registration module performs image registration on the infrared image and the visible light image, and positions of the infrared target and the visible light target are aligned one by one;
(5) the infrared temperature measurement module calculates and provides temperature values of all pixels in the infrared image;
(6) the human body temperature calculation and overtemperature judgment module calculates the arithmetic mean value of continuous multiframe human body temperatures as the body temperature value of each numbered human body according to the human body number, and controls sound alarm and display alarm according to the human body number and the overtemperature judgment result;
(7) the sound alarm module gives out sound alarm;
(8) and the display alarm module carries out alarm marking on the corresponding overtemperature human body.
2. The method of claim 1,
a) according to the visible light face detection result faces and the infrared and visible light image registration information delta x and delta y, infrared face frame position information is obtained, and the calculation method comprises the following steps:
ir_faces[i][j].x = faces[i][j].x + Δx;
ir_faces[i][j].y = faces[i][j].y + Δy;
ir_faces[i][j].width = faces[i][j].width;
ir_faces[i][j].height = faces[i][j].height;
b) counting the highest temperature value in the ith infrared face frame as the ith frame personal temperature value of the jth frame, and recording the highest temperature value as the (i) th temperature of the (i) th infrared face frame, namely the (i) th temperature of the (i) th infrared face frame;
c) suppose that in several consecutive frames before the current j frame, faces [ i ] appear continuously in the human body number i][j]N times, the average temperature value faces [ i ]][j].ave_temperature =
The average temperature value is recorded as the body temperature value of the ith frame of the jth human body;
d) setting an overtemperature alarm threshold value as Thigh, wherein the upper limit of the continuous occurrence times of the human body number i is N, and when any one of the human bodies numbered in the j frame meets the following conditions:
the faces [ i ] [ j ]. ave _ temperature > Thigh, and the faces [ i ] [ j ]. N < N, then sound alarm is driven;
e) when the ith human body in all the numbered human bodies in the j frame meets the following condition:
if the faces [ i ] [ j ]. ave _ temperature > Thigh, the ith human body overtemperature display alarm is given.
3. The method of claim 1,
after the human body numbering is carried out according to the face recognition result, user terminal information of a target person and corresponding emergency contact information are obtained according to the face recognition result, and a corresponding relation table of a face image, the human body number, the terminal information and the emergency contact information is established; when sound alarm is generated or alarm is displayed, the face image, the human body number and the body temperature value are sent to a terminal corresponding to the terminal information and/or the emergency contact information;
wherein, send face image, human body number and body temperature value to the terminal information and/or the terminal that urgent contact information corresponds includes:
when the body temperature value exceeds the alarm threshold value, further identifying the health code information and the age information of the target person, when the health code of the target person is submitted by other people, firstly sending the face image, the human body number and the body temperature value to a terminal corresponding to the terminal information and the emergency contact information, then automatically calling the target person and the terminal corresponding to the emergency contact by using a calling robot, and informing the target person and the emergency contact in a voice calling mode;
when the health code of the target person is submitted by the user and the age is larger than or equal to the upper limit threshold value, firstly, the face image, the human body number and the body temperature value are sent to a terminal of the target person corresponding to the terminal information, then, the terminal of the target person is automatically called by using the calling robot, and the target person is informed in a voice calling mode;
when the health code of the target person is submitted by the user, the age is smaller than an upper limit threshold value and larger than a lower limit threshold value, the human face image, the human body number and the body temperature value are packaged into an early warning data packet, an identifier is arranged on a packet head and then is sent to a terminal of the target person corresponding to terminal information, wherein the identifier is used for indicating the read state of the data packet, namely when the early warning data packet is checked by the target person, the read feedback is automatically carried out, and if the read feedback is not received within specific time, the terminal of the target person is automatically called by a calling robot, and the target person is informed in a voice calling mode;
when the age is less than or equal to the lower limit threshold, packing the face image, the human body number and the body temperature value into an early warning data packet, setting an identifier on a packet header, and then sending the early warning data packet to a terminal of an emergency contact of a target person corresponding to terminal information, wherein the identifier is used for indicating the read state of the data packet, and when the read feedback is not received within a specific time, automatically calling the terminal of the emergency contact of the target person by using a calling robot, and informing the target person in a voice calling mode.
4. The method of claim 3, wherein the method further comprises:
acquiring terminal information, emergency contact information, health codes and track codes of corresponding users according to face recognition results, packaging the information, face images, human body numbers and body temperature values into an early warning data packet, and uploading the early warning data packet to a data center; the method specifically comprises the following steps:
determining network state information between the local equipment and the plurality of access points, and network quality information between the plurality of access points and servers in a server cluster of the data center, selecting a plurality of links according to the network state information and the network quality information, and transmitting the early warning data packet to the data center.
5. The method of claim 4,
selecting a plurality of links according to the network state information and the network quality information, and transmitting an early warning data packet to a data center specifically comprises:
1) acquiring network state information between local equipment and a plurality of access points, determining the transmission rate of a data packet and the packet loss rate to determine the transmission reliability according to the network state information, and selecting two access points with the highest reliability as transit access points;
2) after receiving the early warning data packets, the two transit access points respectively calculate network quality information between the two transit access points and each server and transit quality information, and determine a data sending path according to the network quality information and the transit quality information so as to directly send the early warning data packets to the corresponding servers or send the early warning data packets to the corresponding servers through adjacent access points; the relay quality information is determined according to the reliability of a communication link between a relay access point and an adjacent access point and the network quality information between the adjacent access point and each server; the network quality information is determined according to the current accessed amount of the server, the access permission amount, the overload probability estimated according to the historical information and the reliability of the communication link.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010429268.2A CN111366252A (en) | 2020-05-20 | 2020-05-20 | Method for stabilizing screening result of infrared human body surface temperature rapid screening instrument |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010429268.2A CN111366252A (en) | 2020-05-20 | 2020-05-20 | Method for stabilizing screening result of infrared human body surface temperature rapid screening instrument |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111366252A true CN111366252A (en) | 2020-07-03 |
Family
ID=71207516
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010429268.2A Pending CN111366252A (en) | 2020-05-20 | 2020-05-20 | Method for stabilizing screening result of infrared human body surface temperature rapid screening instrument |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111366252A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109008991A (en) * | 2018-06-21 | 2018-12-18 | 赵宏杰 | Children's body temperature infrared thermal imaging testing and analysis system and method |
| CN112001886A (en) * | 2020-07-17 | 2020-11-27 | 深圳市优必选科技股份有限公司 | Temperature detection method, device, terminal and readable storage medium |
| CN112057074A (en) * | 2020-07-21 | 2020-12-11 | 北京迈格威科技有限公司 | Respiration rate measuring method, respiration rate measuring device, electronic equipment and computer storage medium |
| CN112507783A (en) * | 2020-10-29 | 2021-03-16 | 上海交通大学 | Mask face detection, identification, tracking and temperature measurement method based on attention mechanism |
| CN114783097A (en) * | 2022-04-07 | 2022-07-22 | 南京天溯自动化控制系统有限公司 | Hospital epidemic prevention management system and method |
| CN114828224A (en) * | 2022-06-21 | 2022-07-29 | 四川创智联恒科技有限公司 | Method for quickly acquiring important information of user based on cellular wireless network |
| GB2603467A (en) * | 2021-01-28 | 2022-08-10 | Sita Advanced Travel Solutions Ltd | A method and system for screening a user's temperature |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1838582A (en) * | 2005-03-24 | 2006-09-27 | 松下电器产业株式会社 | Automatic retransmission requesting method using channel decomposition, and transmit/receive processing unit |
| CN101242367A (en) * | 2008-03-07 | 2008-08-13 | 上海华平信息技术股份有限公司 | Method for selecting forward node end in media stream |
| CN101621365A (en) * | 2008-06-30 | 2010-01-06 | 中兴通讯股份有限公司 | Synchronous non-self-adapting hybrid automatic repeat request method and system |
| CN102025594A (en) * | 2009-09-14 | 2011-04-20 | 中国科学院声学研究所 | Dynamic route adjusting method and system under NAT environment |
| CN102368952A (en) * | 2009-04-08 | 2012-03-07 | 皇家飞利浦电子股份有限公司 | System and method for monitoring pulmonary congestion |
| CN102594703A (en) * | 2012-03-19 | 2012-07-18 | 广州华多网络科技有限公司 | Relay-node-based Internet communication system and communication path selection method |
| CN202537488U (en) * | 2012-02-08 | 2012-11-21 | 邵明省 | Non-contact type thermometer special for children |
| CN103356179A (en) * | 2013-08-02 | 2013-10-23 | 青岛三友智控科技有限公司 | Body temperature testing and analyzing system and body temperature testing and analyzing method |
| CN103932683A (en) * | 2014-03-31 | 2014-07-23 | 京东方科技集团股份有限公司 | Temperature measurement device and temperature measurement method |
| CN105030212A (en) * | 2015-08-21 | 2015-11-11 | 北京异度矩阵科技有限公司 | Novel intelligent body temperature monitoring method and system |
| CN108189641A (en) * | 2017-11-29 | 2018-06-22 | 珠海格力电器股份有限公司 | On-board air conditioner control method and device |
| CN108254081A (en) * | 2018-02-13 | 2018-07-06 | 深圳市帝格尔科技有限公司 | Real time temperature monitoring system |
| CN108462970A (en) * | 2017-02-21 | 2018-08-28 | 中国移动通信有限公司研究院 | A kind of packet loss decision method and device |
| CN109191105A (en) * | 2018-09-19 | 2019-01-11 | 何晨亮 | A kind of mobile-payment system and its method of doctors and patients' information sharing platform |
| CN109215806A (en) * | 2018-10-31 | 2019-01-15 | 绍兴文理学院 | A kind of public place health monitoring systems and method based on recognition of face |
| CN110044486A (en) * | 2019-03-11 | 2019-07-23 | 武汉高德智感科技有限公司 | Method, apparatus, the equipment of repetition of alarms are avoided for human body inspection and quarantine system |
| CN110213203A (en) * | 2018-03-06 | 2019-09-06 | 腾讯科技（深圳）有限公司 | Network dispatching method, device and computer storage medium |
-
2020
- 2020-05-20 CN CN202010429268.2A patent/CN111366252A/en active Pending
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1838582A (en) * | 2005-03-24 | 2006-09-27 | 松下电器产业株式会社 | Automatic retransmission requesting method using channel decomposition, and transmit/receive processing unit |
| CN101242367A (en) * | 2008-03-07 | 2008-08-13 | 上海华平信息技术股份有限公司 | Method for selecting forward node end in media stream |
| CN101621365A (en) * | 2008-06-30 | 2010-01-06 | 中兴通讯股份有限公司 | Synchronous non-self-adapting hybrid automatic repeat request method and system |
| CN102368952A (en) * | 2009-04-08 | 2012-03-07 | 皇家飞利浦电子股份有限公司 | System and method for monitoring pulmonary congestion |
| CN102025594A (en) * | 2009-09-14 | 2011-04-20 | 中国科学院声学研究所 | Dynamic route adjusting method and system under NAT environment |
| CN202537488U (en) * | 2012-02-08 | 2012-11-21 | 邵明省 | Non-contact type thermometer special for children |
| CN102594703A (en) * | 2012-03-19 | 2012-07-18 | 广州华多网络科技有限公司 | Relay-node-based Internet communication system and communication path selection method |
| CN103356179A (en) * | 2013-08-02 | 2013-10-23 | 青岛三友智控科技有限公司 | Body temperature testing and analyzing system and body temperature testing and analyzing method |
| CN103932683A (en) * | 2014-03-31 | 2014-07-23 | 京东方科技集团股份有限公司 | Temperature measurement device and temperature measurement method |
| CN105030212A (en) * | 2015-08-21 | 2015-11-11 | 北京异度矩阵科技有限公司 | Novel intelligent body temperature monitoring method and system |
| CN108462970A (en) * | 2017-02-21 | 2018-08-28 | 中国移动通信有限公司研究院 | A kind of packet loss decision method and device |
| CN108189641A (en) * | 2017-11-29 | 2018-06-22 | 珠海格力电器股份有限公司 | On-board air conditioner control method and device |
| CN108254081A (en) * | 2018-02-13 | 2018-07-06 | 深圳市帝格尔科技有限公司 | Real time temperature monitoring system |
| CN110213203A (en) * | 2018-03-06 | 2019-09-06 | 腾讯科技（深圳）有限公司 | Network dispatching method, device and computer storage medium |
| CN109191105A (en) * | 2018-09-19 | 2019-01-11 | 何晨亮 | A kind of mobile-payment system and its method of doctors and patients' information sharing platform |
| CN109215806A (en) * | 2018-10-31 | 2019-01-15 | 绍兴文理学院 | A kind of public place health monitoring systems and method based on recognition of face |
| CN110044486A (en) * | 2019-03-11 | 2019-07-23 | 武汉高德智感科技有限公司 | Method, apparatus, the equipment of repetition of alarms are avoided for human body inspection and quarantine system |
Non-Patent Citations (1)
| Title |
|---|
| 刘晓辉，刘险峰，王雪梅著: "《网络硬件设备完全技术宝典 第3版》", 31 January 2013 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109008991A (en) * | 2018-06-21 | 2018-12-18 | 赵宏杰 | Children's body temperature infrared thermal imaging testing and analysis system and method |
| CN112001886A (en) * | 2020-07-17 | 2020-11-27 | 深圳市优必选科技股份有限公司 | Temperature detection method, device, terminal and readable storage medium |
| CN112057074A (en) * | 2020-07-21 | 2020-12-11 | 北京迈格威科技有限公司 | Respiration rate measuring method, respiration rate measuring device, electronic equipment and computer storage medium |
| CN112507783A (en) * | 2020-10-29 | 2021-03-16 | 上海交通大学 | Mask face detection, identification, tracking and temperature measurement method based on attention mechanism |
| GB2603467A (en) * | 2021-01-28 | 2022-08-10 | Sita Advanced Travel Solutions Ltd | A method and system for screening a user's temperature |
| CN114783097A (en) * | 2022-04-07 | 2022-07-22 | 南京天溯自动化控制系统有限公司 | Hospital epidemic prevention management system and method |
| CN114783097B (en) * | 2022-04-07 | 2023-09-12 | 南京天溯自动化控制系统有限公司 | Hospital epidemic prevention management system and method |
| CN114828224A (en) * | 2022-06-21 | 2022-07-29 | 四川创智联恒科技有限公司 | Method for quickly acquiring important information of user based on cellular wireless network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111366252A (en) | Method for stabilizing screening result of infrared human body surface temperature rapid screening instrument | |
| JP4718253B2 (en) | Image abnormality detection device for surveillance camera | |
| CN111861951B (en) | Dual-band monitoring method, device and system based on infrared light and visible light | |
| CN101720031A (en) | Difference method-based indoor video monitoring and alarming method | |
| US20200172134A1 (en) | Method and Apparatus to Improve Unmonitored Switch Position Reporting | |
| US20130002864A1 (en) | Quality checking in video monitoring system | |
| CN116229668B (en) | Fire disaster positioning system based on infrared video monitoring | |
| CN110705834A (en) | Industrial and mining enterprise dangerous state identification method | |
| CN116165981A (en) | Intelligent monitoring system for industrial industry safety production | |
| CN113542411A (en) | Fire control thing networking integrated management platform | |
| CN113835417A (en) | Fault detection and diagnosis method based on 5G communication network | |
| CN117319609A (en) | Internet of things big data intelligent video monitoring system and method | |
| CN113450534A (en) | Device and method for detecting approach of children to dangerous goods | |
| CN115019236A (en) | Mobile phone playing and off-duty detection alarm system and method based on deep learning | |
| CN117440018A (en) | Network camera equipment fortune pipe platform based on thing networking | |
| CN110175535A (en) | A kind of ship identification scheme and its recognition methods based on deep learning | |
| CN110708519B (en) | Interactive monitoring method for industrial field environment | |
| CN109977962B (en) | Method and system for automatically identifying fault hidden danger of optical cable | |
| CN109344927A (en) | A kind of method and system for realizing factory's automatic detecting based on RFID technique | |
| CN111866145B (en) | Maintenance operation and maintenance system is patrolled and examined to body-building equipment wisdom | |
| CN115937735A (en) | Risk monitoring method and system for monitoring safety of aerial work in real time | |
| CN108428326A (en) | A kind of eco-tour assessment management precaution information system | |
| CN114463873A (en) | Patrol system for community | |
| Dalka et al. | Distributed framework for visual event detection in parking lot area | |
| CN109699018A (en) | Indoor liveness method for real-time measurement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200703 |
|
| RJ01 | Rejection of invention patent application after publication |