CN115414072A - Pharynx swab sampling method and device, sampling equipment and computer program product - Google Patents

Pharynx swab sampling method and device, sampling equipment and computer program product Download PDF

Info

Publication number
CN115414072A
CN115414072A CN202211059856.7A CN202211059856A CN115414072A CN 115414072 A CN115414072 A CN 115414072A CN 202211059856 A CN202211059856 A CN 202211059856A CN 115414072 A CN115414072 A CN 115414072A
Authority
CN
China
Prior art keywords
sampling
mechanical arm
determining
point
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211059856.7A
Other languages
Chinese (zh)
Inventor
任沛
鞠笑竹
徐志远
车正平
奉飞飞
唐剑
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Midea Group Co Ltd
Midea Group Shanghai Co Ltd
Original Assignee
Midea Group Co Ltd
Midea Group Shanghai Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Midea Group Co Ltd, Midea Group Shanghai Co Ltd filed Critical Midea Group Co Ltd
Priority to CN202211059856.7A priority Critical patent/CN115414072A/en
Publication of CN115414072A publication Critical patent/CN115414072A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B10/00—Other methods or instruments for diagnosis, e.g. instruments for taking a cell sample, for biopsy, for vaccination diagnosis; Sex determination; Ovulation-period determination; Throat striking implements
    • A61B10/0045—Devices for taking samples of body liquids
    • A61B10/0051—Devices for taking samples of body liquids for taking saliva or sputum samples
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00194—Optical arrangements adapted for three-dimensional imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/24—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the mouth, i.e. stomatoscopes, e.g. with tongue depressors; Instruments for opening or keeping open the mouth
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/32—Surgical robots operating autonomously
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/361—Image-producing devices, e.g. surgical cameras
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30036—Dental; Teeth

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Physics & Mathematics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Robotics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Hematology (AREA)
  • Dentistry (AREA)
  • Pulmonology (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Gynecology & Obstetrics (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Image Processing (AREA)

Abstract

The application provides a pharynx swab sampling method and device, sampling equipment and a computer program product. Wherein, pharynx swab sampling method is applied to the sampling equipment including arm, and pharynx swab sampling method includes: acquiring a target image, wherein the target image is a three-dimensional image of the interior of an oral cavity; determining a plurality of sampling areas according to a target image, and determining a sampling point in each sampling area; establishing a three-dimensional coordinate system, and determining the position information of each sampling point and each sampling area in the three-dimensional coordinate system; and determining sampling operation information according to the target image and the position information, and controlling the mechanical arm to sample according to the sampling operation information.

Description

Pharynx swab sampling method and device, sampling equipment and computer program product
Technical Field
The application relates to the technical field of sampling, in particular to a pharynx swab sampling method and device, sampling equipment and a computer program product.
Background
With the development and influence of epidemic situations, the conventional nucleic acid detection has become an important means for epidemic situation screening and prevention and control. At present, in order to meet the requirement of collecting a large amount of samples and to increase the sampling speed, automated nucleic acid detecting apparatuses based on a robot arm are gradually put into use. However, the existing automatic nucleic acid detection method based on the mechanical arm has the problems of single sampling flow, poor adaptability and inaccurate sampling, thereby reducing the sampling quality and the sampling efficiency.
Disclosure of Invention
The present application is directed to solving at least one of the problems of the prior art or the related art.
To this end, a first aspect of the present application is directed to a pharyngeal swab sampling method.
A second aspect of the present application is to provide a pharyngeal swab sampling device.
A third aspect of the present application is to provide a pharyngeal swab sampling device.
A fourth aspect of the present application is to provide a sampling device.
A fifth aspect of the present application is to provide a readable storage medium.
A sixth aspect of the present application is directed to a computer program product.
In view of the above, according to one aspect of the present application, there is provided a pharyngeal swab sampling method, applied to a sampling device comprising a robotic arm, the pharyngeal swab sampling method comprising: acquiring a target image, wherein the target image is a three-dimensional image of the interior of the oral cavity; determining a plurality of sampling areas according to a target image, and determining a sampling point in each sampling area; establishing a three-dimensional coordinate system, and determining the position information of each sampling point and each sampling area in the three-dimensional coordinate system; and determining sampling operation information according to the target image and the position information, and controlling the mechanical arm to sample according to the sampling operation information.
The execution main body of the technical scheme of the pharynx swab sampling method provided by the application can be a pharynx swab sampling device, and can also be determined according to actual use requirements, and is not specifically limited herein. In order to more clearly describe the pharyngeal swab sampling method provided by the present application, the pharyngeal swab sampling device will be described below as the subject of implementation of the pharyngeal swab sampling method.
The pharynx swab sampling method provided by the application is characterized in that in the process of sampling a to-be-sampled person through sampling equipment with a mechanical arm, a three-dimensional image, namely a target image, in the oral cavity of the to-be-sampled person is acquired through a distance measuring device arranged on the sampling equipment, then a plurality of sampling areas in the oral cavity of the to-be-sampled person are determined according to the acquired target image, and then a corresponding sampling point is determined in each sampling area. On the basis, a three-dimensional coordinate system is established based on the acquired three-dimensional image of the interior of the oral cavity of the person to be sampled, and each sampling area in the interior of the oral cavity and the position information of the sampling point in the three-dimensional coordinate system are determined. Furthermore, according to the three-dimensional image, each sampling area and the position information of each sampling point in the three-dimensional coordinate system, the sampling operation information of the mechanical arm in the sampling device in the sampling process of the person to be sampled is determined, and then the movement of the mechanical arm in the sampling device is controlled according to the determined sampling operation information, so that the sample collection work of the person to be sampled is realized. Like this, thereby treat the sampler and sample based on treating the inside visual information control mechanical arm of sampler oral cavity and carry out corresponding action, guaranteed the flexibility and the adaptability of mechanical arm sampling action to guaranteed the accuracy that the sampler was treated to the mechanical arm, promoted sampling efficiency.
The pharyngeal swab sampling method according to the present application may further have the following additional features:
in the above technical solution, determining sampling operation information according to a target image and position information includes: determining the sampling sequence of each sampling point by the mechanical arm according to the target image and the position information of each sampling point; determining an entrance angle of the mechanical arm according to the target image and the position information of the sampling point; determining the motion information of the mechanical arm at each sampling point according to the target image, the sampling point and the position information of the sampling area; and determining the sampling sequence, the entrance angle and the motion information as sampling operation information.
According to the technical scheme, the sampling sequence of the mechanical arm for the plurality of sampling points in the oral cavity of the person to be sampled is determined according to the target image and the position coordinates of the sampling points in the three-dimensional coordinate system, namely the motion track of the mechanical arm between the sampling points in the whole sampling process is determined. Further, for the sampling point located at the first position in the sampling sequence, according to the three-dimensional image of the oral cavity of the person to be sampled and the position coordinates of the sampling point in the three-dimensional coordinate system, the entrance angle of the mechanical arm when sampling the sampling point is determined, that is, the direction in which the mechanical arm clamps the sampling swab to enter the oral cavity of the person to be sampled is determined. Further, for each sampling point in the oral cavity of the person to be sampled, specific motion information of the mechanical arm during sampling of each sampling point is determined according to a three-dimensional image of the oral cavity of the person to be sampled, a position coordinate range of each sampling area in the three-dimensional coordinate system and a position coordinate of each sampling point in the three-dimensional coordinate system. And on the basis, the determined sampling sequence, the determined inlet angle and the determined motion information are determined as the sampling operation information of the mechanical arm in the sampling process of the person to be sampled. Like this, based on the inside visual information of waiting to sample the person's oral cavity, confirm the sampling operation information of arm, guaranteed accuracy, flexibility and the adaptability to arm motion control to guaranteed the accuracy that the person's that samples was treated to the arm, promoted sampling efficiency.
In any one of the above technical solutions, a sampling swab is disposed on the mechanical arm, and a sampling sequence of the mechanical arm for each sampling point is determined according to the target image and the position information of each sampling point, including: determining the position information of the sampling swab head in a three-dimensional coordinate system according to the target image; determining distance information between every two sampling points and distance information between every sampling point and the sampling swab head according to the sampling swab head and the position information of each sampling point; determining a sampling sequence according to a shortest path algorithm and distance information; controlling the mechanical arm to sample according to the sampling operation information, comprising: and controlling the mechanical arm to sample each sampling point according to the sampling sequence.
In the technical scheme, the mechanical arm samples a person to be sampled by clamping a sampling swab. Before sampling is started, the position coordinates of the head of the sampling swab in the three-dimensional coordinate system are determined according to the target image, and further, the distance information between any two sampling points and the distance information from the head of the sampling swab to each sampling point are determined according to each sampling point in the oral cavity and the position coordinates of the head of the sampling swab in the three-dimensional coordinate system. On the basis, based on the shortest path algorithm, the sampling sequence of each sampling point of the mechanical arm in the whole sampling process is determined by combining the determined distance information, namely the movement track of the mechanical arm between each sampling point is determined. And further, controlling the mechanical arm to move among the sampling points according to the determined motion track so as to sequentially sample the sampling points according to the determined sampling sequence. Therefore, the accuracy of the motion control of the mechanical arm is guaranteed, the time consumed by the mechanical arm when sampling each sampling point is reduced, and the power resource of the sampling equipment is saved.
In any of the above technical solutions, determining the entry angle of the mechanical arm according to the target image and the position information of the sampling point includes: determining a plane of a sampling port, and determining position information of the plane of the sampling port in a three-dimensional coordinate system; determining depth information from the sampling point to the plane of the sampling port according to the position information of the sampling point and the plane of the sampling port; determining an entrance angle of the mechanical arm relative to the sampling point according to the target image and the depth information; controlling the mechanical arm to sample according to the sampling operation information, comprising: and controlling the mechanical arm to rotate to a target position according to the inlet angle and then to sample the sampling point in the inlet cavity.
In the technical scheme, in the process of determining the inlet angle of the mechanical arm, a sampling port plane when a person to be sampled is determined first, and then the position information of the sampling port plane in the three-dimensional coordinate system is determined. On the basis, a plane equation of the sampling port plane in the three-dimensional coordinate system is determined according to the position information of the sampling port plane in the three-dimensional coordinate system, and the distance information between the sampling point and the sampling port plane is determined according to the plane equation and the position coordinate of the sampling point in the three-dimensional coordinate system. Furthermore, according to the distance information between the sampling point and the plane of the sampling port, the target angle value of the entrance angle when the mechanical arm samples the sampling point is determined, the direction information of the sampling point in the oral cavity is determined according to the target image, and then the target rotation direction of the entrance angle when the mechanical arm samples the sampling point is determined according to the direction information. Further, when sampling is performed on a sampling point, the mechanical arm is controlled to rotate to a target position according to the inlet angle, namely, the mechanical arm is controlled to rotate the target angle value along the target rotation direction, and then the mechanical arm is controlled to clamp the sampling swab to enter the oral cavity of a person to be sampled so as to sample the sampling point. Like this, guaranteed the in close contact with of sampling swab head and sampling point among the sampling process to guaranteed to carry out the sampling quality of sampling to the sampling point, promoted sampling efficiency.
In any of the above technical solutions, the motion information includes a rotation angle when the mechanical arm samples a target sampling point, and the motion information of the mechanical arm at each sampling point is determined according to the target image, the sampling point, and the position information of the sampling area, including: determining a target sampling area corresponding to the target sampling point according to the sampling point and the position information of the sampling area; determining a rotation angle of the mechanical arm when the mechanical arm samples a target sampling point according to the target sampling area; controlling the mechanical arm to sample according to the sampling operation information, comprising: and for each sampling point, controlling the mechanical arm to swing for a preset number of times according to the rotation angle so as to sample the sampling point.
According to the technical scheme, the target sampling area where the target sampling point is located is determined according to the position coordinate range of each sampling area in the three-dimensional coordinate system and the position coordinate of each sampling point in the three-dimensional coordinate system, and then the rotating angle of the sampling swab along the inner wall of the oral cavity when the mechanical arm samples the target sampling point is determined according to the target sampling area. When the target sampling point is sampled, after the sampling swab head reaches the target sampling point, the mechanical arm is controlled to swing back and forth for a preset number of times according to the determined rotation angle, so that the sampling swab head repeatedly rubs the inner wall of the oral cavity at the target sampling point, and the sampling of the target sampling point is realized. Like this, the sampling point type adopts corresponding rotation angle to sample the sampling point, has guaranteed the flexibility and the adaptability of arm sampling action to guaranteed the accuracy that the person who treats the sample sampling of arm, promoted sampling efficiency.
In any of the above technical solutions, the motion information further includes displacement information of the mechanical arm after sampling the target sampling point, and the motion information of the mechanical arm at each sampling point is determined according to the target image, the sampling point, and the position information of the sampling area, and further includes: determining obstacle information in a target sampling area according to a target image; determining displacement information of the mechanical arm after sampling of the target sampling point is completed according to the obstacle information; controlling the mechanical arm to sample according to the sampling operation information, comprising: after sampling the target sampling point, controlling the mechanical arm to move the target distance according to the displacement information and then sampling the next sampling point.
In the technical scheme, barrier information, such as the information of barriers such as the uvula, the teeth and the tongue, in a target sampling region where a target sampling point is located is determined according to a three-dimensional image of the interior of the oral cavity of a person to be sampled. On the basis, after the mechanical arm finishes sampling the target sampling point according to the obstacle information in the target sampling area, the mechanical arm clamps the displacement information of the sampling swab to determine. And under the condition that the target sampling point is sampled, controlling the mechanical arm to move the target distance according to the determined displacement information, so that after the head of the sampling swab moves the target distance, controlling the mechanical arm to clamp the sampling swab to move to the next sampling point for sampling. Therefore, the sampling swab is prevented from touching sensitive parts such as the uvula, the teeth and the tongue when moving in the oral cavity, and the user requirements are met.
In any of the above technical solutions, the motion information includes a motion length of the mechanical arm when the mechanical arm samples each sampling point, and the motion information of the mechanical arm at each sampling point is determined according to the target image, the sampling point and the position information of the sampling area, and the method includes: determining the size of the oral cavity according to the target image; determining a plurality of auxiliary sampling points in the sampling area corresponding to the sampling points according to the oral cavity size, the sampling points and the position information of the sampling area; determining a sampling arc according to the plurality of auxiliary sampling points; determining the length of the sampling arc line as the movement length; controlling the mechanical arm to sample according to the sampling operation information, comprising: and for each sampling point, controlling the mechanical arm to move along the corresponding sampling arc so as to perform sampling.
According to the technical scheme, the oral cavity size of the person to be sampled is determined according to the three-dimensional image of the inside of the oral cavity of the person to be sampled. On the basis, according to the oral cavity size of a person to be sampled, each sampling area and the position coordinates of the target sampling point in the three-dimensional coordinate system, a plurality of auxiliary sampling points are determined in the sampling area where the target sampling point is located, namely the target sampling area. On the basis, the target sampling point and the auxiliary sampling points are sequentially connected to determine a sampling arc line, and the length of the sampling arc line is determined as the movement length of the head of the sampling swab along the inner wall of the oral cavity when the mechanical arm samples the target sampling point. In the process of sampling each sampling point, the mechanical arm is controlled to clamp the sampling swab to move, so that the head of the sampling swab rubs the inner wall of the oral cavity along the corresponding sampling arc line, and the sampling of the sampling points is realized. Like this, according to the oral cavity size of waiting to sample the person, the length of motion of edge oral cavity inner wall when sampling the sampling point to sampling swab head for the length of motion of sampling swab head edge oral cavity inner wall matches with the oral cavity size of waiting to sample the person, has guaranteed the adaptability and the flexibility of arm sampling action, has satisfied user's demand.
According to a second aspect of the present application, there is provided a pharyngeal swab sampling device, for application to a sampling apparatus comprising a robotic arm, the pharyngeal swab sampling device comprising: the acquisition unit is used for acquiring a target image, and the target image is a three-dimensional image of the interior of the oral cavity; the processing unit is used for determining a plurality of sampling areas according to the target image and determining a sampling point in each sampling area; the processing unit is also used for establishing a three-dimensional coordinate system and determining the position information of each sampling point and each sampling area in the three-dimensional coordinate system; and the processing unit is also used for determining sampling operation information according to the target image and the position information and controlling the mechanical arm to sample according to the sampling operation information.
The utility model provides a pharynx swab sampling device treats the sampling person through the sampling equipment that has the arm and carries out the in-process of sampling, obtains the three-dimensional image that treats the inside oral cavity of sampling person that distance measuring device on the sampling equipment gathered through the acquisition unit and be the target image, and then confirms a plurality of sampling regions of treating the inside oral cavity of sampling person according to the target image that acquires through the processing unit, respectively confirms a corresponding sampling point in every sampling region again. On the basis, a three-dimensional coordinate system is established through a processing unit based on the acquired three-dimensional image of the interior of the oral cavity of the person to be sampled, and each sampling area in the oral cavity and the position information of the sampling point in the three-dimensional coordinate system are determined. Furthermore, the processing unit determines sampling operation information of a mechanical arm in the sampling device in the sampling process of a person to be sampled according to the three-dimensional image, each sampling area and the position information of each sampling point in the three-dimensional coordinate system, and then controls the mechanical arm in the sampling device to move according to the determined sampling operation information, so that the sample collection work of the person to be sampled is realized. Like this, thereby treat the sampler and sample based on treating the inside visual information control mechanical arm of sampler oral cavity and carry out corresponding action, guaranteed the flexibility and the adaptability of mechanical arm sampling action to guaranteed the accuracy that the sampler was treated to the mechanical arm, promoted sampling efficiency.
According to a third aspect of the present application, there is provided a pharyngeal swab sampling device, comprising: a memory storing programs or instructions; and a processor, wherein the processor executes the program or the instructions to implement the steps of the pharyngeal swab sampling method according to any one of the above-mentioned technical schemes. Therefore, the pharyngeal swab sampling device proposed in the third aspect of the present application has all the advantages of the pharyngeal swab sampling method according to any one of the above-mentioned first aspects, and the description thereof is omitted here.
According to a fourth aspect of the present application, there is provided a sampling device comprising: a mechanical arm; the pharynx swab sampling device in the above-mentioned third aspect technical scheme, sample through pharynx swab sampling device control arm. Therefore, the sampling device proposed in the fourth aspect of the present application has all the beneficial effects of the pharyngeal swab sampling device in the third aspect, and the details are not repeated herein.
In this technical solution, the sampling apparatus further includes: and the distance measuring device is used for acquiring the three-dimensional image information of the interior of the oral cavity of the person to be sampled.
According to a fifth aspect of the present application, a readable storage medium is proposed, on which a program or instructions are stored, which when executed by a processor implement the pharyngeal swab sampling method according to any one of the above-mentioned claims. Therefore, the readable storage medium proposed by the fifth aspect of the present application has all the advantages of the pharyngeal swab sampling method according to any one of the above-mentioned first aspects, and will not be described herein again.
According to a sixth aspect of the present application, a computer program product is presented, comprising a computer program which, when executed by a processor, implements the pharyngeal swab sampling method according to any one of the previous claims. Therefore, the computer program product according to the sixth aspect of the present application has all the advantages of the pharyngeal swab sampling method according to any one of the above-mentioned first aspects, and will not be described herein again.
Additional aspects and advantages of the present application will be set forth in part in the description which follows, or may be learned by practice of the present application.
Drawings
The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
fig. 1 shows one of the flow diagrams of a pharyngeal swab sampling method of an embodiment of the present application;
fig. 2 shows a second flowchart of a pharyngeal swab sampling method according to an embodiment of the present application;
fig. 3 shows a third flowchart of a pharyngeal swab sampling method according to an embodiment of the present application;
FIG. 4 shows a fourth schematic flow diagram of a pharyngeal swab sampling method of an embodiment of the present application;
FIG. 5 shows a fifth flowchart of a pharyngeal swab sampling method of an embodiment of the present application;
fig. 6 shows a sixth schematic flow diagram of a pharyngeal swab sampling method according to an embodiment of the present application;
fig. 7 shows a seventh schematic flow diagram of a pharyngeal swab sampling method according to an embodiment of the present application;
FIG. 8 shows one of the principle schematics of a pharyngeal swab sampling method of an embodiment of the present application;
FIG. 9 shows a second schematic diagram of a pharyngeal swab sampling method of an embodiment of the present application;
FIG. 10 shows a third schematic diagram of a pharyngeal swab sampling method of an embodiment of the present application;
FIG. 11 illustrates one of the block diagrams of the pharyngeal swab sampling device of the present application;
FIG. 12 illustrates a second block diagram of the pharyngeal swab sampling device of the present application;
fig. 13 shows a block diagram of a sampling device according to an embodiment of the present application.
Detailed Description
In order that the above objects, features and advantages of the present application can be more clearly understood, the present application will be described in further detail with reference to the accompanying drawings and detailed description. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application, however, the present application may be practiced in other ways than those described herein, and therefore the scope of the present application is not limited to the specific embodiments disclosed below.
The pharyngeal swab sampling method and device, the sampling apparatus, and the computer program product provided in the embodiments of the present application are described in detail below with reference to fig. 1 to 13 through specific embodiments and application scenarios thereof.
First embodiment, fig. 1 shows one of the flow diagrams of the pharyngeal swab sampling method according to the present application. The sampling method comprises the following steps 102 to 110:
102, acquiring a target image;
104, determining a plurality of sampling areas according to the target image, and determining a sampling point in each sampling area;
106, establishing a three-dimensional coordinate system, and determining position information of each sampling point and each sampling area in the three-dimensional coordinate system;
step 108, determining sampling operation information according to the target image and the position information;
step 110: and controlling the mechanical arm to sample according to the sampling operation information.
The execution subject of the technical scheme of the pharyngeal swab sampling method provided by the application can be a pharyngeal swab sampling device, and can also be determined according to actual use requirements, and is not particularly limited herein. In order to more clearly describe the pharyngeal swab sampling method provided by the present application, the pharyngeal swab sampling device will be described below as the subject of implementation of the pharyngeal swab sampling method.
The pharynx swab sampling method provided by the application is applied to automatic or semi-automatic sampling equipment comprising a mechanical arm. The sampling device may specifically include one or more mechanical arms, and in an actual application process, the sampling device may specifically be an automatic sampling kiosk with one or more sampling mechanical arms, a movable sampling robot with one or more sampling mechanical arms, an attended semi-automatic sampling kiosk with an automatic sampling mechanical arm, an attended semi-automatic sampling robot with an automatic sampling mechanical arm, and the like, which are not limited specifically herein.
In the sampling process of the sampling equipment, the mechanical arm is controlled to move, so that the mechanical arm performs sampling actions such as sampling container acquisition, sampling container installation, sample collection, sample storage, sample operation and the like, and the sample collection work of a person to be sampled is realized.
Specifically, in the pharyngeal swab sampling method provided by the application, in the process of sampling a person to be sampled by the sampling device, a three-dimensional image, namely a target image, in the oral cavity of the person to be sampled is acquired by the distance measuring device arranged on the sampling device, and then a plurality of sampling areas and a plurality of sampling points in the oral cavity of the person to be sampled are determined according to the acquired target image. The sampling areas and the sampling points are in one-to-one correspondence, namely, after the sampling areas in the oral cavity of the person to be sampled are determined, a corresponding sampling point is determined in each sampling area. On the basis, a three-dimensional coordinate system is established based on the acquired three-dimensional image of the interior of the oral cavity of the person to be sampled, and each sampling area in the interior of the oral cavity and the position information of the sampling point in the three-dimensional coordinate system are determined. That is, the position coordinate range of each sampling region in the oral cavity of the person to be sampled in the three-dimensional coordinate system is determined, and the position coordinates of each sampling point in the three-dimensional coordinate system are determined. Furthermore, according to the three-dimensional image, each sampling area and the position information of each sampling point in the three-dimensional coordinate system, the sampling operation information of the mechanical arm in the sampling device in the sampling process of the person to be sampled is determined, and then the movement of the mechanical arm in the sampling device is controlled according to the determined sampling operation information, so that the sample collection work of the person to be sampled is realized.
That is, in the pharyngeal swab sampling method provided by the present application, in the process of sampling a person to be sampled by a sampling device with a mechanical arm, sampling operation information of the mechanical arm in the sampling process is determined based on three-dimensional image information that is visual information of the inside of the oral cavity of the person to be sampled, and then the mechanical arm is controlled to move according to the determined sampling operation information so as to sample the person to be sampled. Therefore, the mechanical arm is controlled to perform corresponding actions based on the visual information in the oral cavity of the person to be sampled, so that the person to be sampled is sampled, the flexibility and the adaptability of the sampling actions of the mechanical arm are ensured, the accuracy of the motion control of the mechanical arm is ensured, the accuracy of the mechanical arm in sampling the person to be sampled is ensured, and the sampling efficiency is improved.
In an actual application process, the distance measuring device may be a shooting device with a distance measuring function, a sensor with a distance measuring function, or a combination of the two. For example, the distance measuring device may be specifically an RGB-D camera, a combination of an RGB camera and a laser sensor, a combination of an RGB camera and a millimeter wave radar sensor, a combination of an RGB camera and an infrared camera, and the like. The specific type of the distance measuring device can be selected by those skilled in the art according to practical situations, and is not limited in particular.
Further, the three-dimensional coordinate system may be a TCP (Tool Center Point) coordinate system. The TCP coordinate system is a coordinate system with an initial tool center point as an origin, and in the sampling method provided in the present application, a three-dimensional coordinate system may be specifically established with a certain point on the mechanical arm, such as an end point of the central axis of the mechanical arm, as the origin, which is not limited herein. A three-dimensional coordinate system is established based on the tool center point, and then the motion of the mechanical arm is controlled to sample according to the position information of each sampling area and each sampling point in the three-dimensional coordinate system, so that the accuracy of the motion control of the mechanical arm is ensured, and the accuracy of sampling for a person to be sampled is further ensured.
In the second embodiment, fig. 2 shows a second flow chart of the pharyngeal swab sampling method according to the second embodiment of the present application. In the first embodiment, the step S108 may specifically include the following steps 108a to 108d:
108a, determining the sampling sequence of each sampling point by the mechanical arm according to the target image and the position information of each sampling point;
108b, determining an entrance angle of the mechanical arm according to the position information of the sampling point and the target image;
108c, determining the motion information of the mechanical arm at each sampling point according to the sampling area, the position information of the sampling point and the target image;
and step 108d, determining the motion information, the entrance angle and the sampling sequence as the sampling operation information of the mechanical arm.
In this embodiment, on the basis of the first embodiment, in the process of determining the sampling operation information of the mechanical arm in the sampling device during the sampling process of the to-be-sampled person, specifically, according to the position coordinates of each sampling point in the three-dimensional coordinate system and the target image, the sampling sequence of the plurality of sampling points in the oral cavity of the to-be-sampled person is determined, that is, the motion trajectory of the mechanical arm between each sampling point in the whole sampling process is determined. Further, for the sampling point located at the first position in the sampling sequence, according to the three-dimensional image of the oral cavity of the person to be sampled and the position coordinates of the sampling point in the three-dimensional coordinate system, the entrance angle of the mechanical arm when sampling the sampling point is determined, that is, the direction in which the mechanical arm clamps the sampling swab to enter the oral cavity of the person to be sampled is determined. Further, for each sampling point in the oral cavity of the person to be sampled, specific motion information of the mechanical arm during sampling of each sampling point is determined according to a three-dimensional image of the oral cavity of the person to be sampled, a position coordinate range of each sampling area in the three-dimensional coordinate system and a position coordinate of each sampling point in the three-dimensional coordinate system.
On the basis, the determined sampling sequence of the mechanical arm to each sampling point in the whole sampling process, the inlet angle when the mechanical arm starts sampling and the motion information when the mechanical arm samples each sampling point are determined as the sampling operation information of the mechanical arm in the sampling process of a person to be sampled, and then the motion of the mechanical arm is controlled according to the determined sampling operation information, so that the sample collection work of the person to be sampled is realized. Like this, based on the inside visual information of waiting to sample the person's oral cavity, confirm entry angle, sampling order and concrete motion information of arm in the sampling process, and then control the arm and treat the person who samples, guaranteed the flexibility and the adaptability of arm sampling action to and guaranteed the accuracy to arm motion control, thereby guaranteed the accuracy that the person who samples was treated to the arm, promoted sampling efficiency.
In a third embodiment, fig. 3 shows a third flow chart of the pharyngeal swab sampling method according to the third embodiment of the present application. In addition to the second embodiment, the mechanical arm is provided with a sampling swab, the step 108a may specifically include the following steps 108a1 to 108a3, and the step 110 may specifically include the following step 110a:
step 108a1, determining the position information of the head of the sampling swab in a three-dimensional coordinate system according to the target image;
step 108a2, determining distance information between every two sampling points and distance information between the sampling swab head and each sampling point according to the position information of each sampling point and the sampling swab head;
step 108a3, determining the sampling sequence of the mechanical arm according to the distance information and the shortest path algorithm;
and step 110a, controlling the mechanical arm to sample each sampling point according to the sampling sequence.
In this embodiment, in addition to the second embodiment, in the process of sampling the person to be sampled by the sampling device with the mechanical arm, the mechanical arm specifically holds the sampling swab to sample the person to be sampled.
In addition, in the process of determining the sampling sequence of each sampling point in the whole sampling process of the mechanical arm according to the position coordinates of each sampling point in the three-dimensional coordinate system and the target image, specifically, before starting sampling, the position coordinates of the head of the sampling swab in the three-dimensional coordinate system are determined according to the target image, and further, the distance information between any two sampling points and the distance information from the head of the sampling swab to each sampling point are determined according to each sampling point in the oral cavity and the position coordinates of the head of the sampling swab in the three-dimensional coordinate system. On the basis, based on the shortest path algorithm, the sampling sequence of the mechanical arm to a plurality of sampling points in the oral cavity of a person to be sampled is determined by combining the determined distance information, namely the motion trail of the mechanical arm among the sampling points in the whole sampling process is determined, so that the total movement path of the mechanical arm is minimum when the mechanical arm samples the sampling points according to the determined sampling sequence and the motion trail, and the power resource of the sampling equipment is saved while the sampling time is reduced.
For example, in the process of sampling the person to be sampled, the sampling point P on the left side of the oral cavity is determined according to the three-dimensional image of the interior of the oral cavity of the person to be sampled L Oral cavity right side sampling point P R Oral cavity rear side sampling point P P Sampling point P on left side of tongue TL Tongue right side sampling point P TR Five sampling points are obtained, and thus a sampling point set P = { P =isobtained L ，P R ，P P ，P TL ，P TR And all the five sampling points are three-dimensional coordinate points. On the basis, a three-dimensional coordinate system is established, and the five sampling points and the sampling swabs are determinedHead P swab Position coordinates in a three-dimensional coordinate system, and further based on each sampling point and the sampling swab head P swab For distance information between any two sampling points, and a sampling swab head P swab Distance information to each sample point is determined. Further, based on the shortest path algorithm, the sampling sequence of the five sampling points in the sampling point set P is sequenced by combining the determined distance information, and the head P of the sampling swab is obtained swab And concentrating the sampling sequence of each sampling point to the sampling points so as to ensure that the total movement path L of the mechanical arm is minimum when the mechanical arm samples each sampling point.
The total movement distance L of the mechanical arm can be specifically represented by the following formula 1:
equation 1:
Figure BDA0003823952780000131
where n represents the number of sample points in the set of sample points P, dist (P) i -P i+1 ) Representing point P i To point P i+1 A distance between P 1 To P n Representing respective sample points, P, in a set of sample points P 0 Indicating sampling swab head P swab . At the above point P i Has the coordinates of (x) i ,y i ,z i ) Point P i+1 Has the coordinates of (x) i+1 ,y i+1 ,z i+1 ) In the case of (D), the above dist (P) i -P i+1 ) Specifically, it can be calculated by the following formula 2:
equation 2:
Figure BDA0003823952780000132
in this embodiment, further, after determining the sampling sequence of the plurality of sampling points, that is, after determining the motion trajectory of the mechanical arm between the sampling points in the whole sampling process, the mechanical arm is controlled to move between the sampling points according to the determined motion trajectory, so as to sample the sampling points in sequence according to the determined sampling sequence. Therefore, the accuracy of the motion control of the mechanical arm is ensured, the sampling accuracy of the mechanical arm on a sampler is ensured, the time consumed by the mechanical arm in sampling each sampling point is reduced, and the power resource of the sampling equipment is saved.
In addition, in the practical application process, the head of the sampling swab can be used as the origin of the coordinate system to establish the three-dimensional coordinate system, so that the accuracy of controlling the motion of the mechanical arm is further ensured, and the accuracy of sampling the person to be sampled is further ensured.
In a fourth embodiment, fig. 4 shows a fourth flowchart of the pharyngeal swab sampling method according to the fourth embodiment of the present application. In the second embodiment, the step 108b may specifically include the following steps 108b1 to 108b3, and the step 110 may specifically include the following step 110b:
step 108b1, determining a plane of a sampling port, and determining position information of the plane of the sampling port in a three-dimensional coordinate system;
step 108b2, determining depth information between the sampling port plane and the sampling point according to the position information of the sampling point and the sampling port plane;
step 108b3, determining an entrance angle of the mechanical arm relative to the sampling point according to the target image and the depth information;
and step 110b, controlling the mechanical arm to rotate to a target position according to the inlet angle and then sampling the sampling point in the inlet cavity.
In this embodiment, on the basis of the above embodiment, the process of determining the entrance angle when the mechanical arm samples the sampling point according to the three-dimensional image of the inside of the oral cavity of the person to be sampled and the position coordinates of the sampling point in the three-dimensional coordinate system is further defined. Specifically, in the process of determining the entrance angle of the mechanical arm, a sampling port plane when a person to be sampled is determined, the sampling port plane is a plane where lips of the person to be sampled are located, and then position information of the sampling port plane in the three-dimensional coordinate system is determined. When the position information of the sampling port plane is determined, the position information of the sampling port plane in the three-dimensional coordinate system can be directly determined according to the specifically-constructed position information in the sampling device, such as the position information of a component for fixing the chin of a person to be sampled in the automatic sampling kiosk. In the practical application process, a three-dimensional image including the lips of a person to be sampled can be obtained through shooting by a distance measuring device on sampling equipment, and then the position information of the plane of the sampling port in the three-dimensional coordinate system is determined according to the three-dimensional image and the target image. The determination method of the position information of the sampling port plane may be selected by those skilled in the art according to actual situations, and is not limited specifically here.
On the basis, a plane equation of the sampling port plane in the three-dimensional coordinate system is determined according to the position information of the sampling port plane in the three-dimensional coordinate system, and the distance information between the sampling point and the sampling port plane, namely the depth information between the sampling point and the sampling port plane, is determined according to the plane equation and the position coordinates of the sampling point in the three-dimensional coordinate system. Furthermore, according to the distance information between the sampling point and the plane of the sampling port and the target image, the entrance angle of the mechanical arm when the sampling point is sampled is determined, namely the entrance direction of the mechanical arm clamping the sampling swab to enter the oral cavity of the person to be sampled so as to sample the sampling point is determined. Specifically, according to the distance information from the sampling point to the plane of the sampling port, the target angle value of the entrance angle when the mechanical arm samples the sampling point is determined, the orientation information of the sampling point in the oral cavity is determined according to the target image, and then the target rotation direction of the entrance angle when the mechanical arm samples the sampling point is determined according to the orientation information. For example, when the sampling point is located at the left side of the oral cavity, the target rotation direction is rightward rotation, and when the sampling point is located at the right side of the oral cavity, the target rotation direction is leftward rotation.
Further, when sampling is performed on a sampling point, the mechanical arm is controlled to rotate to a target position according to the inlet angle, namely, the mechanical arm is controlled to rotate the target angle value along the target rotation direction, and then the mechanical arm is controlled to clamp the sampling swab to enter the oral cavity of a person to be sampled so as to sample the sampling point. Like this, according to the nimble entry angle of adjusting the sampling swab in the position of sampling point, guaranteed the in close contact with of sampling swab head and sampling point among the sampling process to guaranteed to carry out the sampling quality of sampling to the sampling point, promoted sampling efficiency.
It can be understood that, at the in-process of sampling the sampling point through the sampling swab, when the sampling point of the oral cavity left and right sides was too close to the sampling port edge, if with the planar direction control arm centre gripping sampling swab of perpendicular to sampling port get into treat the person's oral cavity of sampling when inside, the gliding problem of the first edge oral cavity inner wall direction of sampling swab appears easily to make the sampling point contact of sampling swab head and the oral cavity left and right sides not inseparable enough, reduced the sampling quality to this sampling point.
Therefore, in this embodiment, the entry angle at which the robot arm samples the sample point is determined based on the distance information from the sample point to the plane of the sample port and the target image. Specifically, when the distance from the sampling point to the edge of the sampling port is large, namely when the vertical distance from the sampling point to the central axis of the sampling port is small, the sampling point is positioned in the middle area of the oral cavity. At the moment, the mechanical arm is controlled to clamp the sampling swab to enter the oral cavity of the person to be sampled in a direction perpendicular to the plane of the sampling port so as to sample the sampling point. The middle axis of the sampling port is a straight line which is perpendicular to the plane of the sampling port and passes through the central point of the area surrounded by the lips of the user in the plane of the sampling port. And when the distance from the sampling point to the edge of the sampling port is small, namely the vertical distance from the sampling point to the central axis of the sampling port is large, the sampling point is positioned in the two side areas of the oral cavity. At the moment, the specific position of the sampling point in the oral cavity is determined according to the target image, and then the mechanical arm is controlled to rotate the target angle value along the target rotation direction corresponding to the position of the sampling point, and then the sampling swab is clamped to enter the oral cavity of the person to be sampled to sample the sampling point. Like this, according to the position flexibility adjustment sampling swab's of sampling point entry angle, guaranteed the in close contact of sampling swab head and sampling point among the sampling process to the sampling quality of sampling to the sampling point has been promoted.
Exemplary embodiments of the inventionAlternatively, as shown in FIG. 9 (a), the sampling point P is shown at the left side of the oral cavity L Near the edge of the sampling port, if the sampling point P is used L For initial sampling point, the mechanical arm is controlled to clamp the sampling swab to approach the sampling point P in the inlet direction perpendicular to the plane M of the sampling port L The sampling swab head can slide along the direction of the inner wall of the oral cavity, so that the sampling swab head and the sampling point P are caused L Can not be in close contact with each other, thereby reducing the sampling point P L The quality of the sample. At this time, as shown in (b) of fig. 9, at a sampling point P L When the sampling is carried out for the initial sampling point, the mechanical arm can be controlled to clamp the sampling swab and rotate the sampling swab to the right by a target angle theta, and then the mechanical arm is controlled to clamp the sampling swab to enter the oral cavity of a person to be sampled so as to carry out sampling on the sampling point P L Sampling is performed.
Wherein the target angle theta is in the range of 0 to
Figure BDA0003823952780000161
Wherein w is the sampling swab around the sampling point P L The maximum distance that the sampling swab can move in the plane M of the sampling opening during rotation, d p Is a sampling point P L Perpendicular distance to the plane of the sample port.
In the fifth embodiment, fig. 5 shows a fifth flowchart of the pharyngeal swab sampling method according to the present application. In addition to the second embodiment, the motion information includes a rotation angle when the mechanical arm samples the target sampling point, the step 108c may specifically include the following steps 108c1 and 108c2, and the step 110 may specifically include the following step 110c:
step 108c1, determining a target sampling area where a target sampling point is located according to the sampling area and the position information of the sampling point;
step 108c2, according to the target sampling area, determining a rotation angle when the mechanical arm samples a target sampling point;
and step 110c, controlling the mechanical arm to swing for a preset number of times according to the rotation angle for each sampling point so as to sample the sampling point.
In this embodiment, on the basis of the above embodiment, the motion information of the mechanical arm when sampling each sampling point may specifically include a rotation angle of the sampling swab along the inner wall of the oral cavity when the mechanical arm samples a target sampling point. On the basis, in the process of determining the specific motion information of the mechanical arm during sampling according to the three-dimensional image of the inner part of the oral cavity of the person to be sampled, each sampling area and the position information of each sampling point in the three-dimensional coordinate system, specifically, the target sampling area where the target sampling point is located is determined according to the position coordinate range of each sampling area in the three-dimensional coordinate system and the position coordinate of each sampling point in the three-dimensional coordinate system, and further, the rotating angle of the sampling swab along the inner wall of the oral cavity during sampling of the target sampling point by the mechanical arm is determined according to the target sampling area. The specific numerical value of the rotation angle corresponds to the position of the target sampling region in the oral cavity, and the rotation angles corresponding to sampling points in different sampling regions are different. Like this, according to the sampling point in the inside different areas of oral cavity, also according to the difference of sampling point type, adopt different rotation angle to sample the sampling point, guaranteed the flexibility and the adaptability of arm sampling action to guaranteed the accuracy that the person that the arm was treated the sampling, promoted sampling efficiency.
In this embodiment, further, after the rotation angle of the sampling swab along the inner wall of the oral cavity is determined when the mechanical arm samples the target sampling point, and when the sampling swab head reaches the target sampling point, the mechanical arm is controlled to swing back and forth for a preset number of times according to the determined rotation angle, so that the sampling swab head repeatedly rubs the inner wall of the oral cavity at the target sampling point, thereby sampling the target sampling point. The specific value of the preset number may be set by a person skilled in the art according to actual conditions, and is not limited specifically herein.
In a sixth embodiment, fig. 6 shows a sixth schematic flow chart of the pharyngeal swab sampling method according to the third embodiment of the present application. Based on the fifth embodiment, the motion information further includes displacement information obtained after the mechanical arm samples the target sampling point, the step 108c may specifically include the following steps 108c3 and 108c4, and the step 110 may specifically include the following step 110d:
step 108c3, determining the obstacle information in the target sampling area according to the target image;
step 108c4, according to the obstacle information, determining displacement information of the mechanical arm after sampling the target sampling point;
and step 110d, after the target sampling point is sampled, controlling the mechanical arm to move a preset distance according to the displacement information and then sampling the next sampling point.
In this embodiment, on the basis of the above embodiment, the movement information of the mechanical arm in sampling each sampling point may specifically further include displacement information of the mechanical arm holding the sampling swab after the mechanical arm completes sampling the target sampling point. On the basis, in the process of determining specific motion information when the mechanical arm samples according to a three-dimensional image of the inside of the oral cavity of a person to be sampled, each sampling area and position information of each sampling point in the three-dimensional coordinate system, specifically, barrier information such as uvula, teeth, tongue and the like in a target sampling area where a target sampling point is located is determined according to the three-dimensional image of the inside of the oral cavity of the person to be sampled, and then displacement information of the mechanical arm clamping a sampling swab is determined after the mechanical arm finishes sampling the target sampling point according to the barrier information in the target sampling area. Further, under the condition that sampling is completed on the target sampling point, the mechanical arm is controlled to move the target distance according to the determined displacement information, so that after the head of the sampling swab moves the target distance, the mechanical arm is controlled to clamp the sampling swab to move to the next sampling point for sampling. Therefore, the sampling swab is prevented from touching sensitive parts such as the uvula, the teeth and the tongue when moving in the oral cavity, and the user requirements are met.
The specific value of the target distance may be set by a person skilled in the art according to practical situations, for example, the target distance is set to 1 cm, and is not limited in this respect.
Illustratively, in the case where the unit length of the three-dimensional coordinate system is 1 cm, the Z-axis is perpendicular to the plane of the sampling port, and the positive direction of the Z-axis is toward the inside of the oral cavity, the sampling point P is located on the back side of the oral cavity in the direction perpendicular to the plane of the sampling port P When sampling, in order to avoid the sampling swab head touching the uvula in the moving process, a sampling point P is arranged at the rear side of the oral cavity P After sampling is completed, the mechanical arm is firstly controlled to clamp the sampling swab to move so as to move the head of the sampling swab to a coordinate point (x, y, z-1) from the coordinate point (x, y, z), namely, the head of the sampling swab moves 1 cm towards the direction outside the oral cavity, and then the mechanical arm is controlled to clamp the sampling swab to move to the next sampling point for sampling.
Seventh embodiment, fig. 7 shows a seventh schematic flow chart of a pharyngeal swab sampling method according to the present application. In addition to the second embodiment, the motion information includes a motion length when the mechanical arm samples each sampling point, the step 108c may specifically include the following steps 108c5 to 108c8, and the step 110 may specifically include the following step 110e:
step 108c5, determining the size of the oral cavity according to the target image;
step 108c6, determining a plurality of auxiliary sampling points in the sampling area corresponding to the sampling point according to the sampling area, the position information of the sampling point and the oral cavity size;
108c7, determining a sampling arc line according to the plurality of auxiliary sampling points;
step 108c8, determining the length of the sampling arc line as the movement length of the mechanical arm;
and step 110e, controlling the mechanical arm to move along the corresponding sampling arc for each sampling point so as to perform sampling.
In this embodiment, on the basis of the above embodiment, the movement information of the mechanical arm when sampling each sampling point may specifically include the movement length of the sampling swab head along the inner wall of the oral cavity when the mechanical arm samples each sampling point, for example, a target sampling point. On the basis, in the process of determining the specific motion information when the mechanical arm samples according to the three-dimensional image of the inner part of the oral cavity of the person to be sampled, each sampling area and the position information of each sampling point in the three-dimensional coordinate system, specifically, the size of the oral cavity of the person to be sampled, namely the size of the sampling port of the person to be sampled is determined according to the three-dimensional image of the inner part of the oral cavity of the person to be sampled. On the basis, according to the oral cavity size of a person to be sampled, the position coordinate range of each sampling area in the three-dimensional coordinate system and the position coordinate of the target sampling point in the three-dimensional coordinate system, a plurality of auxiliary sampling points are determined in the sampling area where the target sampling point is located, namely the target sampling area. The auxiliary sampling points are respectively positioned on two sides of the target sampling point, and the auxiliary sampling points correspond to each other in pairs under the condition that no barrier exists in the target sampling area.
On the basis, the target sampling point and the auxiliary sampling points are sequentially connected to determine a sampling arc line, and the length of the sampling arc line is determined as the movement length of the head of the sampling swab along the inner wall of the oral cavity when the mechanical arm samples the target sampling point. Further, in the process of sampling each sampling point, the mechanical arm is controlled to clamp the sampling swab to move, so that the head of the sampling swab rubs the inner wall of the oral cavity along the corresponding sampling arc line, and the sampling of the sampling points is realized. Like this, according to the oral cavity size of waiting to sample the person, the length of motion of edge oral cavity inner wall when sampling the sampling swab head to the sampling point sample is confirmed for the length of motion of sampling swab head edge oral cavity inner wall matches with the oral cavity size phase-match who waits to sample the person, has guaranteed the adaptability and the flexibility of arm sampling action, has satisfied user's demand.
Illustratively, as shown in FIG. 10, for the sampling point P on the left side of the mouth L In the process of sampling the sampling point, firstly, four auxiliary sampling points P are determined in the sampling area where the sampling point is positioned L1 、P L2 、P' L1 、P' L2 . Wherein the auxiliary sampling point P L1 And auxiliary sampling point P' L1 About the sampling point P on the left side of the mouth L The axis m is axisymmetric, and the auxiliary sampling point P L2 And auxiliary sampling point P' L2 About the sampling point P on the left side of the mouth L The axis m is axisymmetric. Based on the above, according to the sampling point P on the left side of the oral cavity L And an auxiliary sampling point P L1 、P L2 、P' L1 、P' L2 A sampling arc is determined which can cover the oral cavity wall corresponding to the solid line a and the dashed line b in the figure. Further, the mechanical arm is controlled to clamp the sampling swab to move, so that the head of the sampling swab rubs the inner wall of the oral cavity along a sampling arc line, and therefore the sampling point P on the left side of the oral cavity is achieved L Sampling of (3).
It should be noted that, in the actual application process, the auxiliary sampling point P is shielded by the internal obstacle of the oral cavity L1 、P L2 There is not necessarily a corresponding auxiliary sampling point P' L1 、P' L2 The lengths of the solid line a and the broken line b may or may not be the same.
Further, in the process of determining the auxiliary sampling points, the following conditions should be satisfied between the auxiliary sampling points:
Figure BDA0003823952780000191
and | P Li-x -P L-x |≥d x 。
Wherein i is more than or equal to 1 and less than or equal to n, n is half of the number of auxiliary sampling points under the condition of no obstacle, and k min Represents point P L Minimum slope, k, of a line connecting any point on the arc covered by the solid line a max Representing point P L Maximum slope, P, of a line drawn from any point on the arc covered by the solid line a Li-y Represents point P Li Ordinate value of (A), P L-y Represents point P L Ordinate value of (a), P Li-x Represents point P Li Abscissa value of (A), P L-x Represents point P L Abscissa value of (d) x Representing point P L And point P Li The minimum distance in the x-axis direction. In particular, d x Can take any value from 0.8 cm to 1 cm, and in the practical application process, the d can be adjusted by the person skilled in the art according to the practical situation x Is set toAnd is not particularly limited herein.
In summary, as shown in fig. 8, in the throat swab sampling method provided by the present application, in the process of sampling the person to be sampled, first, according to the visual information inside the oral cavity of the person to be sampled, the sampling point position information and the oral cavity segmentation information inside the oral cavity of the person to be sampled are obtained. On the basis, the type of the sampling point and the oral obstacle information are determined according to the position information of the sampling point and the oral cavity segmentation information, and the track of the sampling point is planned. Further, the self-adaptive sampling action of the mechanical arm in the sampling equipment is adjusted according to the determined sampling point type and the obstacle information in the oral cavity, and the planning of the person to be sampled is started according to the planned sampling point track.
Like this, the pharynx swab sampling method that this application provided can specifically realize following beneficial effect: planning sampling point tracks based on a shortest path algorithm, and effectively reducing the sampling operation time required when a plurality of positions are sampled in the oral cavity; the types of the multiple sampling points are judged, and different sampling operation actions are selected for each type of sampling points, so that the sampling quality is improved; in the process of carrying out sampling operation on the left side and the right side in the oral cavity, the sampling action amplitude can be adjusted in a self-adaptive manner according to the oral cavity sizes of different persons to be sampled, so that the sampling quality is effectively ensured; in the process of carrying out sampling operation on the left side and the right side in the oral cavity, in the face of the condition that a sampling point is too close to the edge of a sampling port, the sliding of the swab head in the sampling process in the oral cavity can be effectively relieved through the self-adaptive rotary sampling swab head, and the sampling efficiency is improved.
Fig. 11 shows a block diagram of a pharyngeal swab sampling apparatus 1100 according to an embodiment of the present application. Wherein the sampling device is applied to a sampling apparatus comprising a robotic arm, the pharyngeal swab sampling device 1100 comprises an acquisition unit 1102 and a processing unit 1104:
an acquiring unit 1102, configured to acquire a target image, where the target image is a three-dimensional image of an inside of an oral cavity;
a processing unit 1104 for determining a plurality of sampling regions from the target image, and determining a sampling point in each sampling region;
the processing unit 1104 is further configured to establish a three-dimensional coordinate system, and determine position information of each sampling point and each sampling area in the three-dimensional coordinate system;
and the processing unit 1104 is further configured to determine sampling operation information according to the target image and the position information, and control the mechanical arm to perform sampling according to the sampling operation information.
The pharynx swab sampling device 1100 provided by the application, in the process of sampling a to-be-sampled person through a sampling device with a mechanical arm, acquires a three-dimensional image, namely a target image, of the to-be-sampled person inside the oral cavity, acquired by a distance measuring device on the sampling device through an acquisition unit 1102, and then determines a plurality of sampling areas and a plurality of sampling points inside the oral cavity of the to-be-sampled person according to the acquired target image through a processing unit 1104. The sampling areas and the sampling points are in one-to-one correspondence, namely, after the sampling areas in the oral cavity of the person to be sampled are determined, a corresponding sampling point is determined in each sampling area. On the basis, a three-dimensional coordinate system is established through the processing unit 1104 on the basis of the acquired three-dimensional image of the interior of the oral cavity of the person to be sampled, and each sampling area in the oral cavity and the position information of the sampling point in the three-dimensional coordinate system are determined. That is, the position coordinate range of each sampling region in the oral cavity of the person to be sampled in the above three-dimensional coordinate system is determined, and the position coordinates of each sampling point in the above three-dimensional coordinate system are determined. Furthermore, the processing unit 1104 determines sampling operation information of the mechanical arm in the sampling process of the to-be-sampled person according to the three-dimensional image, each sampling area and the position information of each sampling point in the three-dimensional coordinate system, and then controls the mechanical arm in the sampling device to move according to the determined sampling operation information, thereby realizing the sample collection work of the to-be-sampled person.
That is to say, in the throat swab sampling device 1100 provided in the present application, during the sampling process of the person to be sampled by the sampling device with the mechanical arm, based on the three-dimensional image information that is the visual information of the inside of the mouth of the person to be sampled, the sampling operation information of the mechanical arm during the sampling process is determined, and then the mechanical arm is controlled to move according to the determined sampling operation information, so as to sample the person to be sampled. Therefore, the mechanical arm is controlled to perform corresponding actions based on the visual information in the oral cavity of the person to be sampled, so that the person to be sampled is sampled, the flexibility and the adaptability of the sampling actions of the mechanical arm are guaranteed, the accuracy of motion control of the mechanical arm is guaranteed, the accuracy of the mechanical arm in sampling the person to be sampled is guaranteed, and the sampling efficiency is improved.
The sampling device is an automated or semi-automated sampling device including a mechanical arm, and the sampling device may specifically include one or more mechanical arms, which is not particularly limited herein.
Further, in an actual application process, the distance measuring device may be a shooting device with a distance measuring function, a sensor with a distance measuring function, or a combination of the two, which is not limited herein.
Further, the three-dimensional coordinate system may be a TCP coordinate system. The TCP coordinate system is a coordinate system using a tool center point in an initial state as an origin, and in the sampling method provided in the present application, a three-dimensional coordinate system may be specifically established using a certain point on the mechanical arm, for example, an end point of a central axis of the mechanical arm, as the origin, which is not limited specifically herein. A three-dimensional coordinate system is established based on the tool center point, and then the motion of the mechanical arm is controlled to sample according to the position information of each sampling area and each sampling point in the three-dimensional coordinate system, so that the accuracy of the motion control of the mechanical arm is ensured, and the accuracy of sampling for a person to be sampled is further ensured.
In this embodiment, further, the processing unit 1104 is specifically configured to: determining the sampling sequence of each sampling point by the mechanical arm according to the target image and the position information of each sampling point; determining an entrance angle of the mechanical arm according to the target image and the position information of the sampling point; determining the motion information of the mechanical arm at each sampling point according to the target image, the sampling point and the position information of the sampling area; the sampling order, the entry angle, and the motion information are determined as sampling operation information.
In this embodiment, further, a sampling swab is disposed on the mechanical arm, and the processing unit 1104 is specifically configured to: determining the position information of the sampling swab head in a three-dimensional coordinate system according to the target image; determining distance information between every two sampling points and distance information between every sampling point and the sampling swab head according to the sampling swab head and the position information of each sampling point; determining a sampling sequence according to a shortest path algorithm and distance information; and controlling the mechanical arm to sample each sampling point according to the sampling sequence.
In this embodiment, further, the processing unit 1104 is specifically configured to: determining a sampling port plane according to the target image; determining depth information from the sampling point to a sampling port plane according to the position information of the sampling point; determining an entrance angle of the mechanical arm relative to the sampling point according to the depth information; and controlling the mechanical arm to rotate the inlet angle along the preset direction and then enter the inner part of the oral cavity to sample the sampling point.
In this embodiment, further, the motion information includes a rotation angle when the mechanical arm samples the target sampling point, and the processing unit 1104 is specifically configured to: determining a target sampling area corresponding to the target sampling point according to the sampling point and the position information of the sampling area; determining a rotation angle of the mechanical arm when the mechanical arm samples a target sampling point according to the target sampling area; and for each sampling point, controlling the mechanical arm to swing for a preset number of times according to the rotation angle so as to sample the sampling point.
In this embodiment, further, the motion information further includes displacement information after the mechanical arm completes sampling of the target sampling point, and the processing unit 1104 is specifically configured to: determining obstacle information in a target sampling area according to a target image; determining displacement information of the mechanical arm after sampling of the target sampling point is completed according to the obstacle information; after sampling the target sampling point, controlling the mechanical arm to move a preset distance according to the displacement information and then sampling the next sampling point.
In this embodiment, further, the motion information includes a motion length of the mechanical arm when sampling each sampling point, and the processing unit 1104 is specifically configured to: determining the size of the oral cavity according to the target image; determining a plurality of auxiliary sampling points in the sampling area corresponding to the sampling points according to the oral cavity size, the sampling points and the position information of the sampling area; determining a sampling arc according to the plurality of auxiliary sampling points; determining the length of the sampling arc line as the movement length; and for each sampling point, controlling the mechanical arm to move along the corresponding sampling arc so as to perform sampling.
In a ninth embodiment, fig. 12 is a block diagram illustrating a pharyngeal swab sampling apparatus 1200 provided by an embodiment of the present application. Wherein, this pharynx swab sampling device 1200 includes:
a memory 1202, the memory 1202 having programs or instructions stored thereon;
processor 1204, the processor 1204 when executing the above programs or instructions implementing the steps of the pharyngeal swab sampling method in any of the above embodiments.
The pharyngeal swab sampling device 1200 provided in this embodiment includes a memory 1202 and a processor 1204, and when executed by the processor 1204, programs or instructions in the memory 1202 implement the steps of the pharyngeal swab sampling method in any of the above embodiments, so that the pharyngeal swab sampling device 1200 has all the advantages of the pharyngeal swab sampling method in any of the above embodiments, and will not be described herein again.
In particular, the memory 1202 and the processor 1204 may be connected by a bus or other means. Processor 1204 may include one or more Processing units, and Processor 1204 may be a Central Processing Unit (CPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or the like.
Tenth embodiment, fig. 13 shows a block diagram of a sampling apparatus 1300 according to an embodiment of the present application. Wherein the sampling device 1300 comprises: a robotic arm 1302; the pharyngeal swab sampling device 1200 of the above-described embodiment.
The sampling device 1300 provided by the present embodiment includes the pharyngeal swab sampling assembly 1200 of the previous embodiments. Accordingly, the sampling device 1300 has all the technical effects of the pharyngeal swab sampling device 1200 in the above embodiments, and will not be described herein again.
The sampling device 1300 includes, but is not limited to: an automated sampling kiosk with one or more sampling robotic arms, a mobile sampling robot with one or more sampling robotic arms, an attended semi-automated sampling kiosk with automated sampling robotic arms, an attended semi-automated sampling robot with automated sampling robotic arms, and the like, without specific limitation herein.
Specifically, in the working process of the sampling device 1300, the motion of the mechanical arm 1302 on the sampling device 1200 is controlled by the throat swab, so that the mechanical arm 1302 performs sampling actions such as sampling container acquisition, sampling container installation, sample collection, sample preservation, sample operation and the like, thereby realizing the sample collection of a person to be sampled.
In this embodiment, as shown in fig. 13, the sampling apparatus 1300 further includes a distance measuring device 1304, and the distance measuring device 1304 may be specifically an RGB-D camera, a combination of an RGB camera and a laser sensor, a combination of an RGB camera and a millimeter wave radar sensor, a combination of an RGB camera and an infrared camera, or the like. The specific type of the distance measuring device 1304 can be selected by those skilled in the art according to practical situations, and is not limited in particular.
Specifically, during the operation of the sampling apparatus 1300, the distance measuring device 1304 collects three-dimensional image information of the interior of the oral cavity of the person to be sampled, and the pharyngeal swab sampling device 1200 controls the mechanical arm 1302 to move according to the three-dimensional image information collected by the distance measuring device 1304, so as to realize the sample collection of the person to be sampled.
Example eleven, an example of a fifth aspect of the present application, provides a readable storage medium. Stored thereon are programs or instructions which, when executed by a processor, carry out the steps of the pharyngeal swab sampling method as in any one of the embodiments described above.
The readable storage medium provided in the embodiments of the present application stores a program or instructions, which when executed by a processor, can implement the steps of the pharyngeal swab sampling method in any of the above embodiments. Therefore, the readable storage medium has all the advantages of the pharyngeal swab sampling method in any one of the above embodiments, and the detailed description thereof is omitted here.
In particular, the readable storage medium may include any medium capable of storing or transmitting information. Examples of readable storage media include electronic circuits, semiconductor Memory devices, read-Only memories (ROMs), random Access Memories (RAMs), compact Disc Read-Only memories (CD-ROMs), flash memories, erasable ROMs (EROMs), magnetic tapes, floppy discs, optical discs, hard discs, optical fiber media, radio Frequency (RF) links, optical data storage devices, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
Twelfth, an embodiment of the sixth aspect of the present application, provides a computer program product comprising a computer program that, when executed by a processor, implements a pharyngeal swab sampling method as in any one of the above embodiments. Therefore, the computer program product according to the sixth aspect of the present application has all the advantages of the pharyngeal swab sampling method according to any one of the embodiments of the first aspect, and will not be described herein again.
In the description herein, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance unless explicitly stated or limited otherwise; the terms "connected", "mounted", "fixed", and the like are to be construed broadly and may include, for example, fixed connections, detachable connections, or integral connections; may be directly connected or indirectly connected through an intermediate. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art as appropriate.
In the description herein, the description of the terms "one embodiment," "some embodiments," "specific embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In addition, technical solutions in the embodiments of the present application may be combined with each other, but it is necessary to be based on the realization of the technical solutions by a person skilled in the art, and when the technical solutions are contradictory to each other or cannot be realized, such a combination of technical solutions should not be considered to exist, and is not within the protection scope claimed in the present application.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (13)

1. A pharyngeal swab sampling method, applied to a sampling device comprising a robotic arm, comprising:
acquiring a target image, wherein the target image is a three-dimensional image of the interior of an oral cavity;
determining a plurality of sampling areas according to the target image, and determining a sampling point in each sampling area;
establishing a three-dimensional coordinate system, and determining position information of each sampling point and each sampling area in the three-dimensional coordinate system;
and determining sampling operation information according to the target image and the position information, and controlling the mechanical arm to sample according to the sampling operation information.
2. The pharyngeal swab sampling method of claim 1, wherein said determining sampling operation information from the target image and the location information comprises:
determining the sampling sequence of the mechanical arm to each sampling point according to the target image and the position information of each sampling point;
determining an entrance angle of the mechanical arm according to the target image and the position information of the sampling point;
determining the motion information of the mechanical arm at each sampling point according to the target image, the sampling points and the position information of the sampling area;
determining the sampling order, the entry angle, and the motion information as the sampling operation information.
3. The pharyngeal swab sampling method according to claim 2, wherein the mechanical arm is provided with sampling swabs, and the determining of the sampling sequence of each sampling point by the mechanical arm according to the target image and the position information of each sampling point comprises:
determining position information of a sampling swab head in the three-dimensional coordinate system according to the target image;
determining distance information between every two sampling points and the distance information between every sampling point and the sampling swab head according to the sampling swab head and the position information of each sampling point;
determining the sampling sequence according to a shortest path algorithm and the distance information;
the controlling the mechanical arm to sample according to the sampling operation information comprises the following steps:
and controlling the mechanical arm to sample each sampling point according to the sampling sequence.
4. The pharyngeal swab sampling method of claim 2, wherein the determining the inlet angle of the robotic arm based on the target image and the position information of the sampling points comprises:
determining a sampling port plane and determining the position information of the sampling port plane in the three-dimensional coordinate system;
determining depth information from the sampling point to the sampling port plane according to the sampling point and the position information of the sampling port plane;
determining an entrance angle of the mechanical arm relative to the sampling point according to the target image and the depth information;
controlling the mechanical arm to sample according to the sampling operation information, and the method comprises the following steps:
and controlling the mechanical arm to rotate to a target position according to the inlet angle and then to enter the cavity to sample the sampling point.
5. The pharyngeal swab sampling method according to claim 2, wherein the motion information comprises a rotation angle of the mechanical arm when the mechanical arm samples a target sampling point, and the determining the motion information of the mechanical arm at each sampling point according to the target image, the sampling point and the position information of the sampling area comprises:
determining a target sampling area corresponding to the target sampling point according to the sampling point and the position information of the sampling area;
determining a rotation angle of the mechanical arm when the mechanical arm samples the target sampling point according to the target sampling area;
controlling the mechanical arm to sample according to the sampling operation information, and the method comprises the following steps:
and for each sampling point, controlling the mechanical arm to swing for a preset number of times according to the rotation angle so as to sample the sampling point.
6. The pharyngeal swab sampling method of claim 5, wherein the motion information further includes displacement information of the mechanical arm after the sampling of the target sampling point is completed, and the motion information of the mechanical arm at each sampling point is determined according to the target image, the sampling point and the position information of the sampling area, further comprising:
determining obstacle information in the target sampling region according to the target image;
determining displacement information of the mechanical arm after the sampling of the target sampling point is completed according to the obstacle information;
controlling the mechanical arm to sample according to the sampling operation information, and the method comprises the following steps:
and after sampling the target sampling point, controlling the mechanical arm to move the target distance according to the displacement information and then sampling the next sampling point.
7. The pharyngeal swab sampling method of claim 2, wherein the motion information includes a length of motion of the robotic arm at the time of sampling each sampling point, and the determining the motion information of the robotic arm at each sampling point from the target image, the sampling point, and the position information of the sampling area includes:
determining the size of the oral cavity according to the target image;
determining a plurality of auxiliary sampling points in a sampling area corresponding to the sampling points according to the oral cavity size, the sampling points and the position information of the sampling area;
determining a sampling arc according to the plurality of auxiliary sampling points;
determining a length of the sampling arc as the movement length;
controlling the mechanical arm to sample according to the sampling operation information, and the method comprises the following steps:
for each sampling point, controlling the mechanical arm to move along the corresponding sampling arc to sample.
8. A pharyngeal swab sampling device, applied to a sampling apparatus comprising a mechanical arm, comprising:
the acquisition unit is used for acquiring a target image, wherein the target image is a three-dimensional image of the interior of the oral cavity;
the processing unit is used for determining a plurality of sampling areas according to the target image and determining a sampling point in each sampling area;
the processing unit is also used for establishing a target coordinate system and determining the position information of each sampling point and each sampling area in the three-dimensional coordinate system;
the processing unit is further configured to determine sampling operation information according to the target image and the position information, and control the mechanical arm to sample according to the sampling operation information.
9. A pharyngeal swab sampling device, comprising:
a memory storing programs or instructions;
a processor which when executed by said program or instructions carries out the steps of a pharyngeal swab sampling method according to any one of claims 1 to 7.
10. A sampling device, comprising:
a mechanical arm;
a pharyngeal swab sampling device according to claim 9, said robotic arm being controlled by said pharyngeal swab sampling device to sample.
11. The sampling device of claim 10, further comprising:
and the distance measuring device is used for acquiring the three-dimensional image information of the interior of the oral cavity of the person to be sampled.
12. A readable storage medium having a program or instructions stored thereon, which when executed by a processor, performs the steps of the pharyngeal swab sampling method of any one of claims 1 to 7.
13. A computer program product comprising a computer program, wherein the computer program when executed by a processor implements the steps of a pharyngeal swab sampling method as claimed in any one of claims 1 to 7.
CN202211059856.7A 2022-08-31 2022-08-31 Pharynx swab sampling method and device, sampling equipment and computer program product Pending CN115414072A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211059856.7A CN115414072A (en) 2022-08-31 2022-08-31 Pharynx swab sampling method and device, sampling equipment and computer program product

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211059856.7A CN115414072A (en) 2022-08-31 2022-08-31 Pharynx swab sampling method and device, sampling equipment and computer program product

Publications (1)

Publication Number Publication Date
CN115414072A true CN115414072A (en) 2022-12-02

Family

ID=84200374

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211059856.7A Pending CN115414072A (en) 2022-08-31 2022-08-31 Pharynx swab sampling method and device, sampling equipment and computer program product

Country Status (1)

Country Link
CN (1) CN115414072A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115922725A (en) * 2022-12-28 2023-04-07 山东大学 Positioning system of throat swab sampling robot
CN116109982A (en) * 2023-02-16 2023-05-12 哈尔滨星云智造科技有限公司 Biological sample collection validity checking method based on artificial intelligence

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115922725A (en) * 2022-12-28 2023-04-07 山东大学 Positioning system of throat swab sampling robot
CN115922725B (en) * 2022-12-28 2024-03-22 山东大学 Throat swab sampling robot positioning system
CN116109982A (en) * 2023-02-16 2023-05-12 哈尔滨星云智造科技有限公司 Biological sample collection validity checking method based on artificial intelligence

Similar Documents

Publication Publication Date Title
CN115414072A (en) Pharynx swab sampling method and device, sampling equipment and computer program product
CN112099493B (en) Autonomous mobile robot trajectory planning method, system and equipment
CN111427361B (en) Recharging method, recharging device and robot
CN105090082B (en) Control method for fan and device
JP4736607B2 (en) Robot controller
CN1226013C (en) Machine cleaner, system using same and method for reconnecting charging device to outside
CN107710283A (en) A kind of filming control method, device and control device
EP3223101B1 (en) Automatic operation vehicle
WO2023179491A1 (en) Battery grabbing and positioning method, apparatus and device
CN110575386B (en) Control method, device and equipment of moxibustion manipulator
CN110374312A (en) Handling system and its control method, floor tile paving system
EP3223102B1 (en) Automatic operation vehicle
CN116400639A (en) PLC (programmable logic controller) collected data intelligent cleaning method and system
CN111618844A (en) Robot system and control method
WO2023070919A1 (en) Product movement track calculation method and apparatus, device, and storage medium
CN110871440A (en) Method and device for controlling robot running
CN108958249B (en) Ground robot control system and method considering unknown control direction
JP6644846B1 (en) Work position and orientation recognition device and picking system
CN109916062B (en) Cabinet air conditioner based on image measurement
CN110244710A (en) Automatic Track Finding method, apparatus, storage medium and electronic equipment
JPH06270083A (en) Work position detector
CN115797332B (en) Object grabbing method and device based on instance segmentation
JP3151906B2 (en) Handling method in robot system
EP4059403A1 (en) Method and apparatus for controlling cleaning device
CN111958597B (en) Method for controlling autonomous obstacle crossing process of mobile robot

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination