CN113100835A

CN113100835A - Human body physiological sample collecting system

Info

Publication number: CN113100835A
Application number: CN202110403126.3A
Authority: CN
Inventors: 孙喜琢; 宫芳芳; 覃金洲; 曾舒怡
Original assignee: Shenzhen Luohu Hospital Group
Current assignee: Shenzhen Luohu Hospital Group
Priority date: 2021-04-14
Filing date: 2021-04-14
Publication date: 2021-07-13
Anticipated expiration: 2041-04-14
Also published as: CN113100835B

Abstract

The application discloses human physiology sample collection system, this collection system includes: a collection device; the device comprises a mechanical arm, wherein the tail end of the mechanical arm is provided with a clamping device and a laser radar, the clamping device is used for clamping an acquisition device, and the laser radar is used for sending a radar signal to the direction pointed by the tail end of the mechanical arm and receiving returned laser radar data; the control device is used for acquiring an image to be positioned of a target organ containing an object to be acquired according to the laser radar data; determining the position information of a target organ according to the image to be positioned; moving the acquisition device to the target organ according to the position information; and controlling the acquisition device to move from the target organ to the target position of the object to be acquired so as to acquire the physiological sample of the target position.

Description

Human body physiological sample collecting system

Technical Field

The invention relates to the technical field of human body physiological sample collection, in particular to a human body physiological sample collection system.

Background

In China, the prevention and control of new crown epidemic situation is effectively controlled, but the epidemic situation is still serious abroad, and the risk of outbreak in local areas in China still exists. Nucleic acid inspection is as the gold standard of appraising new coronavirus, relies on the representativeness and the accuracy of nucleic acid sampling sample, can obtain a large amount of samples through putting into large-scale first-line medical personnel and concentrating manual sampling, but its input human cost is high, and medical personnel quality control is more difficult to control, also has the risk of high risk exposure infection simultaneously.

Therefore, there is a need for an automatic collection system for human physiological samples.

Disclosure of Invention

In order to solve the above problems existing in the prior art, the embodiment of the application provides a human body physiological sample collecting system, which can realize automatic collection of human body physiological samples, reduce the consumption of labor cost and improve the collecting efficiency.

In a first aspect, an embodiment of the present application provides a human body physiological sample collecting system, including:

a collection device;

the device comprises a mechanical arm, wherein the tail end of the mechanical arm is provided with a clamping device and a laser radar, the clamping device is used for clamping an acquisition device, and the laser radar is used for sending a radar signal to the direction pointed by the tail end of the mechanical arm and receiving returned laser radar data;

the control device is used for acquiring an image to be positioned of a target organ containing an object to be acquired according to the laser radar data; determining the position information of a target organ according to the image to be positioned; moving the acquisition device to the target organ according to the position information; controlling the acquisition device to move from the target organ to the target position of the object to be acquired, and respectively acquiring acting force and torque of the head of the acquisition device in a first direction, a second direction and a third direction in the process of moving to the target position of the object to be acquired, wherein the first direction is the advancing direction of the acquisition device, and the first direction, the second direction and the third direction are mutually vertical; and controlling the acquisition device to reach the target position of the object to be acquired according to the reaction force of the head of the acquisition device in the first direction, the torque of the head of the acquisition device in the opposite direction of the first direction, the acting force and the torque of the head of the acquisition device in the second direction and the acting force and the torque of the head of the acquisition device in the third direction so as to acquire the physiological sample of the target position.

In a second aspect, an embodiment of the present application provides a human body physiological sample collecting method, which is applied to a human body physiological sample collecting system, the human body physiological sample collecting system includes a collecting device, a mechanical arm and a control device, a clamping device and a laser radar are arranged at the end of the mechanical arm, the clamping device is used for clamping the collecting device, and the collecting method includes:

the laser radar sends radar signals to the direction pointed by the tail end of the mechanical arm and receives returned laser radar data;

the control device acquires an image to be positioned of a target organ containing an object to be acquired according to the laser radar data;

determining the position information of a target organ according to the image to be positioned;

moving the acquisition device to the target organ according to the position information;

controlling the acquisition device to move from the target organ to the target position of the object to be acquired, and respectively acquiring acting force and torque of the head of the acquisition device in a first direction, a second direction and a third direction in the process of moving to the target position of the object to be acquired, wherein the first direction is the advancing direction of the acquisition device, and the first direction, the second direction and the third direction are mutually vertical;

and controlling the acquisition device to reach the target position of the object to be acquired according to the reaction force of the head of the acquisition device in the first direction, the torque of the head of the acquisition device in the opposite direction of the first direction, the acting force and the torque of the head of the acquisition device in the second direction and the acting force and the torque of the head of the acquisition device in the third direction so as to acquire the physiological sample of the target position.

In a third aspect, an embodiment of the present application provides an electronic device, including: a processor coupled to the memory, the memory for storing a computer program, the processor for executing the computer program stored in the memory to cause the electronic device to perform the method as in the first aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium storing a computer program, the computer program causing a computer to perform the method as in the first aspect.

In a fifth aspect, embodiments of the present application provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program, the computer operable to cause the computer to perform a method as in the first aspect.

The embodiment of this application provides a collection system that can realize human physiology sample's automatic acquisition, and it realizes treating the accurate discernment of the target organ of gathering the object through laser radar, realizes the automatic acquisition to human physiology sample through the arm, alleviates the human cost consumption, promotes collection efficiency, simultaneously, has reduced the probability that the collection personnel infected.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic view of a human body physiological sample collecting system according to an embodiment of the present disclosure;

fig. 2 is a block diagram of functional modules of a control device according to an embodiment of the present disclosure;

fig. 3 is a schematic flow chart of a human body physiological sample collecting method according to an embodiment of the present disclosure;

fig. 4 is a flowchart illustrating a method for determining nose position information according to a face image according to an embodiment of the present application;

FIG. 5 is a schematic flow chart illustrating another method for determining nose position information from a facial image according to an embodiment of the present disclosure;

fig. 6 is a schematic flowchart of a further method for determining nose position information according to a face image according to an embodiment of the present application;

fig. 7 is a flowchart illustrating a method for determining coordinate information of nostrils according to position information of a nose according to an embodiment of the present disclosure;

FIG. 8 is a schematic view of three directions provided by an embodiment of the present application;

FIG. 9 is a schematic view of a swab in three directions as it is advanced through the nose according to embodiments of the present disclosure;

FIG. 10 is a schematic view showing the three-directional forces applied when the head of a swab reaches the nasopharynx according to the present embodiment;

FIG. 11 is a schematic view of another human physiological sample collection system according to an embodiment of the present disclosure;

fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art without any inventive work based on the embodiments in the present application are within the scope of protection of the present application.

The terms "first," "second," "third," and "fourth," etc. in the description and claims of this application and in the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

Reference herein to "an embodiment" means that a particular feature, result, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.

First, the human body physiological sample collection system according to the present application can be applied to collection scenes such as a nasopharyngeal swab, an ear swab, an anal swab, and a vaginal swab. In the present application, the collection scene of the nasopharyngeal swab is mainly taken as an example to illustrate the human body physiological sample collection system, and the human body physiological sample collection system in other scenes is similar to the human body physiological sample collection system in the collection scene of the nasopharyngeal swab, and is not repeated here.

The following description will be made of a specific implementation of the collection of a nasopharyngeal swab, and therefore, in this embodiment, the collection device mentioned later will be described as a swab sampling tube, the target organ will be described as a nose, the target site will be described as a nasopharyngeal portion, and the physiological sample will be described as a cell sample of the nasopharyngeal portion, for example: epithelial cells, the human body part will be described with the face and the entry location will be described with the nostrils.

Referring to fig. 1, fig. 1 is a schematic structural diagram of a human body physiological sample collecting system according to an embodiment of the present application, the human body physiological sample collecting system including: a swab sampling tube 101, a robotic arm 102, and a control device 103.

In this embodiment, swab sampling tube 101 includes sampling tube 106, swab 107, and cover 108. Wherein the sampling tube contains preserving fluid and swab 107, and cover 108 covers the opening of sampling tube 101.

In the present embodiment, the holding device 104 is provided at the end of the robot arm 102, and the six-axis force sensor is attached to the holding device 104, and is capable of directly measuring X, Y, Z three-directional forces and Rx, Ry, and Rz three-directional torques in a cartesian coordinate system constructed with the head of the swab held by the holding device 104 as the center when the human body physiological sample collection system is sampling.

Meanwhile, the holding device 104 can complete a series of activities such as uncapping the swab sampling tube 101, clamping the swab 107, sampling, storing and covering the sample. Specifically, the holding device can grip the cover 108 of the swab sampling tube 101 and rotate to remove the cover 108, grip the swab 107 into the nasal cavity and sample it at the nasopharynx, then exit the nasal cavity straight, place the swab 107 into the sampling tube 106, and finally break the swab and tighten the cover 108.

Meanwhile, in an alternative embodiment, a single-axis force sensor may be installed in each joint of the robot arm 102 to form a multi-axis force-controlled robot arm, for example, when the robot arm 102 has seven joints, a seven-axis force-controlled robot arm is formed. Thus, force data of the head of a swab held by the holding device 104 may be calculated by the robotic arm itself modeling.

In this embodiment, the end of the mechanical arm 102 is further provided with a laser radar 105, and the laser radar 105 can continuously scan and collect environmental data after the human body physiological sample collection system is started, for example, send a radar signal to the direction pointed by the end of the mechanical arm 102, and receive the returned laser radar data.

In this embodiment, the control device 103 may be a remote control host, and is remotely connected to the robot arm 102 via a network. As shown in fig. 2, the remote control host may be loaded with a control and communication system, a high precision optical calibration system, a visual positioning and servo system, a motion planning system, and a force control servo system. The control device 103 is used for receiving laser radar data of the laser radar 105 and controlling the mechanical arm 102 to complete nasopharynx body fluid collection.

For example, the six-axis force sensor carried by the holding device 104 can transmit the force data of the held swab 107 during the advancing process in the nasal cavity to the force control servo system in the remote control host, and the force control servo system adjusts the position and the depth of the swab 107 in real time according to the mechanical characteristics of the swab 107 sampling in the nasal cavity, so that the human body physiological sample collection system can complete the sampling flexibly and safely.

The laser radar 105 can transmit the received laser radar data to a high-precision optical calibration system in the remote control host, so that the high-precision optical calibration system can be assisted to calibrate a nose, a swab and the like. Meanwhile, the laser radar 105 may also transmit the received laser radar data to a visual positioning and servo system in the remote control host, so that the visual positioning and servo system automatically detects a human face in real time and identifies a nose, thereby controlling the mechanical arm 102 to position the head of the swab 107 in the center of the nostril.

Specifically, as shown in fig. 3, in terms of controlling the mechanical arm 102 to complete the nasopharyngeal fluid collection, the control device 103 is specifically configured to perform the following operations:

301: an image to be located containing the nose of the subject to be acquired is acquired.

In this embodiment, the image to be positioned may be a facial image of the object to be collected, and is obtained by a laser radar installed at the end of the mechanical arm. Specifically, first, the acquisition environment may be scanned by a laser radar, and spatial position information and attitude information of an object to be acquired are determined. The spatial position information is used to represent the position of the object to be acquired in the acquisition environment, and the posture information is used to represent the current posture of the object to be acquired, for example: standing or sitting.

Then, according to the spatial position information and the posture information, the mechanical arm holding the swab is controlled to move towards the face of the object to be collected, in the moving process, the laser radar which is also located at the tail end of the mechanical arm continuously sends radar signals to the object to be collected, and first laser radar data returned from the object to be collected are received.

And finally, determining a face image containing the nose of the object to be acquired according to the first laser radar data.

302: and determining the position information of the nose according to the image to be positioned.

In this embodiment, in the process of controlling the robot arm holding the swab to move toward the face of the object to be collected, the laser radar also located at the end of the robot arm continuously transmits a radar signal to the object to be collected, and receives the first laser radar data returned from the object to be collected. Therefore, after the first laser radar data is processed, a group of face images containing the nose of the object to be collected can be obtained. Therefore, in the embodiment, the position information of the nose can be dynamically updated by continuously analyzing the latest acquired face image in real time, so that the positioning is more accurate.

Meanwhile, as the distance between the mechanical arm and the object to be collected is closer and closer, the proportion of the face in the collected face image is larger when the mechanical arm is closer to the object to be collected. When a certain threshold distance is reached, only a partial image of the face containing the nose may be acquired. In view of the above, the present embodiment proposes the following solutions, which may analyze the face image in different ways according to the completeness of the obtained face image and the missing reason of the missing part when the face image is incomplete, so as to determine the position information of the nose, specifically as follows:

(1) when the face image is a complete image of a face including a nose, referring to fig. 4, the method of determining position information of a nose from the face image may include:

401: from the face image, a plurality of first anatomical landmarks of the face are determined.

Illustratively, feature extraction may be performed on the facial image via a CNN neural network to determine a plurality of first anatomical landmarks of the face based on the extracted image features.

402: and performing organ segmentation processing on the face according to the plurality of first anatomical calibration points to obtain a processing result.

For example, the face may be segmented according to the definition of the anatomical partition of the face in the anatomy and the plurality of first anatomical calibration points, and the segmented plurality of regions may be used as the processing result.

403: according to the processing result, the position information of the nose is determined.

For example, the area where the nose is located may be determined among the obtained plurality of areas, and the position information of the area where the nose is located in the face image may be used as the position information of the nose.

(2) When the face image is a partial image including a nose and the reason why the face image is incomplete is that a complete face image cannot be acquired because the mechanical arm is close to the object to be acquired, referring to fig. 5, the method for determining the position information of the nose according to the face image may include:

501: and acquiring second laser radar data containing a complete face in the process that the acquisition device moves towards the object to be acquired.

In the present embodiment, when an incomplete face image is detected for the first time, the last laser radar data may be directly acquired as the second laser radar data including the complete face.

502: and acquiring a first image according to the second laser radar data.

In this embodiment, the second lidar data may be optically imaged to obtain a corresponding first image.

503: from the first image, a plurality of first anatomical landmarks of the face are determined.

In this embodiment, the processing method is similar to the processing method in step 401, and is not described herein again.

504: for each first anatomical calibration point of the plurality of first anatomical calibration points, first color information and first texture information of a first region corresponding to each first anatomical calibration point are respectively obtained.

In this embodiment, the first region may be a circular region determined at a preset radius from each of the first anatomical calibration points. Of course, other shaped regions centered around each first anatomical landmark are also possible, and the present application is not limited thereto.

Meanwhile, in the present embodiment, the first texture information of the first region corresponding to each first anatomical calibration point may be determined by a structured light sensor mounted on the mechanical arm, and the first color information of the first region corresponding to each first anatomical calibration point may be determined by an RGBD sensor also mounted on the end of the mechanical arm.

505: and acquiring second color information and second texture information of a second region corresponding to each pixel point in the face image.

In this embodiment, the method for determining the second region is similar to the method for determining the first region in step 504, and is not described herein again. Taking a circular area as an example, when the second area is determined, the radius may be enlarged by a corresponding ratio according to the ratio of the area occupied by the image to be positioned in the first image.

Likewise, in this embodiment, the manner of obtaining the second color information and the second texture information is similar to the manner of obtaining the first color information and the first texture information in step 504, and is not repeated herein.

506: and determining a first matching value between the first color information of the first area corresponding to each first anatomical calibration point and the second color information of the second area corresponding to each pixel point, and a second matching value between the first texture information of the first area corresponding to each first anatomical calibration point and the second color information of the second area corresponding to each pixel point.

For example, feature extraction may be performed on the first color information of the first region corresponding to each first anatomical calibration point to obtain a first color vector. And performing feature extraction on the second color information of the second area corresponding to each pixel point to obtain a second color vector. And then, calculating a first similarity between the first color vector and the second color vector, and determining a first matching value between the first color information of the first region corresponding to each first anatomical calibration point and the second color information of the second region corresponding to each pixel point.

Similarly, feature extraction may be performed on the first texture information of the first region corresponding to each first anatomical calibration point to obtain a first texture vector. And performing feature extraction on the second texture information of the second area corresponding to each pixel point to obtain a second texture vector. And then calculating a second similarity between the first texture vector and the second texture vector, and determining a second matching value between the first texture information of the first region corresponding to each first anatomical calibration point and the second texture information of the second region corresponding to each pixel point.

507: and under the condition that the first matching value between each first anatomical calibration point and the pixel point A is greater than a first threshold value and the second matching value between each first anatomical calibration point and the pixel point A is greater than a second threshold value, taking the pixel point A as a second anatomical calibration point corresponding to each first anatomical calibration point.

In this embodiment, the pixel point a is any one pixel point in the face image.

In addition, in an alternative embodiment, the first depth information of the first region corresponding to each first anatomical calibration point and the second depth information of the second region corresponding to each pixel point may also be determined by a ToF sensor mounted at the end of the mechanical arm. And performing feature extraction on the first depth information of the first region corresponding to each first anatomical calibration point to obtain a first depth vector. And performing feature extraction on the second depth information of the second area corresponding to each pixel point to obtain a second depth vector. And then, calculating a third similarity between the first depth vector and the second depth vector, and determining a third matching value between the first depth information of the first region corresponding to each first anatomical calibration point and the second depth information of the second region corresponding to each pixel point.

Therefore, under the condition that the first matching value between each first anatomical calibration point and the pixel point A is larger than the first threshold, the second matching value between each first anatomical calibration point and the pixel point A is larger than the second threshold, and the third matching value between each first anatomical calibration point and the pixel point A is larger than the third threshold, the pixel point A is used as the second anatomical calibration point corresponding to each first anatomical calibration point, and the matching precision of the second anatomical calibration point is improved.

508: and obtaining a plurality of second anatomical calibration points in the face image according to the second anatomical calibration points corresponding to each first anatomical calibration point.

509: and performing organ segmentation processing on the face according to the plurality of second anatomical calibration points in the face image to obtain a processing result.

In this embodiment, the processing method is similar to the processing method in step 402, and is not described herein again.

510: according to the processing result, the position information of the nose is determined.

In this embodiment, the processing method is similar to the processing method in step 403, and is not described herein again.

Furthermore, it should be noted that, in the present embodiment, when an incomplete face image is detected for the first time, the first image may be used as a reference image, a plurality of second anatomical calibration points in the face image may be determined based on the reference image, and in the process of detecting an incomplete face image each time after the incomplete face image is detected for the first time, the face image including the plurality of second anatomical calibration points obtained in the previous processing may be used as a reference image for the current processing, and the current processing may be performed, so that it is not necessary to acquire a complete face image again by using laser radar data and perform identification processing of the anatomical calibration points, and processing efficiency is improved.

(3) When the facial image is a partial image containing a nose and the reason for the incomplete facial image is that the mechanical arm is further moved away from the object to be acquired, so that the swab located at the end of the mechanical arm together with the lidar shields the face, resulting in a complete facial image that cannot be acquired, referring to fig. 6, the method for determining the position information of the nose according to the facial image may include:

601: and acquiring second laser radar data containing a complete face in the process that the acquisition device moves towards the object to be acquired.

In this embodiment, when the blocked face image is detected for the first time, the last laser radar data may be directly acquired as the second laser radar data including the complete face.

602: and acquiring a first image according to the second laser radar data.

In this embodiment, the processing method is similar to the processing method in step 502, and is not described herein again.

603: and carrying out finite element simulation on the shielded part in the face image according to the first image so as to complement the shielded part in the face image.

Specifically, the human body physiological sample collection method provided by the application maintains a standard human face model. And by stretching the first image, the anatomical calibration point of the standard human face is superposed with the anatomical calibration point of the first image. And meanwhile, acquiring finite element energy in the pulling process, and determining that the stretching is finished when the energy corresponding to the pulling deformation is minimum. The finite element energy is shown in formula (I):

E＝(R(a',b',c')-(a,b,c))………①

wherein, R is rigid transformation, (a ', b ', c ') represents any three anatomical calibration points in the supplemented face image, and (a, b, c) represents three corresponding anatomical calibration points of the any three anatomical calibration points on the standard face model.

604: from the completed face image, a plurality of first anatomical landmark points of the face are determined.

605: and performing organ segmentation processing on the face according to the plurality of first anatomical calibration points to obtain a processing result.

606: according to the processing result, the position information of the nose is determined.

It should be noted that, when performing the first finite element simulation, the first image may be used as a reference image, the finite element simulation may be performed on the portion shielded in the face image based on the reference image, and in the process of each finite element simulation after the first finite element simulation, the face image obtained by the last finite element simulation may be used as a reference image for the current finite element simulation, so as to perform the current sub-finite element simulation, thereby avoiding the need of acquiring a complete face image through the lidar data again, and improving the simulation efficiency.

303: the swab is moved to the position of the nose based on the position information.

In the present embodiment, after the position information of the nose is determined, a spatial coordinate system may be established with the head of the swab as the origin, and the spatial coordinates of the nose in the coordinate system may be determined from the position information of the nose. A path of movement is then generated for movement toward the nose, such that the robotic arm can move the swab to the location of the nose along the path of movement. For example, the mechanical arm can be controlled to ascend or descend by calculating the vertical distance and the horizontal distance between the spatial coordinates of the nose and the origin, so that the swab held by the mechanical arm is positioned on the same horizontal line with the nose, and the swab is translated to the position of the nose.

304: the swab is controlled to move from the nose towards the nasopharynx of the subject to be collected.

In this embodiment, the coordinate information of the nostrils may be determined according to the position information of the nose, so as to control the collecting device to enter the nose from the nostrils, thereby moving towards the nasopharynx of the object to be collected.

Specifically, the present application provides a method of determining coordinate information of nostrils according to position information of a nose, as shown in fig. 7, the method including:

701: and determining the center coordinates of the nose according to the position information.

702: at least four third anatomical landmarks are determined among the plurality of first anatomical landmarks.

In this embodiment, the distance between each of the at least four third anatomical calibration points and the center coordinate meets a preset condition. For example, the four first anatomical landmarks closest to the center coordinate are taken as the third anatomical landmarks. Meanwhile, if the nearest first anatomical landmark is not satisfactory, the observable nearest first anatomical landmark may be used as the third anatomical landmark.

703: and establishing a barycentric coordinate system (barycentric coordinate system) according to the at least four third anatomical calibration points, and determining the coordinates of each third anatomical calibration point in the barycentric coordinate system to obtain at least four barycentric coordinates which are in one-to-one correspondence with the at least four third anatomical calibration points.

704: and determining coordinate information of the entry position according to the at least four barycentric coordinates.

Specifically, through a barycentric coordinate system established by four points in the space, the weighted values of the coordinates of the four points can be used to identify the coordinates of any one point in the space, as shown in formula (ii):

x＝w1×x1+w2×x2+w3×x3+w4×x4………②

wherein, x1, x2, x3 and x4 are barycentric coordinates of four points establishing a barycentric coordinate system, w1, w2, w3 and w4 are weights, and w1, w2, w3 and w4 satisfy formula (c):

w1+w2+w3+w4＝1………③

305: respectively acquiring the acting force and the torque of the swab head in the first direction, the second direction and the third direction in the process of moving the swab to the nasopharynx of the object to be collected.

In this embodiment, the first direction is a direction in which the collecting device advances, and the first direction, the second direction and the third direction are perpendicular to each other. Specifically, the second direction is different from the third direction. It will be appreciated that in this embodiment, a cartesian coordinate system may be established with reference to the swab pose. Specifically, when the head of the swab reaches the central point of the tissue or organ as an origin, the second direction may be a rightward direction perpendicular to the first direction, and the third direction may be a downward direction perpendicular to the first direction; alternatively, the second direction may be a leftward direction perpendicular to the first direction, and the third direction may be an upward direction perpendicular to the first direction, which is not limited herein.

Exemplarily, referring to fig. 8, fig. 8 is a schematic diagram of three directions provided by the embodiment of the present application. As shown in fig. 8, at the nasal cavity entrance, with the head of the swab reaching the nose center point as an origin, the first direction (positive direction of the z-axis) is a swab advance direction, the second direction (positive direction of the x-axis) may be a right direction perpendicular to the first direction, and the third direction (positive direction of the y-axis) may be a downward direction perpendicular to the first direction.

In fig. 8 to 10, the positive direction of the x axis is the second direction, the positive direction of the y axis is the third direction, and the positive direction of the z axis is the first direction.

In this embodiment, the robot arm may be a six-axis robot arm or a seven-axis robot arm. Therefore, if the robot is a six-axis robot, the robot end effector may be mounted with an end-type six-axis force sensor, and based on this, step 305 may include: the acting force and the torque of the head of the swab in the first direction, the second direction and the third direction when the swab advances in the tissue or organ of the object to be collected are respectively obtained through a terminal six-axis force sensor.

If the mechanical arm is a seven-axis mechanical arm, the acting force and the torque of the head of the swab in the first direction, the second direction and the third direction meet a formula (IV):

w＝(J^T)^-1τ………④

wherein w comprises the vectors of forces and torques in the first, second and third directions of the swab head, J^TFor the transpose of the jacobian, τ comprises the vector of the moment of each axis of the mechanical arm.

Where τ comprises the vector of the moment of each axis of the robot arm, it can be understood as: τ comprises the vector of the moments for each of the seven axes of the robotic arm.

Illustratively, in the present application, a force sensor is mounted on each axis of a seven-axis robotic arm.

Further, w may be represented by the formula (v):

wherein x is the acting force in the second direction, y is the acting force in the third direction, z is the acting force in the first direction, and gamma_xIs a torque in a second direction, r_yIs a torque in a third direction, r_zIs a torque in a first direction.

Further, τ may be expressed by the formula:

wherein, tau₁To tau₆The vector of the moment of each axis of the robot arm.

In an alternative embodiment, prior to step 305, the method further comprises: adjusting the advancing direction of the swab in the nose according to the acting force and the torque of the head of the swab in the second direction and the acting force and the torque of the head of the swab in the third direction, so as to obtain the advancing direction of the swab in the nose after adjustment; the swab is controlled to advance in the nose according to the adjusted direction of advancement of the swab in the nose.

In the above technical solution, the direction in which the swab advances through the nose is adjusted according to the force and torque of the swab head in the second direction and the force and torque of the swab head in the third direction, so that the direction in which the swab head advances through the nose can be adjusted in real time when a machine is used for sampling. Meanwhile, the swab is controlled to move forwards in the nose according to the direction of the adjusted swab moving forwards in the nose, so that the situation of mistaken sampling when the target position in the tissue or organ is not reached is avoided.

Illustratively, the adjusting the advancing direction of the swab in the organ according to the acting force and the torque of the swab head in the second direction and the acting force and the torque of the swab head in the third direction, and the adjusted advancing direction of the swab in the nose comprises: when the first preset condition is met, the advancing direction of the swab in the nose is adjusted according to the second direction and/or the reverse direction of the third direction in the first preset condition, and the advancing direction of the adjusted swab in the nose is obtained.

Wherein the first preset condition comprises one or more of the following: the acting force of the swab head in the second direction is greater than or equal to a first threshold value; the torque of the swab head in the second direction is greater than or equal to a second threshold; the acting force of the swab head in the third direction is greater than or equal to a third threshold value; the torque of the swab head in the third direction is greater than or equal to a fourth threshold.

The first threshold, the second threshold, the third threshold, and the fourth threshold may all be the same, or partially the same, or all different, and are not limited herein.

Illustratively, referring to fig. 9, fig. 9 is a schematic view of a swab of the present application in three directions as the swab advances through a nose. As shown in fig. 9, it can be seen that when the swab head is advanced into the nasal cavity, the swab head has a lateral force, i.e., the swab head is subjected to lateral pressure by the nasal cavity, the position determining means can determine the force and torque of the lateral force in the first, second and third directions.

It can be seen that, according to the technical scheme, the direction of the head of the swab advancing in the nose can be adjusted in real time based on the acting force in the second direction and/or the third direction when the swab is used for sampling. Meanwhile, by adjusting the direction in which the head of the swab advances in the nose based on the reverse direction of the second direction and/or the third direction, a situation in which a wrong sampling is performed when the target position in the tissue or organ is not reached is avoided.

306: controlling the swab to reach the nasopharynx of the object to be collected according to the reaction force of the swab head in the first direction, the torque of the swab head in the direction opposite to the first direction, the acting force and the torque of the swab head in the second direction, and the acting force and the torque of the swab head in the third direction, so as to obtain a cell sample at the nasopharynx as a physiological sample.

In this embodiment, when the second predetermined condition is satisfied, the swab can be determined to reach the target location in the tissue or organ. Specifically, the second preset condition may include: the reaction force of the swab head in the first direction is greater than or equal to a fifth threshold value, the torque of the swab head in the direction opposite to the first direction is greater than or equal to a sixth threshold value, the force of the swab head in the second direction is less than or equal to a seventh threshold value, the torque of the swab head in the second direction is less than or equal to an eighth threshold value, the force of the swab head in the third direction is less than or equal to a ninth threshold value, and the torque of the swab head in the third direction is less than or equal to a tenth threshold value.

The fifth threshold may be the same as or different from the sixth threshold, and is not limited herein.

The seventh threshold, the eighth threshold, the ninth threshold, and the tenth threshold may all be the same or partially the same or all different, and are not limited herein.

Illustratively, referring to fig. 10, fig. 10 is a schematic diagram illustrating forces applied in three directions when the head of a swab reaches the nasopharynx according to an embodiment of the present application. When the swab head is subjected to a counter force in a first direction, as shown in fig. 10, it is determined that the swab head has reached the nasopharynx.

Thus, by determining that the swab is obstructed in the forward direction, i.e., the counter force in the first direction and the torque in the counter direction to the first direction are greater than or equal to the threshold value, and the forces and torques in the other directions are not applied, it is possible to determine that the head of the swab has reached the target position in the tissue or organ.

In an alternative embodiment, as shown in fig. 11, the human physiological sample collecting system further includes: a storage station 109, a disinfection device 110 and a sampling station 111.

In this embodiment, the storage station 109 is used to store the swab sampling tube 101; the disinfecting device 110 can be an infrared automatic spray disinfecting system, and is used for disinfecting the mechanical arm 102, specifically, before the swab sampling tube 101 is clamped by the clamping device 104 at the end of the mechanical arm 102, the control device 103 controls the disinfecting device 110 to disinfect the mechanical arm 102, so that cross infection is avoided, and sampling safety is improved; the sampling station 111 is used to carry the robot arm 102, the storage station 106 and the disinfection device 107.

In an alternative embodiment, the remote control host may be a high-performance computer, connected to the various components via a high-performance network. The remote control host is provided with a plurality of subsystems such as a high-precision optical calibration system, a motion planning system, a visual positioning and servo system, a force control servo system and a control and communication system. Wherein the visual positioning and servo system processes 30 frames per second. The force control servo system processes 1000 times per second. The motion planning system of the system plans 1000 times per second.

Specifically, the high-precision optical calibration system comprises mechanical arm hand-eye calibration and swab calibration. The mechanical arm hand-eye calibration is calibrated once when the human physiological sample collection system is started, the influence of theoretical output and actual output errors of the mechanical arm 102 on system calculation is reduced through kinematics calibration and dynamics calibration of the mechanical arm 102, and the operation precision, safety and service life of the mechanical arm 102 are improved. The swab calibration is performed after the swab 107 is clamped every time, and the mechanical arm 102 moves to the open position after clamping the swab to perform automatic calibration under the assistance of the laser radar 105, so that the calculation error caused by the production difference of the swab 107 is reduced, and the precision and the safety of the body physiological sample acquisition system are improved.

In this embodiment, the motion planning system is configured to calculate an optimal obstacle avoidance motion path from a starting point to a target point, and specifically, has the following functions:

(1) the mechanical arm 102 is accurately and safely operated to a fixed position under the infrared automatic spraying sterilization system for sterilization;

(2) moving the gripping device 104 to a secured position aligns the cover 108 of the swab sampling tube 101, rotating the cover open in the correct direction, and placing the cover 108 in a predetermined position;

(3) returning the gripping device 104 to the uncapped sampling tube 106, and gripping the swab 107 through the opening of the sampling tube 106 to a fixed point for calibration;

(4) rotating the swab 107 held by the holding device 104 in a predetermined direction by a predetermined angle in the nasopharynx to perform sampling;

(5) after sampling, the holding device 104 holds the swab 107 to exit the nasal cavity;

(6) moving the swab 107 held by the holding device 104 over the sampling tube 106, placing the head of the swab vertically down through the opening of the sampling tube 106 into the sampling tube 106 to a fixed depth, breaking the swab 107 by rotating it horizontally by 90 °, and placing the broken swab stick in a fixed position;

(8) the gripper 104 is moved to a predetermined position to grip the cap 108, return to above the sample tube 106, and rotate the cap 108 in a desired direction.

In this embodiment, the visual positioning and servo system is used for real-time face recognition and to guide the motion of the mechanical arm 102 so as to bring the head of the held swab 107 to the center of the nostril. In the process, if the human face of the object to be acquired moves, the vision positioning and servo system will automatically adjust the posture of the swab 107 and perform the positioning again, so that the positioning is accurate. Meanwhile, the visual positioning and servo system supports multi-task analysis, and comprises multiple functions of key point detection, organ segmentation, attitude estimation and the like.

In this embodiment, the force control servo system is used to position the nasopharynx of the object to be collected, and at the same time, accurately and safely control the mechanical arm 102 to hold the swab 107 to reach the nasopharynx for sampling. The force control servo system guides the swab 107 to automatically adjust the force in the whole inserting and sampling process according to the contact pressure of the swab 107 and the nasal cavity, so that the invasion feeling of a to-be-collected object is effectively reduced, and the damage to the to-be-collected object is reduced.

In the embodiment, the control and communication system is used for coordinating operation and data communication of all components in the whole human body physiological sample acquisition system.

It can be seen that, in the embodiment of the present application, the target organ of the object to be acquired is accurately positioned in a visual positioning manner, so as to realize that the acquisition device automatically moves to the target organ. Then, acting forces and torques of the head of the acquisition device in the first direction, the second direction and the third direction when the acquisition device moves to the target position of the object to be acquired in the target organ of the object to be acquired are acquired, so that the head of the acquisition device can be controlled to reach the target position according to the reaction force of the head of the acquisition device in the first direction, the torque of the head of the acquisition device in the opposite direction of the first direction, the acting force and the torque of the head of the acquisition device in the second direction and the acting force and the torque of the head of the acquisition device in the third direction, and the fact that the acquisition device reaches the target position can be accurately determined when the machine samples, so that the sampling accuracy can be increased, and the accuracy of downstream sample detection is guaranteed. Simultaneously, confirm collection system and reached the target location of tissue or organ through effort and the moment of torsion based on the equidirectional, avoided treating the adverse reaction that the collection object caused when the dynamics is too big, promoted user experience, considered the experience of treating the collection object when being sampled.

In addition, in this embodiment, a method for collecting a human body physiological sample is further provided, where the method is applicable to the above human body physiological sample collecting system, the human body physiological sample collecting system includes a collecting device, a mechanical arm and a control device, a clamping device and a laser radar are disposed at a tail end of the mechanical arm, the clamping device is used for clamping the collecting device, and the collecting method includes:

(1): the laser radar sends radar signals to the direction pointed by the tail end of the mechanical arm and receives returned laser radar data;

(2): the control device acquires an image to be positioned of a target organ containing an object to be acquired according to the laser radar data;

(3): determining the position information of a target organ according to the image to be positioned;

(4): moving the acquisition device to the target organ according to the position information;

(5): controlling the acquisition device to move from the target organ to the target position of the object to be acquired, and respectively acquiring acting force and torque of the head of the acquisition device in a first direction, a second direction and a third direction in the process of moving to the target position of the object to be acquired, wherein the first direction is the advancing direction of the acquisition device, and the first direction, the second direction and the third direction are mutually vertical;

(6): and controlling the acquisition device to reach the target position of the object to be acquired according to the reaction force of the head of the acquisition device in the first direction, the torque of the head of the acquisition device in the opposite direction of the first direction, the acting force and the torque of the head of the acquisition device in the second direction and the acting force and the torque of the head of the acquisition device in the third direction so as to acquire the physiological sample of the target position.

Referring to fig. 12, fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in fig. 12, the electronic device 1200 includes a transceiver 1201, a processor 1202, and a memory 1203. Connected to each other by a bus 1204. The memory 1203 is used for storing computer programs and data, and the data stored by the memory 1203 may be transferred to the processor 1202.

The processor 1202 is configured to read the computer program in the memory 1203 to perform the following operations:

In an embodiment of the present invention, before the obtaining, according to the lidar data, an image to be located of a target organ including an object to be acquired, the processor 1202 is specifically configured to:

determining an environment image containing the storage table through the laser radar;

determining the position of a collecting device in the storage table according to the environment image;

and moving the mechanical arm to the position of the collecting device in the storage table, and clamping the collecting device in the storage table through the clamping device.

In an embodiment of the present invention, before the acquiring, by the lidar, an image to be located of a target organ including an object to be acquired, the processor 1202 is specifically configured to:

determining, by the lidar, a position of a cover of the acquisition device;

moving the clamping device to the upper part of the cover so as to open the cover in a rotating mode and placing the cover at a preset position;

and moving the clamping device to the position above the opening of the collecting device after the cover is opened, and clamping the sampling rod in the collecting device.

In an embodiment of the present invention, before the determining the position of the storage station by the lidar, the processor 1202 is specifically configured to:

determining, by the lidar, a position of the disinfection device;

and moving the mechanical arm to the position of the disinfection device, and controlling the disinfection device to disinfect the mechanical arm.

In an embodiment of the present invention, after the acquiring device is controlled to reach the target position of the object to be acquired according to the reaction force of the acquiring device head in the first direction, the torque of the acquiring device head in the opposite direction of the first direction, the force and the torque of the acquiring device head in the second direction, and the force and the torque of the acquiring device head in the third direction, so as to acquire the physiological sample of the target position, the processor 1202 is specifically configured to:

moving the sampling rod to the position above the opening of the collecting device after the cover is opened, vertically and downwards putting the head of the sampling rod into the collecting device through the opening of the collecting device to a fixed depth, and horizontally rotating for 90 degrees to break the sampling rod;

and clamping the cover by the moving value of the clamping device at the preset position, returning to the position above the acquisition device, and screwing the cover in a rotating manner.

In an embodiment of the present invention, the end of the mechanical arm is installed with a laser radar, so that, in terms of acquiring an image to be positioned of a target organ including an object to be acquired, the processor 1202 is specifically configured to:

determining spatial position information and attitude information of an object to be acquired;

controlling the acquisition device to move towards the object to be acquired according to the spatial position information and the attitude information;

in the process of controlling the acquisition device to move, sending a radar signal to an object to be acquired through a laser radar, and receiving first laser radar data returned from the object to be acquired;

and acquiring an image to be positioned of a target organ containing the object to be acquired according to the first laser radar data.

Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention can be implemented by combining software and a hardware platform. With this understanding in mind, all or part of the technical solutions of the present invention that contribute to the background can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments or some parts of the embodiments.

Therefore, the present application also provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement part or all of the steps of any one of the human body physiological sample collection methods as set forth in the above method embodiments. For example, the storage medium may include a hard disk, a floppy disk, an optical disk, a magnetic tape, a magnetic disk, a flash memory, and the like.

Embodiments of the present application also provide a computer program product comprising a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any one of the human physiological sample collection methods as set forth in the above method embodiments.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are all alternative embodiments and that the acts and modules referred to are not necessarily required by the application.

In the above embodiments, the description of each embodiment has its own emphasis, and for parts not described in detail in a certain embodiment, reference may be made to the description of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is merely a logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware, or may be implemented in the form of a software program module.

The integrated units, if implemented in the form of software program modules and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, and the memory may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the methods and their core ideas of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A human physiological sample acquisition system, the acquisition system comprising:

a collection device;

the device comprises a mechanical arm, wherein the tail end of the mechanical arm is provided with a clamping device and a laser radar, the clamping device is used for clamping the acquisition device, and the laser radar is used for sending a radar signal to the direction pointed by the tail end of the mechanical arm and receiving returned laser radar data;

the control device is used for acquiring an image to be positioned of a target organ containing an object to be acquired according to the laser radar data; determining the position information of the target organ according to the image to be positioned; moving the acquisition device to the target organ according to the position information; controlling the acquisition device to move from the target organ to the target position of the object to be acquired, and respectively acquiring acting force and torque of a head of the acquisition device in a first direction, a second direction and a third direction in the process that the acquisition device moves to the target position of the object to be acquired, wherein the first direction is the advancing direction of the acquisition device, and the first direction, the second direction and the third direction are mutually perpendicular; controlling the acquisition device to reach the target position of the object to be acquired according to the reaction force of the head of the acquisition device in the first direction, the torque of the head of the acquisition device in the direction opposite to the first direction, the acting force and the torque of the head of the acquisition device in the second direction, and the acting force and the torque of the head of the acquisition device in the third direction, so as to acquire the physiological sample of the target position.

2. The acquisition system of claim 1, further comprising:

the storage table is used for storing the acquisition device;

the disinfection device is used for disinfecting the mechanical arm;

the control device is further used for controlling the disinfection device to disinfect the mechanical arm before the clamping device clamps the acquisition device;

and the sampling platform is used for bearing the mechanical arm, the storage platform and the disinfection device.

3. The acquisition system according to claim 2, wherein, before said acquiring, from said lidar data, an image to be located of a target organ containing an object to be acquired, said control device is further configured to:

4. The acquisition system according to claim 3, characterized in that, before said acquisition, by said lidar, of an image to be positioned of a target organ containing an object to be acquired, said control means are also adapted to:

determining, by the lidar, a position of a cover of the acquisition device;

5. The acquisition system according to claim 4, wherein, prior to said determining the position of said storage station by said lidar, said control means is further configured to:

determining, by the lidar, a position of the disinfection device;

6. The acquisition system according to claim 3, wherein after the acquisition device is controlled to reach the target position of the object to be acquired according to the reaction force of the acquisition device head in the first direction, the torque of the acquisition device head in the direction opposite to the first direction, the force and the torque of the acquisition device head in the second direction, and the force and the torque of the acquisition device head in the third direction to acquire the physiological sample of the target position, the control device is further configured to:

7. The acquisition system according to claim 3, characterized in that the terminal end of said mechanical arm is equipped with a lidar, and in said acquisition of the image to be positioned of the target organ containing the object to be acquired, said control device is particularly adapted to:

determining spatial position information and attitude information of the object to be acquired;

in the process of controlling the acquisition device to move, sending a radar signal to the object to be acquired through the laser radar, and receiving first laser radar data returned from the object to be acquired;

8. A human body physiological sample collection method is characterized by being applied to a human body physiological sample collection system, wherein the human body physiological sample collection system comprises a collection device, a mechanical arm and a control device, a clamping device and a laser radar are arranged at the tail end of the mechanical arm, the clamping device is used for clamping the collection device, and the collection method comprises the following steps:

the laser radar sends a radar signal to the direction pointed by the tail end of the mechanical arm and receives returned laser radar data;

determining the position information of the target organ according to the image to be positioned;

controlling the acquisition device to move from the target organ to the target position of the object to be acquired, and respectively acquiring acting force and torque of a head of the acquisition device in a first direction, a second direction and a third direction in the process that the acquisition device moves to the target position of the object to be acquired, wherein the first direction is the advancing direction of the acquisition device, and the first direction, the second direction and the third direction are mutually perpendicular;

controlling the acquisition device to reach the target position of the object to be acquired according to the reaction force of the head of the acquisition device in the first direction, the torque of the head of the acquisition device in the direction opposite to the first direction, the acting force and the torque of the head of the acquisition device in the second direction, and the acting force and the torque of the head of the acquisition device in the third direction, so as to acquire the physiological sample of the target position.

9. An electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the one or more programs including instructions for performing the method of claim 8.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executed by a processor to implement the method as claimed in claim 8.