CN111973152A - Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform - Google Patents

Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform Download PDF

Info

Publication number
CN111973152A
CN111973152A CN202010556719.9A CN202010556719A CN111973152A CN 111973152 A CN111973152 A CN 111973152A CN 202010556719 A CN202010556719 A CN 202010556719A CN 111973152 A CN111973152 A CN 111973152A
Authority
CN
China
Prior art keywords
robot
ear
data
data acquisition
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010556719.9A
Other languages
Chinese (zh)
Inventor
不公告发明人
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tan Sicong
Original Assignee
Tan Sicong
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tan Sicong filed Critical Tan Sicong
Priority to CN202010556719.9A priority Critical patent/CN111973152A/en
Publication of CN111973152A publication Critical patent/CN111973152A/en
Priority to AU2021290732A priority patent/AU2021290732A1/en
Priority to PCT/CN2021/100647 priority patent/WO2021254444A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/227—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for ears, i.e. otoscopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/24—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for the mouth, i.e. stomatoscopes, e.g. with tongue depressors; Instruments for opening or keeping open the mouth
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/103—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining refraction, e.g. refractometers, skiascopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/107—Measuring physical dimensions, e.g. size of the entire body or parts thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61C—DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE
    • A61C19/00—Dental auxiliary appliances
    • A61C19/04—Measuring instruments specially adapted for dentistry
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01G—WEIGHING
    • G01G19/00—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups
    • G01G19/44—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups for weighing persons
    • G01G19/50—Weighing apparatus or methods adapted for special purposes not provided for in the preceding groups for weighing persons having additional measuring devices, e.g. for height
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • Public Health (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Molecular Biology (AREA)
  • Medical Informatics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Dentistry (AREA)
  • Theoretical Computer Science (AREA)
  • Radiology & Medical Imaging (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Optics & Photonics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Epidemiology (AREA)
  • Ophthalmology & Optometry (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Medicines Containing Antibodies Or Antigens For Use As Internal Diagnostic Agents (AREA)

Abstract

The robot integrates the medical data acquisition, analysis and intelligent diagnosis of the five sense organs and the surgery, and is a physical examination medical data acquisition and analysis robot platform system which is built by combining an artificial intelligent robot system, each data acquisition device and nodes. The robot comprises a robot platform, a camera, a plurality of sensors, a data acquisition module, a data analysis module and a data analysis module. The medical data are classified by using a neural network algorithm, and the problems of eye, ear, oral cavity abnormality and the like are analyzed by using a data analysis technology. Body temperature was measured using infrared sensing. Medical data are efficiently collected under the guidance of robot voice. The method comprises the steps of effectively checking, regularly detecting abnormal symptoms, and realizing intelligent physical examination result feedback and regular examination prompt according to the physical standard, analysis data, intelligent inquiry, remote inquiry health and the like.

Description

Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform
Technical Field
The invention discloses a physical examination robot research and development platform based on five sense organs and surgical medical data acquisition and analysis, which is a platform physical examination medical data acquisition and analysis robot platform system constructed by combining an artificial intelligent robot system, various data acquisition devices and nodes.
Background
Is currently applied in the field of health physical examination; in the examination process, due to various human factors, the disease problems existing in the oral cavity, the teeth, the eyes and the like are difficult to be effectively identified and diagnosed by naked eyes. The physical examination efficiency is low, the data acquisition is difficult, the data acquisition is not accurate, and the like.
Cause the physical examination inefficiency, take time, take consequences such as effort, to the physical examination inefficiency, data acquisition is difficult, the inaccurate scheduling problem of data acquisition, utilize the camera that the robot carried, eye vision test, refraction collection equipment, the sound level meter, infrared sensor, B ultrasonic viscera image etc. passes through medical image data intelligent monitoring ear, the eye, each item abnormal index of oral cavity, effectively discern intraoral disease, eyes far-sighted, ametropia, the ear dysplasia, internal organs image classification, remote inquiry, medical data analysis, abnormal data classification discernment, intelligence feedback is unusual and the disease result, periodic inspection. Health detection, physical examination and the like, high frequency and high precision intelligent detection data.
The current products on the market are simple robot inspection equipment for measuring body temperature, height and weight, and the like. Still have the equipment simple and easy, the low scheduling problem of precision.
Disclosure of Invention
The invention aims to provide a health physical examination system based on an artificial intelligent robot. The platform medical data acquisition and analysis robot platform system is constructed by combining the artificial intelligent robot system and nodes such as various data acquisition devices.
The data such as image, eye data, ear data, oral cavity, temperature are effectively gathered in the realization. Realize the high-precision measurement of body temperature, height and weight. Accurate analysis is realized, and abnormal data of each organ is classified. The method realizes accurate identification of common problems of mouth, ear, eye and the like.
The technical scheme adopted by the invention is as follows:
artificial intelligence robot medical data acquisition, analysis health physical examination system, the robot device includes:
and the robot main system module is used for realizing the main control of the robot, and is communicated from the camera and sensor data acquisition module, the B ultrasonic module, the medical ear and eye inspection equipment data acquisition module to the medical data analysis module, and is used for interaction among the robot arm action planning control module, the voice module and the user.
The system comprises a camera and a sensor data acquisition module, wherein the data acquisition module is used for acquiring medical images in ears and eyes, body temperature, height, weight, heart and other measured medical data; the data module is used for interaction and voice guidance between the main control system and a user;
the data analysis module is used for analyzing the medical data according to the standard value and finding out medical abnormal data;
and the data module is used for classifying ear, eye and mouth medical images.
And the robot arm action planning and collecting module is used for action planning and interaction between the robot arm action and a user.
In this scheme, can be through the main control system of robot, camera and sensor data acquisition module, B ultrasonic module, medical ear, the medical treatment ear of eye inspection equipment data acquisition module is used for gathering ear, eye, the internal temperature of mouth, height, weight, heart check out test set medical data to according to the robot arm action planning acquisition module, voice module guides the user, strengthens the interaction between robot and user, realizes intelligent collection. The medical data analysis is used for analyzing the medical data according to the standard value and intelligently finding out medical abnormal data; and the image classification module is used for accurately classifying ear, eye and mouth medical images and intelligently positioning the positions of the five sense organs. The accuracy of intelligent acquisition and the accuracy of medical data abnormity identification are improved, and the flexibility and the possibility of medical image classification, remote acquisition of analysis and remote diagnosis are improved.
Further, the ROS main system of the robot is used for realizing main control, data acquisition, image classification, voice interaction, action interaction, intelligent acquisition, intelligent analysis of abnormal data, intelligent diagnosis and remote diagnosis of the robot.
As a further improvement of the invention, the data acquisition module of the medical detection device, the medical ear and eye inspection device is used for acquiring ear, eye and mouth images, body temperature, height and weight.
As a further improvement of the invention, the voice module comprises a voice recognition module and a voice synthesis module, and is used for interaction and voice guidance between the main control system and the user.
As a further improvement of the invention, the action module comprises an action planning module and an action acquisition module, and is used for acquiring action images of the robot arm and action interaction between the main control system and the user.
As a further improvement of the invention, the action module comprises an action planning module and an ear collection action plan. And the eye acquisition action plan and the intraoral acquisition action plan are used for the action interaction between the main control system and the user and the acquisition of the action image of the robot arm.
The acquisition method of the target of the robot arm extending to the five sense organs comprises the following steps:
the camera head tracking five sense organs data acquisition device method comprises the following steps:
STEP 1: setting a target
STEP 2: setting target parameters (target name, left and right arm joint)
STEP 3: setting communication targets
STEP 4: issuing targets, parameters (target position and pose marks)
STEP 5: setting position and pose marks
STEP 6: setting target to head id, target pose and direction values
STEP 7: setting a timestamp
STEP 8: setting a pose marker as an origin of coordinates and a direction value
The communication method of the visual camera and the facial feature acquisition device comprises the following steps:
step 1: initial point cloud node
Step 2: node parameters of the set holder publisher (target name, pose mark)
Step 3: setting up camera subscriber node parameters (Point cloud, nearest point cloud list)
Step 4: defining and obtaining a list of recent point clouds
Step 5: defining the nearest point and converting it into an array of points
Step 6: calculating COG
Step 7: confirming point cloud parameters NaN and returning point cloud information
Step 8: setting pose direction values as point objects
Step 9: issuing COG as target pose
Step 10: setting target parameters (pose flag, timestamp, target to head Id, COG target pose, orientation value)
Step 11: issuing gripper target node
The five sense organs target collection method comprises the following steps:
step 1: setting acquisition zone position
Step 2: when the movement planning fails, the movement planning is allowed to be re-planned
Step 3: reference coordinate system for setting target position of five sense organs
Step 4: setting time limits for each exercise program
Step 5: setting target acquisition zone position
Setting the position of a medical collecting area, the position of a seat and the position of an auxiliary instrument
Step 6: setting a collection area position, a seat position and an auxiliary instrument position to be put into a scene
(the parameters include hospital bed ID, collection area color and standard point pose, area.position.x, area.position.y, area.position.z, area.position.orientation.w; seat ID, seat color, height and seat standard point pose, seat.position.x, seat.position.y, seat.position.z, seat.position.orientation.w; auxiliary instrument ID, auxiliary instrument color, auxiliary instrument pose, auxiliary instrument position.x, access machine position.y, access machine position.position.z, access machine position.z, access machine position.orientation.w; and the above parameters are added to the diagnosis and treatment scenario.
Step 7: identifying objects of five sense organs
Step 8: setting scene colors
Step 9: setting color label of collecting area, color of chair, and color of auxiliary instrument
Step 10: a color tag is set into the DEMO (including initializing planning scene objects, monitoring set scene differences,
setting the color of a lying scene, the color of a left side lying scene and the color of a right side lying scene under a release color label).
A method of deep neural network algorithm refinement, the method comprising the steps of:
s1, establishing a human face mathematical model and a mathematical model for identifying ear, mouth and eye images of the face of an individual
S2, extracting the outline characteristics of five sense organs, including the characteristics of color, shape, outline and the like
S3, extracting characteristic value (color, shape, contour) of image, etc
S4, inputting characteristic value of detection item
S5, improving the weight optimizer, and obtaining an output value through image training
S6, classifying ear, mouth and eye images according to the output result, and accurately positioning ear, mouth and eye positions
Intelligent classification of ear-mouth internal eye images by using improved deep neural network algorithm and accurate organ positioning intelligent acquisition
A method for improving SVM classification algorithm, said method comprising the steps of:
s1, establishing an internal organ mathematical model
S2, extracting the characteristics of the internal contour of the organ, including the characteristics of color, shape, contour and the like
S3, extracting characteristic value (color, shape, contour) of image, etc
S4, inputting item characteristic value
S5, improving SVM machine learning algorithm, and calculating to obtain output value
S6, classifying the visceral organ images according to the output result, and accurately classifying the visceral organs including breast, lung, liver, gallbladder, spleen, kidney and other images
Intelligent classification of visceral organ images by using improved SVM algorithm and accurate positioning of visceral organ positions
In conclusion, the robot has the beneficial effects that aiming at the problems of low physical examination efficiency, time consumption, energy consumption, low disease identification degree and the like, the problems of low physical examination efficiency, difficult data acquisition, inaccurate data acquisition and the like in the prior art are solved through the intelligent data acquisition of the camera and each sensor module carried by the robot.
Through various medical indexes of medical data, the abnormity and the diseases which are difficult to be effectively identified by naked eyes are identified, and the disease problems existing in the oral cavity, the teeth, the eyes and the like are diagnosed. Efficiently identify and manage diseases. The physical examination intelligent research and development platform can realize health management, effective detection, analysis and body abnormity identification, intelligent diagnosis, remote diagnosis of eye, ear, oral cavity and other problems, and visceral abnormal diseases and other health problems.
The remote diagnosis of the abnormality and the disease in the viscera improves the accuracy and the efficiency of physical examination, and can intelligently detect, analyze and diagnose the disease. An artificial intelligence robot and five sense organs examination medical system is effectively created.
Description of the drawings:
fig. 1 is a block diagram of a camera and a sensor data acquisition module according to an embodiment of the present disclosure.
Fig. 2 is a schematic structural diagram of a medical data acquisition and analysis robot for physical examination according to an embodiment of the present application.
Reference numerals: 10-robot master system simulation device; 201, 202-camera simulation means; 30-a sensor simulation unit; 101-a voice module; 103-a robot arm module;
the specific implementation mode is as follows:
the embodiment of the application solves the problems of low physical examination efficiency, difficult data acquisition, inaccurate data acquisition and the like in the prior art by providing a physical examination medical data acquisition simulation device, a simulation robot system and a method for analyzing medical data classification medical images, realizes effective detection, data analysis, body abnormity identification, intelligent diagnosis, far-end diagnosis of eyes, ears, oral cavity and other problems, and health problems such as abnormal diseases in visceral organs and the like.
The technical scheme in the implementation of the application is as follows for solving the technical problems:
artificial intelligence robot medical data acquisition, analysis health physical examination system, the robot device includes: and the robot main system module is used for realizing the main control of the robot, and is used for communicating from the camera and sensor data acquisition module, the medical ear and eye inspection equipment data acquisition module to the medical data analysis module, and is used for interaction among the robot arm action planning control module, the voice module and the user. The system comprises a camera and a sensor data acquisition module, wherein the data acquisition module is used for acquiring medical images in ears and eyes, body temperature, height, weight, heart and other measured medical data; the data module is used for interaction and voice guidance between the main control system and a user; the data acquisition module is used for acquiring medical data of ear and eye internal inspection equipment; the robot arm action planning and collecting module is used for action planning and interaction between the robot arm action and a user;
in order to better understand the technical solutions, the present invention will be further described in detail with reference to the following examples and drawings, but the embodiments of the present invention are not limited thereto.
Example 1:
as shown in fig. 1, an artificial intelligence robot medical data collecting and analyzing health examination system, the robot device includes:
the main control system 10 of robot, the module is used for realizing the main control and camera and the sensor data acquisition module of robot, and main control system carries on with the robot arm, and eye, mouth, ear inspection equipment data acquisition module communication for the robot arm action plans the collection, communicates with voice module, is used for the voice interaction between robot and user.
The camera 20 and the sensor data acquisition module 30, the medical ear and eye inspection equipment data acquisition module is used for acquiring ear, eye, intraoral body temperature, height, weight, heart detection equipment medical data and medical images in B-ultrasonic visceral organs; and the acquisition module 103 and the voice module 101 are planned according to the actions of the robot arm to guide the user, enhance the interaction between the robot and the user and realize intelligent acquisition. The medical data analysis is used for analyzing the medical data according to the standard value and intelligently finding out medical abnormal data; and the image classification module is used for accurately classifying ear, eye and mouth medical images, and intelligently positioning and classifying the positions of the five sense organs.
The main control system 10 of robot, the communication between the main control system of robot and each module the module is used for realizing the main control of robot, and with camera 20 and sensor data acquisition module 30, the communication of B ultrasonic image acquisition module 102, main control system and robot arm carry on, the eye, mouth, ear inspection equipment data acquisition module communication 103 for robot arm action planning gathers, and with voice module 101 communication, be used for the voice interaction between robot and user.
In the embodiment of the application, the robot master control system is connected with the depth camera simulation unit 20 and the robot arm simulation device 103 through the ROS system 10; and the simulation robot main control system device 10 is in communication connection with the voice module 30. And the simulation robot main control system device 10 is in communication connection with the robot arm carrying, eye, mouth and ear inspection equipment data acquisition module communication 103. And the simulation robot main control system device 10 is in communication connection with the voice module 101. In this embodiment, the master control system of robot is connected with degree of depth camera and is used for people's face, ear, mouth, eye image acquisition, infrared sensing etc. is used for height, body temperature data acquisition, and microphone and pickup apparatus are used for the voice interaction.
The camera simulation unit 20 is used for collecting data of human face, eyes, mouth and ears, issuing image data according to instructions of the robot master control system simulation device 10, communicating with the image recognition node and recognizing the five sense organs. The TF packet under the ROS robot master control system 10 returns the position information of the mouth, ear and eye of the body, and the robot arm 103 moves to the position of the five sense organs to acquire data. Thereby precisely positioning the five sense organs. The method comprises the steps of designing the motion of a robot by utilizing a Moveit package under an ROS robot system and planning motion interaction by utilizing a main system of the robot, designing the motion of the robot, designing and collecting positions, angles and motions of a camera, a sensor and the like aiming at ears, eyes and mouths, realizing friendly interaction of a human-robot, and efficiently collecting data.
The data acquisition module 30 is used for acquiring height, weight and temperature data, issuing medical data according to instructions of the robot master control system simulation device 10, communicating with the data analysis program node, returning data acquired by each sensor and checking abnormal data. And (4) infrared body temperature measurement. Body temperature, height and weight of the physical examination are sensed through infrared measurement.
Medical image data is issued according to the instructions of the robot master control system simulation device 10, and the medical image data is communicated with the organ identification program node to identify the organ in the organ. The TF packet under the ROS robot master control system 10 is used for returning the information of each position of the body, and the robot arm 103 moves to the position of the internal organs of the body to acquire data. Thereby accurately positioning the organs in the viscera. The organ name, image, and data value of each organ are returned.
Position and time, the robot master control system simulates the device 10 action commands and moves to the organ position. And Moveit is adopted to realize the action planning and data acquisition of the robot arm under the robot system. The robot arm action is planned by adopting a Moveit engineering bag under an ROS robot system, a camera, a sensor and the like are adopted to carry on the robot arm, and data such as oral cavity, teeth, ears, eyes and the like are effectively acquired through the robot arm movement planning, action interaction and the like, so that accurate data acquisition is realized.
Example 2:
based on example 1, several methods for locating and identifying the five sense organs are provided, as shown in fig. 2, the method for identifying the five sense organs is as follows:
s1, establishing a mathematical model of image recognition by using an improved deep neural network algorithm, wherein the mathematical model comprises characteristics of hands, feet and oral cavities, extracting characteristic values (color, shape and contour) of an image and the like, and inputting characteristic values of detection items. The method comprises the steps of utilizing an improved artificial neural network algorithm (BP algorithm), conducting reversely, adjusting weight parameters to obtain output values, and distinguishing normal physical signs or hand-foot-and-mouth diseases, decayed teeth and other health problems of the children according to the range of the output values.
And S2, extracting the feature values of the contours and colors of the ears, eyes, oral cavity, teeth and the like of the face by using the feature values such as the contours, improving a clustering algorithm, and accurately classifying the facial organs (oral cavity, ears and eyes) of the human.
S3, improving a deep neural network algorithm, designing a training function by using the ratio of the training times and the weight between reverse conduction and an acceleration parameter through contour features, color features and the like to improve the neural network algorithm and the like, so as to reduce the training times and accelerate the training speed, and the method can be applied to solving the problem of intelligent disease recognition under a large data platform.
The method for locating the five sense organs comprises the following steps:
and researching an improved clustering algorithm method. And establishing eye, ear and mouth contour feature models, and improving the positions of the intelligently classified eyes, mouths and ears by utilizing algorithms such as clustering and the like so as to indicate the moving direction and position of the robot arm.
A medical ear and eye mouth examination equipment data acquisition module is used for acquiring ear, eye and mouth internal temperature, height, weight and heart detailed detection methods as follows:
the eye positioning method comprises the following steps:
the eye is intelligently classified and identified by using an artificial neural network transducer flow frame, the eye position is returned, the position coordinates are converted by using tf packets, the robot arm is moved to the position 35 cm parallel to the eye, the eye sight is attracted by using the light source and animation marking actions, the eye refraction data acquisition is realized, the sphere diameter and column diameter combined refraction state is detected, and the eye data is analyzed.
The eye examination method comprises the following steps:
the light reflected by human eyes passes through a group of micro-lens arrays by the Hartman Shack sensor principle and then is imaged on a sensor, and then is converted into a refraction state with a spherical diameter and a column diameter combined by an operation formula, and parameter values of different stages are compared to judge the development condition and disease abnormity of the eyes.
The refractive data of the eye is expressed by the combined refractive state of the cylindrical diameters according to the Hartman Shack sensor principle, the light reflected by the eyes passes through a group of micro lens arrays and then is imaged on the sensor, and then is converted into the spherical diameters through an operation formula.
The ear examination method comprises the following steps:
the ear disease screening method comprises the steps of carrying a camera by a robot arm, collecting ear image data, identifying images by an artificial neural network algorithm, calculating actual ear development values, and comparing development value ranges of all stages to screen ear diseases. The graphic features of the image surface are extracted, and the robot arm carries a sound level meter to measure pure sound audiometry screening instrument to monitor the abnormal audios.
The mouth, tooth positioning method:
the mouth position is intelligently classified and recognized by an artificial neural network Tensorflow framework, the position data is returned by a tf packet and is moved to the position near the mouth, the mouth is opened by close interaction, the image data in the oral cavity including the tooth quantity and decayed tooth position data is collected, the problems of teeth, decayed teeth and the like are intelligently detected, and the diseases of hand-foot-and-mouth disease and the like are judged by utilizing the data in the oral cavity and combining an image recognition technology.
The tooth inspection method comprises the following steps:
the tooth inspection method and the oral disease identification method are as follows:
s1, guiding the variable model contour to evolve to target characteristics, identifying the number of teeth through the tooth contour, and monitoring the problems of decayed teeth and the like through gray scale contrast.
S2, improving an SVM algorithm, adopting the characteristics of larger gray value difference between the background and the target in the oral cavity and the like, dividing the gray gradient value of the image by the SVM algorithm, distinguishing the oral tooth area from the hand-foot-and-mouth disease target point, and judging and distinguishing teeth and identifying common teeth diseases according to the characteristics of the central concave point of the dental crown and the characteristics of small difference between the tooth root and the joint gray value.
And S3, extracting the image characteristics of the image surface in the oral cavity, guiding the profile of the variable model to evolve to the target characteristics, when the model reaches the image characteristics, the numerical value is minimum, and because the curvature extreme value difference between the tooth crown and the tooth root is large, the value of the parameter is determined by taking the position interval of the tooth crown and the tooth root in the Z-axis direction as the basis.
S4, classifying the background and the target in the oral cavity: the method comprises the steps of dividing the gray scale gradient value of an image by a clustering algorithm and marking a tooth target, identifying the outline of the model by a cross section, a sagittal plane and a coronal plane, distinguishing the number of teeth according to the characteristics of the central concave point of the crown and the characteristics of small difference between the tooth root and the joint gray scale value, and according to the tooth profile after tooth classification, comparing the gray scale difference of the coronal plane of a single tooth by the clustering algorithm, dividing a carious tooth area, and intelligently identifying the positions of the carious tooth and the carious tooth.
The ear development examination method comprises the following steps:
the ear positioning method comprises the following steps:
utilize artificial neural network Tensorflow frame intelligent classification discernment ear position, intelligent identification ear position removes the robotic arm to the position of pasting the ear, and the structural image develops in the ear, adopts the sound level meter to test hearing to smile or action feedback hearing data are gathered in the cooperation of camera and light source.
The ear examination method comprises the following steps:
the ear disease screening method comprises the steps of carrying a camera by a robot arm, collecting ear image data, identifying images by an artificial neural network algorithm, calculating actual ear development values, and comparing development value ranges of all stages to screen ear diseases. Actually calculating the following values, and detecting growth and development data of all age groups: comprises an auricle temporal angle 1 (an included angle between a basal point plane auricle on the ear and the temporal scalp); auriculo-temporal angle 2 (the angle between the concha and the surface skin of the temporal bone); ear length measurement; ear width measurement results; ear width and face width data; ear length and morphology level. And analyzing and screening the ear dysplasia according to normal ear data.
The sound level meter is adopted to measure a pure sound audiometric screening instrument, a game type measuring method (clapping smiles and the like) is adopted to prompt a hearing test result, transient induced otoacoustic emission screening is adopted, a proper earplug is adjusted, the position of a probe is adjusted, and the external auditory canal sealing development condition is tested.

Claims (10)

1. The invention discloses a robot for collecting, analyzing and diagnosing medical data of five sense organs and surgery, and a platform. Artificial intelligence robot medical data acquisition, analysis health physical examination system, the robot device includes:
and the robot main system module is used for realizing the main control of the robot, and is used for communicating from the camera and sensor data acquisition module, the medical ear-nose and eye-mouth inspection equipment data acquisition module to the medical data analysis module, and is used for interaction among the robot arm action planning control module, the voice module and the user.
The system comprises a camera and a sensor data acquisition module, wherein the data acquisition module is used for acquiring medical images in ears, noses and eyes, body temperature, height, weight, heart and other measured medical data; the data module is used for interaction and voice guidance between the main control system and a user;
the data analysis module is used for analyzing the medical data according to the standard value and finding out medical abnormal data;
and the data module is used for classifying ear-nose, eye and mouth medical images.
And the robot arm action planning and collecting module is used for action planning and interaction between the robot arm action and a user.
2. A robot for collecting, analyzing and diagnosing medical data of five sense organs and surgery is a platform and is characterized in that the neural network classification improvement method is used for realizing recognition of human mouth parts, eye parts and ear-nose parts, intelligently positioning the five sense organs and accurately recognizing the five sense organs.
3. The robot device according to claim 1, wherein the coordinate transformation package under the robot system is used to return the position information of mouth, ear, nose and eyes of the body, and the robot system is used to carry the robot arm to move to the position of data acquisition of each part of the body, so as to accurately position the organs of five sense organs.
4. The robotic device of claim 1, wherein the robotic arm carries a camera for collecting ear image data, the artificial neural network algorithm is used for recognizing images, calculating actual values of ear development, comparing normal value ranges for screening ear diseases, and the robotic arm carries a sound level meter for measuring pure tone audiometry and screening for monitoring hearing abnormalities.
5. The robotic device of claim 1, wherein the method of eye data acquisition and the method of intelligently identifying eye refractive error are utilized. After light rays reflected by human eyes pass through a group of micro-lens arrays through a sensor, the light rays are converted into a refraction state with a spherical diameter and a column diameter combined through an operation formula according to the Hartman Shack sensor principle, and parameter values of different stages are compared to judge the development condition and disease abnormity of the eyes.
6. The robotic device of claim 1, wherein the method for ear dysplasia detection using ear data collection and intelligent identification uses a robotic arm carrying a camera to collect ear image data, uses an artificial neural network algorithm to identify images, calculates ear actual values, and screens ear diseases against a range of normal values.
7. The robotic device of claim 1, wherein the graphical features of the image surface are extracted to guide the evolution of the variable model contour to target features by intra-oral data acquisition and intelligent identification of tooth development, intra-oral disease, tooth number identification by tooth contour, caries monitoring by gray scale contrast, etc.
8. The robot apparatus of claim 1, wherein a neural network algorithm improvement method is used to build a mathematical model for image recognition, to input the characteristic values of the detection items, including the characteristics in the oral cavity, to extract the characteristic values (color, shape, contour) of the image, to obtain output values by adaptively adjusting the weight parameters, to distinguish the problems of normal signs or diseases, caries, etc. according to the range of the output values, which is beneficial to improve the intelligent physical examination efficiency.
9. The robotic device of claim 1, wherein the feature models of the eye, ear, and mouth contours are created based on an improved method of machine learning by SVM algorithm, and wherein the positions of the eye, mouth, and ear are intelligently classified by machine learning such as SVM to indicate the direction and position of the robot arm movement. The eye, ear, the categorised of oral area profile, accurate discernment, the intelligence location has improved recognition efficiency greatly.
10. The robot device of claim 1, wherein the robot arm is used for moving, grabbing, human action interaction, effective action guidance, and data acquisition by using a robot arm and an action planning design method.
CN202010556719.9A 2020-06-17 2020-06-17 Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform Pending CN111973152A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202010556719.9A CN111973152A (en) 2020-06-17 2020-06-17 Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform
AU2021290732A AU2021290732A1 (en) 2020-06-17 2021-06-17 Five-sense-organ and surgical medical data acquisition, analysis and diagnosis robot and platform
PCT/CN2021/100647 WO2021254444A1 (en) 2020-06-17 2021-06-17 Five-sense-organ and surgical medical data acquisition, analysis and diagnosis robot and platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010556719.9A CN111973152A (en) 2020-06-17 2020-06-17 Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform

Publications (1)

Publication Number Publication Date
CN111973152A true CN111973152A (en) 2020-11-24

Family

ID=73442108

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010556719.9A Pending CN111973152A (en) 2020-06-17 2020-06-17 Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform

Country Status (3)

Country Link
CN (1) CN111973152A (en)
AU (1) AU2021290732A1 (en)
WO (1) WO2021254444A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112990101A (en) * 2021-04-14 2021-06-18 深圳市罗湖医院集团 Facial organ positioning method based on machine vision and related equipment
CN113425440A (en) * 2021-06-24 2021-09-24 广州华视光学科技有限公司 System and method for detecting caries and position thereof based on artificial intelligence
CN113440258A (en) * 2021-07-12 2021-09-28 曹庆恒 Ear-nose robot and using method thereof
WO2021254444A1 (en) * 2020-06-17 2021-12-23 谈斯聪 Five-sense-organ and surgical medical data acquisition, analysis and diagnosis robot and platform
WO2021254427A1 (en) * 2020-06-17 2021-12-23 谈斯聪 Integrated robot and platform for ultrasound image data acquisition, analysis, and recognition
CN113855250A (en) * 2021-08-27 2021-12-31 谈斯聪 Medical robot device, system and method
CN113855067A (en) * 2021-08-23 2021-12-31 谈斯聪 Visual image and medical image fusion recognition and autonomous positioning scanning method
CN113855068A (en) * 2021-08-27 2021-12-31 谈斯聪 Method for intelligently identifying chest organs and autonomously positioning and scanning chest organs
EP4005462A1 (en) * 2020-11-25 2022-06-01 KHS GmbH Operation assistance system and method for the generation of control signals for voice control of a motor-controlled mobile robot kinematics of such an operation assistance system
CN116913519A (en) * 2023-07-24 2023-10-20 东莞莱姆森科技建材有限公司 Health monitoring method, device, equipment and storage medium based on intelligent mirror

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118039134B (en) * 2024-04-09 2024-06-04 达州市中心医院(达州市人民医院) Medical information data enhancement method and system based on big data

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110273669A1 (en) * 2007-08-21 2011-11-10 Marc Abitbol Multifunctional opthalmic measurement system
CN107752984A (en) * 2017-11-15 2018-03-06 李玉东 A kind of high intelligent general medical practice operation robot based on big data
CN109431452A (en) * 2018-10-25 2019-03-08 武汉目明乐视健康科技有限公司 Without human eye human health screening instrument
US20200037930A1 (en) * 2018-08-01 2020-02-06 Idx Technologies Inc. Autonomous diagnosis of ear diseases from biomarker data
US20200098461A1 (en) * 2011-11-23 2020-03-26 Remedev, Inc. Remotely-executed medical diagnosis and therapy including emergency automation
WO2020080819A1 (en) * 2018-10-16 2020-04-23 주식회사 큐티티 Oral health prediction apparatus and method using machine learning algorithm

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106725295A (en) * 2016-11-29 2017-05-31 瑞达昇科技(大连)有限公司 A kind of miniature check-up equipment, device and its application method
CN106965193A (en) * 2017-03-31 2017-07-21 旗瀚科技有限公司 A kind of intelligent robot diagnosis guiding system
US10842378B2 (en) * 2018-01-26 2020-11-24 Shiv Prakash Verma Digital healthcare practice system for digital citizens
CN111973152A (en) * 2020-06-17 2020-11-24 谈斯聪 Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110273669A1 (en) * 2007-08-21 2011-11-10 Marc Abitbol Multifunctional opthalmic measurement system
US20200098461A1 (en) * 2011-11-23 2020-03-26 Remedev, Inc. Remotely-executed medical diagnosis and therapy including emergency automation
CN107752984A (en) * 2017-11-15 2018-03-06 李玉东 A kind of high intelligent general medical practice operation robot based on big data
US20200037930A1 (en) * 2018-08-01 2020-02-06 Idx Technologies Inc. Autonomous diagnosis of ear diseases from biomarker data
WO2020080819A1 (en) * 2018-10-16 2020-04-23 주식회사 큐티티 Oral health prediction apparatus and method using machine learning algorithm
US20210398275A1 (en) * 2018-10-16 2021-12-23 Qtt Co. Oral health prediction apparatus and method using machine learning algorithm
CN109431452A (en) * 2018-10-25 2019-03-08 武汉目明乐视健康科技有限公司 Without human eye human health screening instrument

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021254444A1 (en) * 2020-06-17 2021-12-23 谈斯聪 Five-sense-organ and surgical medical data acquisition, analysis and diagnosis robot and platform
WO2021254427A1 (en) * 2020-06-17 2021-12-23 谈斯聪 Integrated robot and platform for ultrasound image data acquisition, analysis, and recognition
EP4005462A1 (en) * 2020-11-25 2022-06-01 KHS GmbH Operation assistance system and method for the generation of control signals for voice control of a motor-controlled mobile robot kinematics of such an operation assistance system
CN112990101A (en) * 2021-04-14 2021-06-18 深圳市罗湖医院集团 Facial organ positioning method based on machine vision and related equipment
CN112990101B (en) * 2021-04-14 2021-12-28 深圳市罗湖医院集团 Facial organ positioning method based on machine vision and related equipment
CN113425440A (en) * 2021-06-24 2021-09-24 广州华视光学科技有限公司 System and method for detecting caries and position thereof based on artificial intelligence
CN113440258A (en) * 2021-07-12 2021-09-28 曹庆恒 Ear-nose robot and using method thereof
CN113855067A (en) * 2021-08-23 2021-12-31 谈斯聪 Visual image and medical image fusion recognition and autonomous positioning scanning method
CN113855250A (en) * 2021-08-27 2021-12-31 谈斯聪 Medical robot device, system and method
CN113855068A (en) * 2021-08-27 2021-12-31 谈斯聪 Method for intelligently identifying chest organs and autonomously positioning and scanning chest organs
WO2023024397A1 (en) * 2021-08-27 2023-03-02 谈斯聪 Medical robot apparatus, system and method
CN116913519A (en) * 2023-07-24 2023-10-20 东莞莱姆森科技建材有限公司 Health monitoring method, device, equipment and storage medium based on intelligent mirror

Also Published As

Publication number Publication date
WO2021254444A1 (en) 2021-12-23
AU2021290732A1 (en) 2023-03-02

Similar Documents

Publication Publication Date Title
CN111973152A (en) Five sense organs and surgical medical data acquisition analysis diagnosis robot and platform
EP3713446B1 (en) Handheld dental tracking device
KR101898414B1 (en) A 3-dimensional measurement method for eye movement and fully automated deep-learning based system for vertigo diagnosis
US11314983B2 (en) Method for analyzing an image of a dental arch
US10172552B2 (en) Method for determining and analyzing movement patterns during dental treatment
CN109069097A (en) Dental three dimensional data processing device and its method
CN111839489B (en) Non-contact physiological and psychological health detection system
CN111973228A (en) B-ultrasonic data acquisition, analysis and diagnosis integrated robot and platform
CN102715965B (en) Dental jaw movement locus recording device and dental jaw relationship transferring method
CN110200601A (en) A kind of pulse condition acquisition device and system
CN102247128A (en) Human body pulse information collecting device and human body health status monitoring device
CN106901689B (en) Cognitive and motor function detection system based on physical interaction and data processing method
Popat et al. Quantitative analysis of facial movement—A review of three-dimensional imaging techniques
US20240041569A1 (en) Method for analyzing an image of a dental arch
CN114999646A (en) Newborn exercise development assessment system, method, device and storage medium
Lee et al. Analysis of gait sub-movements to estimate ataxia severity using ankle inertial data
CN117558428B (en) Imaging optimization method and system for liver MRI
Pham et al. Lad: A hybrid deep learning system for benign paroxysmal positional vertigo disorders diagnostic
CN110364260A (en) Autism earlier evaluations apparatus and system based on indicative language paradigm
CN110673721A (en) Robot nursing system based on vision and idea signal cooperative control
JP5506024B2 (en) Temporomandibular disorder diagnosis support system and apparatus provided with pain detector
KR101309013B1 (en) Diagnosis Device For Face Form Using Facial Image and Cephalometric Image
WO2021254427A1 (en) Integrated robot and platform for ultrasound image data acquisition, analysis, and recognition
KR20160052995A (en) functional malocclusion diagnostic system
US20230397877A1 (en) Swallowing capture and remote analysis systems including motion capture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 510403 room 801, No. 27, Tangjing South Street, Jichang Road, Baiyun District, Guangzhou, Guangdong

Applicant after: Tan Sicong

Address before: 510000 439-a807 Baiyun Avenue, Baiyun District, Guangzhou City, Guangdong Province

Applicant before: Tan Sicong

CB02 Change of applicant information
DD01 Delivery of document by public notice

Addressee: Yu Zhiying

Document name: Notice of First Examination Opinion

DD01 Delivery of document by public notice
DD01 Delivery of document by public notice

Addressee: Yu Zhiying

Document name: Deemed withdrawal notice

DD01 Delivery of document by public notice