AU2021290732A1

AU2021290732A1 - Five-sense-organ and surgical medical data acquisition, analysis and diagnosis robot and platform

Info

Publication number: AU2021290732A1
Application number: AU2021290732A
Authority: AU
Inventors: Sicong TAN
Original assignee: Individual
Current assignee: Individual
Priority date: 2020-06-17
Filing date: 2021-06-17
Publication date: 2023-03-02
Also published as: CN111973152A; WO2021254444A1

Abstract

A five-sense-organ and surgical medical data acquisition, analysis and diagnosis integrated robot and a platform. By means of a camera (20, 210, 220) and multiple sensors (30) of a robot system, the functions of robot voice interaction, arm action interaction, data acquisition, and data analysis are implemented by using a robot node communication principle. The robot system monitors data of height, weight, electrocardiogram, blood pressure, oral cavity, ears, nose, and eyes. Medical indicators are intelligently detected, comprising common oral diseases, ear and nose diseases, eye diseases, vision ametropia classification, and the like. By using a neural network algorithm and a machine learning algorithm, medical data is classified, and problems such as eye, ear, nose, and oral cavity abnormalities are analyzed. Regular detection of abnormal symptoms, physique detection, intelligent consultation, remote consultation are performed, thereby implementing intelligent physical examination result feedback and regular examination prompting.

Description

TECHNICAL FIELD The present invention disclosure relates to medical data collection and data analyse , relates to physical examination robot apparatus, to medical robot system and medical image recognition method,which are widely used in physical examination and artificial intelligent yield.

BACKGROUND ART These are many problems such as effectiveness of physical examination, too expensive human resource cost and too heavy work press. In order to perform tasks faster and more accuracy. A medical robot apparatus is designed to implement physical examination of five sensory organs, teeth, temperature, weight, height. It is widely used in the medical field and in the industrial field. In the market, there are only physical examination devices with simple functions such as measuring temperature, weight and height. This patent invention perform physical examination by medical robot apparatus which include vision devices, examination devices of five sensory organs, teeth and sensors, optometry device, Phoropter, Refractor, sound level meter, infrared sensor. According to physical examination data and physical examination images, the medical robot apparatus perform medical data analysis and disease detection to warn disease and recognition image as accuracy as possible. It is effect to classify medical image, to recognize medical images and to warn diseases of five sensory organs and teeth diseases by self-controlled mode. It is also effect to physical examination by remote controlled mode. It is more effectively to recognize medical images of Dentistry, recognize medical images of Antimetropia and Amblyopia, recognize medical images of Ear Disease. It is more effectively to perform medical data analysis, to detect disease and to warn disease. Commonly, A medical robot apparatus are including as follow: In robot main control system, a drive control module, which is configured to control driving of the joint modules based on states of the joint modules. A robot apparatus and physical examination robot main control system device are used to control robot. The control function module which is configured to control another joint function modules. The robot main control system device and control function module connects to machine vision devices, sensors, examination devices of ear, eye, mouth, teeth, another examination devices, and which communicates with these devices, medical data analysis model, robot arm motion planing model, medical images recognition model. A machine vision device of joint modules, which machine vision devices, sensors, and collection devices are used to collect physical examination data, such as medical data of ear, eye, mouth, teeth and temperature, weight ,height. A robot arm of joint modules are to connect a plurality of links robot motion planning collection device which is used to motion planning. A drive control module is configured to control driving of the arm joint modules based on states of the joint modules and arm motion planning.

Speech model and speech recognition device are used to speech interaction between robot main control system device and users by speech induction and by speech command. Medical physical examination apparatus of joint modules, medical images analysis device is used to classify and to recognize physical examination images in ENT Department , in Ophthalmology Department, in Stomatology Department. Medical image classification model is used to classification images of ENT Department. Robot arm motion planing and collection model is used to motion planing and to motion interaction between robot arm and users. The robot is according to robot main system device which communicates with machine vision device, sensors, examine devices, data collection modules, medical image collection modules, medical data analysis module, medical image recognition module. Examine devices of ENT Department are used to collect medical images of ear, eye, mouth, teeth , to measure temperature, weight,height, electrocardiogram. Robot main control system device communicates with motion planning function module and speech function module to remote control robot arm, to motion interaction and to collect medical data more intelligently. Robot arm motion planing and collection model is used to motion planing and to motion interaction between robot arm and users. A drive control module which is configured to control driving of the joint modules based on states of the joint modules. This Patent performs physical examination by medical robot apparatus which is used to collect and to analyse medical data more accuracy and to detect disease, to warning abnormal disease and to recognition images as accuracy as possible. Robot main control system device communicates with machine vision device, sensors, medical data collection device and model, medical image collection function module, medical data analysis function module, medical image recognition function module. Robot main control system device communicates with motion planning function module and speech function module to remote control robot arm, to motion interaction and to collect medical data more intelligently. Speech model is used to speech induction and speech interaction between robot main system device and users to collect data intelligently. Medical data analysis model is used to analysis medical data more accurately and detect abnormal data more intelligently. Medical images classification model is used to classify images of ENT Department Locate the five sensory organs of ENT Department, Ophthalmology Department, Stomatology Department. It is more effective to perform collection and recognition more accurately, to perform physical examination and images classification by remote controlled mode and self controlled mode. Further, in robot main control system, it is necessary to collect data, classification medical images, speech interaction between robot system and users, motion interaction to implement medical data self-controlled collection and self-controlled analysis and self-controlled diagnose in robot system and remote controlled physical examination. Further, in robot main control system, machine vision device, cameras, sensors and collection devices are provided and used to collect physical examination data and images of ear, eye, mouth, teeth,and temperature, weight, height. Further, in robot main control system, speech model is used to recognition speech, to input and output text by speech, to speech interaction between robot system and users by sound control command and sound induction. Further, in robot main control system, motion model which include motion planning model, motion collection model which are used to plan motion and to motion interaction between robot system and users, to collect images. Further, motion planning model is including examination motion of ENT department Ophthalmology Department , Stomatology Department and collection motion and motion interaction between robot system and users.

Medical data collection and image collection are performed by robot arm and motion planning model. The target collection method in ENT Department will be proceed in the following order. STEP1: Set targets. STEP2: Set target parameters which are including target name,left arm link and joint ,right arm link and joint. STEP3: Set parameters of communication target. STEP4: Public targets and Set parameters of targets which are including target position and the position mark. STEPS: Mark the position of targets. STEP6: Set parameters of head target which are including head Id, position of target and the values of X location,Y location,Z location. STEP7: Set timestamp. STEP8: Set position mark as origin location(0,0,0) and the values of X location ,Y location ,Z location.

Novel communication method between camera and collection device is as bellowing steps, the description will proceed in the following order. Step: Initialize point cloud node. Step2: Set parameters of public nodes (robot arm) which are including target name, mark of target position. Step3: Set parameters of subscribe nodes (camera) which are including point cloud listed in the point cloud lists. Step4: Define point cloud list and return the list. Step5: Define the nearest points and transfer the points into points array cloud list and return the list. Step6: Compute COG. Step7: Confirm parameters of point cloud and return information of point cloud. Step8: Set parameters of position as point. Step9: Public COG as position of target. Step10: Set parameters of targets which are including mark of target position.time stamp, target ofhead ID, COG position of target, values of X location ,Y location,Z location .

Step1l: Public target node (robot arm).

The collection method of target of ENT Department is as bellowing steps, The description will proceed in the following order. Step: Set location of collection zone location. Step2: Re-plan collection motion is permitted when motion planning is failed. Step3: Set parameters of target zone location of ENT Department. Step4: Set the time limit of motion planning each time. Step5: Set parameters of target zone location which is including medical collection location, seat location, medical assistive machine location. Step6: Set collection zone location,chairs location, medical assistive machine location which are marked in the demo, Set parameters which are including medical bed ID, color of collection zone and standard point position(area.pose.position.x,area.pose.position.y,area.pose.position.z,area.pose.orientation; w.seat.pose.position.x,seat.pose.position.yseat.pose.position.z,seat.pose.c.w; medical assistive machine pose.position.x, medical assistive machine.pose.position.y, medical assistive machine.pose.position.z, medical assistive machine.pose.orientation.w) Set above parameters in the medical examination demo. Step7: Recognition targets of ENT department. Step8: Set colors in the demo. Step9: Set color label in the collection zone, seat color, medical assistive machine color. Step10: Set color label in the demos which are including initial demo, monitor demo and the differ between former demo and later demo. Set public color labels of supine position demo, left lateral position demo, right lateral position demo.

A Novel Neural network method for classification face image is as bellowing steps: S1, Design novel face model and ear model ,nose model, eye model ,mouth model,teeth model S2, Extract features of ear model,eye model,nose model, mouth model, teeth model which are including a set of facial features such as color features,shape pattern features, outlines features. S3, Input examination items features value. S4, Improve weight optimizer, train images and output values. S5, According to output value, classify ear images ,eye images,nose images, mouth images, teeth images, locate five sensory organs more accurately and return the location information of five sensory organs.

SUMMARY OF INVENTION CONTENT OF INVENTION There is a medical robot apparatus provided. The apparatus is including a plurality of joint modules configured to connect camera links and multi-sensors link structure to implement intelligent collection to solve physical examination problems such as effectiveness of physical examination, too expensive human resource cost and too heavy work pressure. A medical robot apparatus is provided. The patent is with medical image recognition functions, with medical data analysis functions, with warning diseases of five sensory organs functions and with healthy data management functions. In the recent years, It is very hard to recognition medical images. In order to perform tasks faster and more accuracy, robot apparatuses have been designed to solve these problems. The patent is effect to recognition medical images more accuracy. The medical robot apparatus is also controlled by remote controlled mode and self-controlled mode. In order to perform tasks faster and more accuracy, robot apparatuses have been designed to solve these problems. In order to improve the accuracy to analysis medical data and to recognition medical images of ENT Department, Ophthalmology Department, Stomatology Department. Physical examination robot apparatuses and physical examination system have been designed to solve these problems.

DESCRIPTION OF DIAGRAM: FIG.1 is a model diagram for describing an application example of machine vision device, sensor, medical data collection device, medical images collection device. FIG.2is a construction diagram for describing an application example of examination data analysis collection illustrating an external appearance of a robot arm apparatus. FIG.3 marks are as bellowing. - robot main control system device; 201,202 - machine vision device; 30 - sensors; 101 - speech device; 103 - robot arm;

DESCRIPTION OF EMBODIMENTS: There is robot apparatuses device and a medical robot main control system provided. The system is used to control this robot apparatuses which are including a medical robot arm apparatus and physical examination devices. It is effect to recognition medical image of five sensory organs. In order to perform tasks faster and more accuracy, robot apparatuses have been designed to solve these problems. These are many problems such as effectiveness of physical examination, too expensive human resource cost and too heavy work press. Machine vision devices are including cameras and sensors. Collection devices are used to collect physical examination data which are including the data of eareye,mouth,teeth and temperature, weight ,height. A medical robot apparatus is provided. The medical robot apparatus are with recognition functions of medical image, with analysis functions of medical data, with warning diseases functions of five sensory organs , and with management functions of health data. A medical robot apparatus with machine vision devices of joint modules, robot arm of joint modules, medical physical examination apparatus of joint modules, speech recognition device, medical data analysis device, sensors, and various of examination devices to implement physical examination of ear, eye, mouth, teeth and temperature, weight, height. It is widely used in the medical field and in the industrial field.

REVIEW OF MEDICAL ROBOT APPARATUS: In application case 1: An application example of a robot apparatus for the medical data collection, analysis and examination system, the robot device includes the following as FIG. 1. FIG. 1. illustrates an exemplary medical procedure by the robot apparatus according to the present. Robot main control system 10 is used to perform control tasks, data collection tasks by machine vision device and sensor, robot main control system communicates with robot arm103 to perform collection tasks and motion planning tasks. Robot main control system 10 communicates with speech recognition device to perform speech interaction between robot main system device device10 and users. A machine vision device 20, camera, sensors30 are used to collect physical examination data including medical data of ear, eye, mouth, teeth and temperature, weight, height. A robot arm103 of joint modules connects a plurality of links. Robot motion planning collection device is used to motion planning. A drive control module which is configured to control driving of the arm joint modules based on states of the joint modules and arm motion planning. According to joint modules which are used to collect physical examination data including medical data of eareye,mouth,teeth and temperature, weight ,height. A machine vision device20 of joint modules, which camera and sensors are used to collect physical examination data including ear, nose, eye, mouth, teeth and temperature, weight height. Robot main control system device 10 is connected to machine vision device20 and it is also connected to robot arm 103. Robot main system device 10 communicates with speech recognition device. Robot main system device 10 is also connected to data collection and examination devices 103 of ENT department. Robot main system device 10 is connected to robot arm103 which is connected to speech recognition devicel0l. Robot main system device 10 communicates with speech recognition device101. In this application example of a robot apparatus, robot main system and depth camera are connected together to collect medical images of ENT department. The sensors are used to measure weight, height and temperature, sound input and output devices are used to speech interaction. A machine vision device of joint modules ,which camera and sensors collection device is used to collect physical examination data including eareye,mouth,teeth and temperature, weight ,height. According to the command of robot main system device10, public image data and communicate with image recognition node and recognition location of nose, ear, eye, mouth, teeth. robot arm103 move to the location of nose, ear, eye, mouth, teeth. robot main system device 10 will return the location of nose, ear, eye, mouth, teeth. The robot arm 103 will move to the collection location by TF package and locate the nose, ear, eye, mouth, teeth of ENT department. A robot apparatus and robot main control system 10 with control modules configured to control driving of the joint modules based on states of the joint modules. Design robot motion. Robot main control system 10 plan motion and collect data. Design robot motion and location of collect motion and angle of the motion to perform human - robot motion interaction and collect data more accurately. Robot main control system 10 command robot arm 30 to planing motion and moving to the location of nose, ear, eye, mouth, teeth. Robot main control system 10 and robot arm 30 plan all of the motion by motion package. Motion interaction tasks and collection motion tasks are performed more effectively and more accurately by motion planing model.

In application case 2: An application example of a robot arm apparatus is according to an embodiment of the present disclosure for a medical purpose will be described as FIG.2. The location and recognition method of ENT department is described as bellowing steps FIG.2. The recognition method description of ENT department will proceed in the following order. S1, Improve the neural network algorithm and novel medical image recognition model is proposed which are including the features of hand, mouth and the feature value extracted of medical image recognition such as color, shape, outline. S2, Input features values. S3, Design the novel method related to the novel neural network algorithm and improved BP algorithm. Weight values are self- trained by weight improve tool which is designed in this patent.The computation results are output. S4, According to output value range, distinguish the normal and abnormal diseases and healthy problem. S5, Extract facial outline features of nose, ear, eye, mouth, teeth of ENT department. and color features. Improve cluster algorithms. According to algorithms, classification the facial image of ENT department. S6, Improve neural network algorithms, Input outline features and color features, according to train weight value and speed parameters rate to improve neural network algorithms to reduce training count and speed up recognition effectively to solve disease recognition problem.

Location method of ENT department is described. Improve cluster algorithm method and design outline features model of eye, ear, mouth and improve cluster algorithm to classification location image of eye,mouth,ear by moving robot arm. Medical data collection model of ear, eye, mouth is used to collect medical data,examination of ENT department and weight and height measures. Improve cluster algorithm method and classification and recognition images of eyes. Return location information of eyes and locate eyes. Transform locations by transformation matrix of coordinates package. Design outline feature model of eye, ear, mouth and improve cluster algorithm to classification location image of eye,mouth,ear,nose by moving robot arm to the target location where is parallel to targets. Attract sight by light mark and emotion mark and evaluate the amblyopia test data and refraction test data. To analyse examination data of eyes and evaluate the ametropic amblyopia and astigmatism by cylindro-spherometer. According to fundamentals of sensors, images transform by a set of micro-lens array and imaging on the sensor. Transform them to evaluate the ametropic amblyopia and astigmatism by cylindro-spherometer computing. Compare normal value to evaluate the ametropic amblyopia and astigmatism, warning disease. The growth of eye axis induces that astigmatism status is from hyperopia to myopia, ametropic amblyopia , normal dioptre is reducing by aging up. Transform location by transformation matrix of coordinates package. Design outline features model of eye, ear, mouth and improve cluster algorithm to classification location image of eye,mouth,ear by moving robot arm to the target location. Collect medical images of ear by moving robot arm which is connected to camera and ear examination device. According to normal data, distinguish and warning the disease data and ear detection screening. Extract the features of ear image and recognition the ear images. Evaluate the ear examination by robot arm which are connected to acoustimeter and otoacoustic emission devices to test ear data and warning disease. Collect medical images and data of mouth and teeth by moving robot arm which are connected to camera and examination device of mouth. Locate the mouth and teeth, return location data of the mouth and teeth. Classification and recognition the location in the mouth images and teeth images by novel neural network algorithms proposed. A set of motions are planned, For example, motion interaction between user and robot, such as mouth collection motion , ear and nose collection motion. Collect medical images which are used to recognition the number of teeth and location of dental caries. Extract the features of ear images and recognition the ear images. Evaluate the examination by robot arm which are connected to acoustimeter and otoacoustic emission devices to test ear data and warning disease. Improve cluster method and design outline features of five sensory organs and improve cluster method to classification location image of five sensory organs by moving robot arm. Medical data collection model of five sensory organs is used to collect medical data for examination of ENT department and for measuring weight and height. Improve cluster method, classification method and recognition image method of ENT department and return location of ENT department and locate ENT department. Transform the location by transformation matrix of coordinates package. Design outline features model of five sensory organs and improve cluster method to classification image of five sensory organs by moving robot arm to the target location where is parallel to five sensory organs.

Claims

Claims 1. A five-sense-organ and surgical medical data acquisition, analysis and diagnosis integrated robot and a platform, wherein, include the bellowing devices and modules A robot main control system, a drive control module, which is configured to control driving of the joint modules based on states of the joint modules. A robot apparatus and robot main control system devices are used to control robot. The control function module which is configured to control another joint function modules. The robot main control system device and control function module connects to machine vision devices, sensors, examination devices of ear, eye, mouth, teeth, another examination devices, and which communicates with these devices, medical data analysis model, robot arm motion planing model, medical images recognition model. Machine vision devices of joint modules, which camera,sensors,and collection devices are used to collect physical examination data, such as medical data of ear, eye, mouth, teeth and temperature, weight ,height. Speech model and speech recognition device is used to speech interaction between robot main control system device and users by speech induction and by speech command. Medical physical examination apparatus of joint modules, medical image analysis module is used to analyse, to classify and to recognize physical examination images in ENT Department , in Ophthalmology Department , in Stomatology Department. Medical image classification model is used to classify images of ENT Department. Robot arm and motion planning model are used for examination motion, collection motion and motion interaction between robot system and users in ENT Department, Ophthalmology Department, Stomatology Department.
2. A five-sense-organ and surgical medical data acquisition, analysis and diagnosis integrated robot and a platform, wherein, robot main system is used to control robot. Robot main system is linking to another robot joint devices include machine vision devices which collect images and medical data by control mode. Control mode includes remote controlled mode and robot self controlled mode. Remote controlled mode is used to communicate between robot main system and users. All of parameters are set and controlled by robot self in robot self controlled mode.
3. The robot of claim 1, wherein, location transformation module which is communicated with robot main system which is used to locate all of five sense organisms including ear, eye, nose, mouth, teeth, return the location information to locate. The location transformation module is connected to the robot arm and robot main system to perform movement functions and collection functions of robot arms to collect medical data of five sense organ.
4. The robot of claim 1, wherein, robot arm is connected to the machine vision devices to which collect medical images and medical data. robot arm is connected to robot main system to collect face images, five-sense images, oral cavity images,tongues images, teeth images, outer ear images,outer nose images and ear images to measure ear data and nose data in ENT Department, to compute the values of ENT Department, comparing the normal range, robot arm is connected to the examination devices to measure data in ENT Department.
5.The robot of claim 1, wherein, examination devices of eyes in Ophthalmology Department are connected to robot arms and collection devices which are used to collect medical data of eyes by remote-controlled mode and self-controlled mode to analyse examination data of eyes and to evaluate the ametropic amblyopia and astigmatism by cylindro-spherometer, according to fundamentals of sensors, images transform by a set of micro-lens array and imaging on the sensor, transform them to evaluate the ametropic amblyopia and astigmatism by cylindro-spherometer computing, compare normal value to evaluate the ametropic amblyopia and astigmatism, warning disease. Examine devices are used to collect medical images and medical data of eyes in Ophthalmology Department, to compute the values of Ophthalmology Department, comparing the normal range, robot arm is connected to the examination devices to measure medical data in Ophthalmology Department.
6.The robot of claim 1, wherein, robot arm is connected to the examination devices to measure ear data and nose data in ENT Department, Examination devices of ENT Department are used to collect medical images of ear, to compute the values of ENT Department, comparing the normal range, robot arm is connected to the examination devices to measure ear data in ENT Department.
7.The robot of claim 1,wherein, medical data collection module of oral cavity, oral mucous, tongues, teeth, which extract outline features and color features of mouth, teeth in ENT department. recognize the location information in the oral cavity, teeth location to collect examination data, recognize images in oral cavity to recognize teeth account and oral diseases. Locate collection locations in oral cavity to recognize teeth, teeth diseases, oral diseases in the oral cavity. Improve neural network methods, Input outline features and color features, according to color differences, recognition teeth disease and oval disease of mouth problem in ENT department and in Stomatology Department. According to value range, distinguish the normal and abnormal disease, it is effective to speed up recognition effectively.
8.A five-sense-organ and surgical medical data acquiasition, analysis and diagnosis integrated robot and a platform, wherein, a novel neural network method for classification face image is as bellowing steps: S1, Design novel face model,hand model,mouth model,teeth model; S2, Extract features of ear model,eye model,nose model, mouth model, teeth model which are including a set of facial features such as color features,shape pattern features, outlines features. S3, Input examination items features value. S4, Improve weight optimizer, train images and output values. S5, According to output value, classify ear images ,eye images,nose images, mouth images, teeth images, locate five sensory organs more accurately and return the location information of five sensory organs.
9.A five-sense-organ and surgical medical data acquisition, analysis and diagnosis integrated robot and a platform, wherein, a novel neural network method for classification image, design novel face model and ear model ,nose model, eye model ,mouth model ,teeth model; Extract features of ear model,eye model,nose model, mouth model, teeth model which are including a set of facial features such as color features,shape pattern features, outlines features; Input examination items features value;Improve weight optimizer, train images and output values; According to output value, classify ear images ,eye images,nose images, mouth images, teeth images, locate five sensory organs more accurately and return the location information of five sensory organs.
10. The robot of claim 1, wherein, robot main system is connected to robot arm, examination devices are connected to robot arm. Medical data and medical images are collected by robot arm and motion plan method, to collect physical examination images and medical data in ENT Department. In robot main control system, motion models which include motion planning model, motion collection model which are used to plan motion and to motion interaction between robot system and users, to collect images and medical data in ENT department. Motion planning model is including examination motion, collection motion and motion interaction between robot system and users in ENT Department, Ophthalmology Department, Stomatology Department.