CN113807207A - Human body meridian recognition method and device based on multiple cameras and human body meridian conditioning equipment - Google Patents
Human body meridian recognition method and device based on multiple cameras and human body meridian conditioning equipment Download PDFInfo
- Publication number
- CN113807207A CN113807207A CN202111004279.7A CN202111004279A CN113807207A CN 113807207 A CN113807207 A CN 113807207A CN 202111004279 A CN202111004279 A CN 202111004279A CN 113807207 A CN113807207 A CN 113807207A
- Authority
- CN
- China
- Prior art keywords
- meridian
- human body
- recognition
- camera
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003750 conditioning effect Effects 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013528 artificial neural network Methods 0.000 claims abstract description 28
- 238000006243 chemical reaction Methods 0.000 claims description 56
- 239000011159 matrix material Substances 0.000 claims description 33
- 230000009466 transformation Effects 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 19
- 230000010354 integration Effects 0.000 claims description 11
- 230000008569 process Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 3
- 238000001467 acupuncture Methods 0.000 description 9
- 238000012549 training Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000002175 menstrual effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 210000002784 stomach Anatomy 0.000 description 1
Images
Abstract
The application relates to a human body meridian recognition method and device based on multiple cameras and human body meridian conditioning equipment, wherein the method comprises the following steps: acquiring human body images acquired by cameras arranged on human body meridian conditioning equipment; inputting each human body image into a pre-constructed meridian recognition neural network, and recognizing each human body image by the meridian recognition neural network to obtain a meridian recognition result of each human body image; and integrating the meridian recognition results to obtain a final human meridian recognition result. Compared with a meridian identification result obtained by meridian identification of a human body image acquired from a single angle in the related technology, the method has the advantages that multi-angle identification of the human body image is effectively increased, the situation that meridian identification is omitted due to the fact that the human body image is acquired from the single angle is effectively avoided, and finally accuracy and integrity of the human body meridian identification result are effectively guaranteed.
Description
Technical Field
The application relates to the technical field of intelligent medical conditioning mechanical equipment, in particular to a human body meridian recognition method and device based on multiple cameras and human body meridian conditioning equipment.
Background
Deep learning is a new field in machine learning, and the motivation is to establish and simulate a neural network of human brain for analysis and learning, and to simulate the mechanism of human brain through the established neural network for data analysis, such as: images, sounds, text, etc. The human body channels and collaterals are recognized by adopting a deep learning mode, and the method has a good effect of improving the efficiency of recognizing the acupoints by the human body channels and collaterals. However, in the related art, when the meridian of the human body is identified through the established neural network, since the acquired image is acquired through a single camera, and the single camera cannot acquire all images of the human body, the accuracy and integrity of the identification result obtained when the meridian of the human body is identified through deep learning are not high enough.
Disclosure of Invention
In view of this, the present application provides a method for recognizing human meridians based on multiple cameras, which can effectively improve the accuracy and integrity of the recognition result of the human meridians.
According to an aspect of the application, a method for recognizing human meridians based on multiple cameras is provided, which comprises the following steps:
acquiring human body images acquired by cameras arranged on human body meridian conditioning equipment;
inputting each human body image into a pre-constructed meridian recognition neural network, and recognizing each human body image by the meridian recognition neural network to obtain a meridian recognition result of each human body image;
and integrating the meridian identification results to obtain a final human meridian identification result.
In a possible implementation manner, when the meridian recognition results are integrated to obtain a final human meridian recognition result, an operation of performing coordinate transformation on each meridian recognition result is included.
In a possible implementation manner, when performing coordinate transformation on each meridian recognition result, a transformation matrix from each camera to a mechanical arm in the human body meridian conditioning equipment is obtained through camera calibration, and the coordinate transformation of each meridian recognition result is performed based on the transformation matrix.
In one possible implementation manner, the performing coordinate transformation on each meridian recognition result based on a transformation matrix from each camera to a mechanical arm in the human body meridian conditioning device includes:
acquiring two-dimensional coordinates of each recognition target in the meridian recognition result under a corresponding camera coordinate system;
adding corresponding depth information to the two-dimensional coordinates of each recognition target and converting the two-dimensional coordinates into three-dimensional coordinates of the recognition target under a corresponding camera coordinate system;
and multiplying the three-dimensional coordinates of each recognition target under the corresponding camera coordinate system by the conversion matrix through points to obtain the three-dimensional coordinates of each recognition target under the mechanical arm coordinate system.
According to another aspect of the present application, there is also provided a multi-camera based human meridian recognition apparatus, including: the system comprises an image acquisition module, an image identification module and a result integration module;
the image acquisition module is configured to acquire human body images acquired by cameras configured on the human body meridian conditioning equipment;
the image identification module is configured to input each human body image into a pre-constructed meridian identification neural network, and the meridian identification neural network identifies each human body image to obtain a meridian identification result of each human body image;
the result integration module is configured to integrate the meridian recognition results to obtain a final human body meridian recognition result.
In a possible implementation manner, the system further comprises a coordinate conversion processing module;
the coordinate conversion processing module is configured to perform coordinate conversion processing on each meridian recognition result when the result integration module performs integration processing on each meridian recognition result to obtain a final human meridian recognition result.
In a possible implementation manner, when the coordinate conversion processing module is configured to perform coordinate conversion on each meridian recognition result, a conversion matrix from each camera to a mechanical arm in the human body meridian conditioning equipment is obtained through camera calibration, and the coordinate conversion of each meridian recognition result is performed based on the conversion matrix.
In a possible implementation manner, the coordinate conversion processing module includes a coordinate obtaining sub-module, a depth information adding sub-module, and a coordinate conversion sub-module;
the coordinate acquisition submodule is configured to acquire two-dimensional coordinates of each recognition target in the meridian recognition result under a corresponding camera coordinate system;
the depth information adding submodule is configured to add corresponding depth information to the two-dimensional coordinates of each recognition target and then convert the two-dimensional coordinates into three-dimensional coordinates of the recognition target under a corresponding camera coordinate system;
and the coordinate conversion submodule is configured to multiply the three-dimensional coordinates of each recognition target in the corresponding camera coordinate system by the conversion matrix through points to obtain the three-dimensional coordinates of each recognition target in the mechanical arm coordinate system.
According to another aspect of the present application, there is also provided a human meridian conditioning apparatus including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to execute the executable instructions to implement any of the methods described above;
further comprising:
a camera;
the number of the cameras is multiple, and the cameras are respectively arranged at different positions of the human body meridian conditioning equipment.
In one possible implementation manner, the number of the cameras is four;
the four cameras are respectively arranged at the front side and the top of the left side, the right side and the top of the conditioning bed in the human body meridian conditioning equipment.
According to the method, when the human body channels and collaterals are identified, the plurality of cameras are configured on the human body channels and collaterals conditioning equipment, the plurality of cameras are used for collecting human body images from different angles respectively, then the human body channels and collaterals are identified based on the human body images collected from different angles, and then the channels and collaterals identification results identified by the human body images from different angles are integrated to obtain a final human body channels and collaterals identification result, so that the obtained human body channels and collaterals identification result is more complete. Compared with a meridian identification result obtained by carrying out meridian identification on a human body image acquired from a single angle in the related technology, the method has the advantages that the multi-angle identification of the human body image is effectively increased, the situation that meridian identification is omitted due to the fact that the human body image is acquired from the single angle is effectively avoided, and finally the accuracy and the integrity of the human body meridian identification result are effectively guaranteed.
Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the application and, together with the description, serve to explain the principles of the application.
FIG. 1 is a flow chart of a multi-camera based human meridian identification method according to an embodiment of the present application;
fig. 2 is a schematic diagram illustrating an installation structure of each camera in a human body meridian conditioning device in a multi-camera-based human body meridian recognition method according to an embodiment of the present application;
fig. 3 shows a block diagram of a multi-camera based human meridian recognition device according to an embodiment of the present application.
Detailed Description
Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers can indicate functionally identical or similar elements. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
Furthermore, in the following detailed description, numerous specific details are set forth in order to provide a better understanding of the present application. It will be understood by those skilled in the art that the present application may be practiced without some of these specific details. In some instances, methods, means, elements and circuits that are well known to those skilled in the art have not been described in detail so as not to obscure the present application.
Fig. 1 shows a flowchart of a multi-camera based human meridian recognition method according to an embodiment of the present application. As shown in fig. 1, the method includes: and step S100, acquiring human body images acquired by all cameras arranged on the human body meridian conditioning equipment. Here, it should be noted that, in the method of the embodiment of the present application, the human body meridian conditioning device may be an apparatus for implementing human body meridian conditioning, as shown in fig. 2. The method of the embodiment of the application can be particularly applied to instruments for conditioning the channels and collaterals of the human body. A plurality of cameras are installed on the human body meridian conditioning equipment, and each camera is installed at different positions of the human body meridian conditioning equipment, so that the cameras can collect human body images from different directions.
After the human body images in different directions are acquired through the cameras arranged on the human body meridian conditioning equipment, step S200 can be executed, each human body image is input into a pre-constructed meridian recognition neural network, and each human body image is recognized by the meridian recognition neural network, so as to obtain a meridian recognition result of each human body image. Further, in step S300, the meridian recognition results are integrated to obtain a final meridian recognition result.
Therefore, when the human body channels and collaterals are identified, the plurality of cameras are configured on the human body channels and collaterals conditioning equipment, the plurality of cameras are used for collecting human body images from different angles respectively, then the human body channels and collaterals are identified based on the human body images collected from different angles, and then the channels and collaterals identification results identified by the human body images from different angles are integrated to obtain a final human body channels and collaterals identification result, so that the obtained human body channels and collaterals identification result is more complete. Compared with a meridian identification result obtained by carrying out meridian identification on a human body image acquired from a single angle in the related technology, the method has the advantages that the multi-angle identification of the human body image is effectively increased, the situation that meridian identification is omitted due to the fact that the human body image is acquired from the single angle is effectively avoided, and finally the accuracy and the integrity of the human body meridian identification result are effectively guaranteed.
It should be noted that, in the method of the embodiment of the present application, when each human body image is input into a pre-constructed meridian recognition neural network and each human body image is recognized by the meridian recognition neural network to obtain a meridian recognition result of each human body image, the constructed meridian recognition neural network may adopt a target recognition network model that is conventional in the art, and details are not described here.
Meanwhile, it should be noted that, when the pre-constructed meridian recognition neural network is used for recognizing the meridians of the human body, the meridian recognition neural network needs to be trained first. For training of the meridian recognition neural network, a corresponding training sample set needs to be constructed, and each training sample in the training sample set needs to be labeled. Here, it should be noted that, in the method of the embodiment of the present application, the training samples in the training sample set are constructed by actually shooting and collecting various human body images from a network, and marking the image data of the corresponding human body acupuncture points in each human body image in combination with the human body acupuncture point map.
Furthermore, after the human body images of different angles collected by the cameras are input into the meridian recognition neural network and the meridian recognition neural network recognizes the acupuncture points in the human body images, the meridian recognition results of the human body images can be integrated to be used as the final human body meridian recognition result. In a possible implementation manner, the process of integrating the meridian recognition result of each human body image includes an operation of performing coordinate transformation on the meridian recognition result of each human body image.
This is because the meridian recognition result of each human body image is a two-dimensional coordinate in each camera coordinate system, and the coordinate systems of different cameras are different. Meanwhile, the coordinate system of each camera is different from the coordinate system of the mechanical arm in the human meridian conditioning equipment. Therefore, in order to facilitate integration of the meridian recognition results of the individual human body images, it is necessary to unify the coordinate systems of the meridian recognition results of the individual human body images. In a possible implementation manner, the coordinates of the meridian recognition result of each human body image can be completely converted into the coordinates of a mechanical arm coordinate system in the human body meridian conditioning equipment, so that the positions of all acupuncture points can be smoothly and accurately recognized when the mechanical arm performs massage or conditioning according to the recognized acupuncture points.
Specifically, when the coordinate conversion is performed on each meridian recognition result, a conversion matrix from each camera to a mechanical arm in human body meridian conditioning equipment can be obtained through camera calibration, and the coordinate conversion of each meridian recognition result is performed based on the conversion matrix. Namely, the coordinate system calibration is carried out on each camera, the coordinate system calibration is carried out on the mechanical arm in the human body meridian conditioning equipment, and then the conversion matrix of each camera and the mechanical arm is obtained according to the calibrated coordinate system of each camera and the coordinate system of the mechanical arm.
The coordinate transformation matrix of the image acquisition equipment and the mechanical arm can be realized by respectively calibrating the coordinates of the image acquisition equipment and the mechanical arm and then correspondingly calculating according to the calibrated coordinates of the image acquisition equipment and the mechanical arm. In a possible implementation manner, the coordinate calibration of the image capturing device and the mechanical arm may be performed in a checkerboard manner.
Specifically, the image acquisition equipment and the mechanical arm are calibrated by using a checkerboard through urx open source libraries and opencv open source libraries, the image acquisition equipment acquires three-dimensional coordinates of the center of the checkerboard under coordinates of the image acquisition equipment, the mechanical arm acquires three-dimensional coordinates of the checkerboard center under a coordinate system of the mechanical arm, the mechanical arm walks according to certain 3 x 3 grids to acquire checkerboard center coordinates under the coordinate system of 9 groups of image acquisition equipment and checkerboard center coordinates under the coordinate system of the mechanical arm, and a conversion matrix from the coordinates of the image acquisition equipment to the coordinates of the mechanical arm (namely, a coordinate conversion matrix from the image acquisition equipment to the coordinates of the mechanical arm) can be obtained through calculation
It should be noted that there should be a conversion matrix for the different cameras to the robotic arm. And when the coordinates of the identified hole sites in each human body image are converted, the conversion is carried out according to a conversion matrix of the camera and the mechanical arm for collecting each human body image.
More specifically, when the coordinate conversion of each meridian recognition result is performed based on the conversion matrix from each camera to the mechanical arm in the human body meridian conditioning equipment, the coordinate conversion can be realized in the following manner.
First, two-dimensional coordinates of each recognition target in the meridian recognition result under a corresponding camera coordinate system are acquired. And then, adding corresponding depth information to the two-dimensional coordinates of each recognition target, and converting the two-dimensional coordinates into three-dimensional coordinates of the recognition target under a corresponding camera coordinate system. And then, the three-dimensional coordinates of each recognition target in the corresponding camera coordinate system are subjected to dot-product transformation matrix to obtain the three-dimensional coordinates of each recognition target in the mechanical arm coordinate system.
Here, it should be noted that the two-dimensional coordinates of each recognition target in the acquired meridian recognition result in the corresponding camera coordinate system can be directly realized according to the positions of the acupuncture points recognized by the meridian recognition neural network on the human body image. Namely, each identification target is a human body acupuncture point identified from the human body image. The two-dimensional coordinates of each recognition target under the corresponding camera coordinate system can be obtained according to the positions of each human body acupoint identified in the human body image. Here, as will be understood by those skilled in the art, obtaining two-dimensional coordinates of each acupoint in the corresponding camera coordinate system according to the position of each acupoint in the human body image can be implemented by means of conventional techniques in the art, such as: the mapping relationship between the internal parameters and the external parameters of each camera is directly realized, and the details are not repeated here.
Further, after adding corresponding depth information to the two-dimensional coordinates of each recognition target, the two-dimensional coordinates are converted into three-dimensional coordinates of the recognition target in the corresponding camera coordinate system, and then the addition of the corresponding depth information can be realized through Azure Kinect SDK, which is not described here any more.
After each recognition target in the meridian recognition result is converted from two-dimensional coordinates to three-dimensional coordinates by any one of the above methods, the coordinate data of each recognition target is also the coordinate data in the corresponding camera coordinate system. Therefore, the robot arm is required to smoothly and accurately recognize the positions of the human body acupuncture points by converting the coordinate data of each recognition target into the robot arm coordinate system.
According to the foregoing, in the method of the embodiment of the present application, the conversion of each recognition target in the camera coordinate system to the robot arm coordinate system may be realized by dot-by-dot multiplication of the conversion matrix. Here, it should be noted that for meridian recognition results of different human body images, it is necessary to use different coordinate transformation matrices between the camera and the robot arm.
For example, four cameras are arranged in the human body meridian conditioning equipment, and each camera is respectively located at different positions of the human body meridian conditioning equipment, namely a camera a, a camera B, a camera C and a camera D. The human body image PictrueA is collected by the camera A, the human body image PictrueB is collected by the camera B, the human body image PictrueC is collected by the camera C, and the human body image PictrueD is collected by the camera D.
And sequentially inputting the human body image PictrueA, the human body image PictrueB, the human body image PictrueC and the human body image PictrueD into a meridian recognition neural network, respectively carrying out meridian recognition on each human body image by the meridian recognition neural network, and recognizing corresponding human body acupuncture points in each human body image. At this time, the coordinate data of each identified human body acupoint is under the corresponding camera coordinate system. Therefore, the coordinates of each human body acupoint in each human body image need to be converted into the mechanical arm coordinate system.
In the coordinate conversion, for each human body acupoint identified from the human body image a, conversion needs to be performed based on a conversion matrix between the camera a and the mechanical arm. For each body acupoint identified from the body image B, a conversion is required based on a conversion matrix between the camera B and the mechanical arm. Similarly, for each body acupoint identified from the body image C, conversion needs to be performed based on the conversion matrix between the camera C and the mechanical arm. For each body acupoint identified in the body image D, a conversion needs to be performed based on the conversion matrix between the camera C and the robotic arm.
Here, it should be noted that the coordinate calibration of each camera and the coordinate calibration of the mechanical arm may be implemented by using a coordinate calibration technical means that is conventional in the art, and a transformation matrix between each camera and the mechanical arm, which is obtained based on the calibrated coordinate system of each camera and the coordinate system of the mechanical arm, may also be implemented by using a conventional technical means that is conventional in the art, which is not described herein again.
Correspondingly, based on any one of the human body meridian recognition methods based on the multiple cameras, the application also provides a human body meridian recognition device based on the cameras. Because the working principle of the human body meridian recognition device based on the multiple cameras provided by the application is the same as or similar to the principle of the human body meridian recognition method based on the multiple cameras provided by the application, repeated parts are not repeated.
Referring to fig. 3, the multi-camera based human meridian recognition apparatus 100 provided by the present application includes an image acquisition module 110, an image recognition module 120, and a result integration module 130. The image acquisition module 110 is configured to acquire human body images acquired by cameras configured on the human body meridian conditioning equipment. And the image recognition module 120 is configured to input each human body image into a pre-constructed meridian recognition neural network, and recognize each human body image by the meridian recognition neural network to obtain a meridian recognition result of each human body image. And a result integrating module 130 configured to integrate the meridian recognition results to obtain a final human meridian recognition result.
In a possible implementation manner, a coordinate conversion processing module (not shown in the figure) is further included. Wherein, the coordinate conversion processing module is configured to perform coordinate conversion processing on each meridian recognition result when the result integrating module 130 performs integration processing on each meridian recognition result to obtain a final human meridian recognition result.
In a possible implementation manner, when the coordinate conversion processing module is configured to perform coordinate conversion on each meridian recognition result, a conversion matrix from each camera to a mechanical arm in human body meridian conditioning equipment is obtained through camera calibration, and the coordinate conversion of each meridian recognition result is performed based on the conversion matrix.
In one possible implementation, the coordinate conversion processing module includes a coordinate obtaining sub-module, a depth information adding sub-module, and a coordinate conversion sub-module (none of which is shown in the figure). And the coordinate acquisition submodule is configured to acquire two-dimensional coordinates of each recognition target in the meridian recognition result under the corresponding camera coordinate system. And the depth information adding submodule is configured to add corresponding depth information to the two-dimensional coordinates of each recognition target and then convert the two-dimensional coordinates into three-dimensional coordinates of the recognition target under a corresponding camera coordinate system. And the coordinate conversion submodule is configured to multiply the three-dimensional coordinates of each recognition target in the corresponding camera coordinate system by a point-by-point conversion matrix to obtain the three-dimensional coordinates of each recognition target in the mechanical arm coordinate system.
Still further, according to another aspect of the present application, there is also provided a human meridian conditioning apparatus 200. Referring to fig. 2, the human meridian conditioning apparatus 200 according to the embodiment of the present application includes a processor (not shown), a camera, and a memory (not shown) for storing processor-executable instructions. Wherein the processor is configured to execute the executable instructions to implement any one of the above-mentioned multi-camera based human meridian identification methods.
Here, it should be noted that the number of processors may be one or more. Meanwhile, in the human body meridian conditioning apparatus according to the embodiment of the present application, an input device and an output device (neither shown) may be further included. The processor, the memory, the input device, and the output device may be connected by a bus, or may be connected by other means, and are not limited specifically herein.
The memory, which is a computer-readable storage medium, may be used to store software programs, computer-executable programs, and various modules, such as: the intelligent conditioning method provided by the embodiment of the application corresponds to a program or a module. The processor executes various functional applications and data processing of the human meridian conditioning apparatus by operating software programs or modules stored in the memory.
The input device may be used to receive an input number or signal. Wherein the signal may be a key signal generated in connection with user settings and function control of the device/terminal/server. The output means may comprise a display device such as a display screen.
The number of the cameras is multiple, and the cameras are respectively arranged at different positions of the human body meridian conditioning equipment. In one possible implementation manner, the number of the cameras can be four, and the four cameras are respectively arranged at the left side, the right side, the head side and the top of the conditioning bed in the human body meridian conditioning equipment.
Specifically, referring to fig. 2, the at least one camera 220 is disposed above one side of the conditioning bed 210, i.e., the at least one camera 220 is disposed above the left side of the conditioning bed 210. When a human body lies on the back on the conditioning bed 210, the camera 220 arranged above the left side of the conditioning bed 210 is positioned obliquely above the left side surface of the human body, and the meridians of the left side and the inner side of the right leg of the human body can be obtained through the camera 220; when the human body lies on the conditioning bed 210 in the prone position, the camera 220 disposed above the left side of the conditioning bed 210 is located obliquely above the right side of the human body, and the meridians of the right side and the inner side of the left leg of the human body can be obtained by the camera 220. The at least one camera 220 is disposed above the other side of the conditioning bed 210, i.e., the at least one camera 220 is disposed above the right side of the conditioning bed 210. When a human body lies on the back on the conditioning bed 210, the camera 220 arranged above the right side of the conditioning bed 210 is positioned obliquely above the right side surface of the human body, and the meridians of the right side and the inner side of the left leg of the human body can be obtained through the camera 220; when the human body lies on the conditioning bed 210 in the prone position, the camera 220 disposed above the right side of the conditioning bed 210 is located obliquely above the left side surface of the human body, and the meridians of the left side and the inner side of the right leg of the human body can be obtained through the camera 220. The at least one camera 220 is disposed above one end of the conditioning bed 210, i.e., the at least one camera 220 is disposed above the head end of the conditioning bed 210. The camera 220 provided above the head end of the conditioning bed 210 is located obliquely above the head of the human body. When the human body lies on the back on the conditioning bed 210, the meridians of the top of the head, the face and the shoulders can be obtained through the camera 220; when the human body lies prone on the conditioning bed 210, the meridians at the top of the head and the shoulders and the back can be obtained through the camera 220. The at least one camera 220 is disposed directly above the conditioning bed 210, i.e., the at least one camera 220 is disposed directly above the conditioning bed 210. When the human body lies on the back on the conditioning bed 210, the meridians on the front side of the human body can be obtained through the camera 220; when the human body lies on the conditioning bed 210 in the prone position, the meridians on the back side of the human body can be obtained through the camera 220. Note that the camera 220 is a depth camera or a 3D camera. On the whole, when the human body lies on the back or the stomach, the channels of the human body can be identified in all directions, which is beneficial for the massager 120 to locate and search each channel, and effectively improves the conditioning effect. Here, it should be noted that each of the camera 220, the driver 128, and the robot arm 140 is electrically connected to the controller. Each camera 220 can transmit the acquired menstrual flow information to the controller. The controller controls the robot arm 140 to work to control the massager 120 to locate and find each meridian. The controller controls the first motor 122 to operate via the driver 128. Moreover, the cameras 220 work in a mutually matched mode, and the human body channels are identified in different regions in a multi-angle mode, so that the identification precision is effectively improved, the influence of image distortion is reduced, the shielding risk is reduced, the positioning accuracy of the massager 120 is improved, and the conditioning effect is improved.
In an embodiment of the human meridian conditioning apparatus 200 of the present application, a first support bar 230 and a second support bar 240 are further included. The two first support rods 230 are vertically arranged and respectively arranged at two opposite sides of the conditioning bed 210, and the upper parts of the two first support rods are bent towards the right above the conditioning bed 210 to form a first fixing part 231. At least one camera 220 is fixed on one side surface of the first fixing part 231 of one first supporting rod 230 facing to the right upper side of the conditioning bed 210, and at least one camera 220 is fixed on one side surface of the first fixing part 231 of the other first supporting rod 230 facing to the right upper side of the conditioning bed 210. The first support bar 230 and the second support bar 240 are integrally of an arc-shaped structure, and have better mechanical strength. The first fixing portion 231 is of a convex structure, and a lighting device can be further mounted on the first fixing portion 231 to improve the conditioning environment. The second support rod 240 is vertically disposed at one end of the conditioning bed 210, and the upper portion thereof is bent toward the right above the conditioning bed 210 to form a second fixing portion 241. It should be noted that the second support rod 240 is disposed at the head end of the conditioning bed 210, i.e. the second support rod 240 is disposed at the end of the bed board 112 with the larger cross section. At least one camera 220 is fixed on the bottom surface of the second fixing portion 241 of the second supporting bar 240, and at least one camera 220 is fixed on one side surface of the middle portion facing the conditioning bed 210. The orthographic projection of the second fixing portion 241 from top to bottom is a convex structure, and a lighting device is mounted on the bottom surface of the second fixing portion 241 to improve the conditioning environment. The lighting device comprises a plurality of LED lamps, and the LEDs are arranged in an array manner. In addition, an included angle between the axial direction of the detection port of the camera 220 fixed at the middle of the second support rod 240 and the plane of the bed plate 112 is 30-60 degrees, an included angle between the axial direction of the detection port of the camera 220 fixed at the second fixing portion 241 and the plane of the bed plate 112 is 85-95 degrees, and an included angle between the axial direction of the detection port of the camera 220 fixed at the first fixing portion 231 and the plane of the bed plate 112 is 30-60 degrees. Thus, the obtaining effect of the camera 220 in obtaining the meridians is improved, the shielding risk is reduced, the positioning accuracy of the massager 120 is improved, and the conditioning effect is improved.
Having described embodiments of the present application, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the techniques in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (10)
1. A human body meridian identification method based on multiple cameras is characterized by comprising the following steps:
acquiring human body images acquired by cameras arranged on human body meridian conditioning equipment;
inputting each human body image into a pre-constructed meridian recognition neural network, and recognizing each human body image by the meridian recognition neural network to obtain a meridian recognition result of each human body image;
and integrating the meridian identification results to obtain a final human meridian identification result.
2. The method of claim 1, wherein the integrating process of the meridian recognition results to obtain the final meridian recognition result of the human body includes an operation of performing coordinate transformation of the meridian recognition results.
3. The method according to claim 2, wherein when performing coordinate transformation on each meridian recognition result, a transformation matrix from each camera to a mechanical arm in the human meridian conditioning equipment is obtained through camera calibration, and the coordinate transformation of each meridian recognition result is performed based on the transformation matrix.
4. The method of claim 3, wherein the performing of the coordinate transformation of each meridian recognition result based on the transformation matrix of each camera to a mechanical arm in the human meridian conditioning device comprises:
acquiring two-dimensional coordinates of each recognition target in the meridian recognition result under a corresponding camera coordinate system;
adding corresponding depth information to the two-dimensional coordinates of each recognition target and converting the two-dimensional coordinates into three-dimensional coordinates of the recognition target under a corresponding camera coordinate system;
and multiplying the three-dimensional coordinates of each recognition target under the corresponding camera coordinate system by the conversion matrix through points to obtain the three-dimensional coordinates of each recognition target under the mechanical arm coordinate system.
5. A human meridian recognition device based on multiple cameras is characterized by comprising: the system comprises an image acquisition module, an image identification module and a result integration module;
the image acquisition module is configured to acquire human body images acquired by cameras configured on the human body meridian conditioning equipment;
the image identification module is configured to input each human body image into a pre-constructed meridian identification neural network, and the meridian identification neural network identifies each human body image to obtain a meridian identification result of each human body image;
the result integration module is configured to integrate the meridian recognition results to obtain a final human body meridian recognition result.
6. The apparatus of claim 5, further comprising a coordinate transformation processing module;
the coordinate conversion processing module is configured to perform coordinate conversion processing on each meridian recognition result when the result integration module performs integration processing on each meridian recognition result to obtain a final human meridian recognition result.
7. The apparatus according to claim 6, wherein the coordinate transformation processing module, when configured to perform coordinate transformation on each meridian recognition result, obtains a transformation matrix from each camera to a mechanical arm in the human meridian conditioning device through camera calibration, and performs coordinate transformation on each meridian recognition result based on the transformation matrix.
8. The apparatus of claim 7, wherein the coordinate conversion processing module comprises a coordinate acquisition sub-module, a depth information adding sub-module, and a coordinate conversion sub-module;
the coordinate acquisition submodule is configured to acquire two-dimensional coordinates of each recognition target in the meridian recognition result under a corresponding camera coordinate system;
the depth information adding submodule is configured to add corresponding depth information to the two-dimensional coordinates of each recognition target and then convert the two-dimensional coordinates into three-dimensional coordinates of the recognition target under a corresponding camera coordinate system;
and the coordinate conversion submodule is configured to multiply the three-dimensional coordinates of each recognition target in the corresponding camera coordinate system by the conversion matrix through points to obtain the three-dimensional coordinates of each recognition target in the mechanical arm coordinate system.
9. A human meridian conditioning apparatus, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the method of any one of claims 1 to 4 when executing the executable instructions;
further comprising:
a camera;
the number of the cameras is multiple, and the cameras are respectively arranged at different positions of the human body meridian conditioning equipment.
10. The apparatus of claim 8, wherein the number of cameras is four;
the four cameras are respectively arranged at the front side and the top of the left side, the right side and the top of the conditioning bed in the human body meridian conditioning equipment.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111004279.7A CN113807207A (en) | 2021-08-30 | 2021-08-30 | Human body meridian recognition method and device based on multiple cameras and human body meridian conditioning equipment |
| PCT/CN2022/113488 WO2023030036A1 (en) | 2021-08-30 | 2022-08-19 | Multi-camera-based human body meridian identification method and apparatus, and human body meridian conditioning device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111004279.7A CN113807207A (en) | 2021-08-30 | 2021-08-30 | Human body meridian recognition method and device based on multiple cameras and human body meridian conditioning equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113807207A true CN113807207A (en) | 2021-12-17 |
Family
ID=78942053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111004279.7A Pending CN113807207A (en) | 2021-08-30 | 2021-08-30 | Human body meridian recognition method and device based on multiple cameras and human body meridian conditioning equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113807207A (en) |
| WO (1) | WO2023030036A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023030036A1 (en) * | 2021-08-30 | 2023-03-09 | 中科尚易健康科技(北京)有限公司 | Multi-camera-based human body meridian identification method and apparatus, and human body meridian conditioning device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10124658A (en) * | 1996-05-22 | 1998-05-15 | Electron & Telecommun Res Inst | Method for correcting image distortion of camera by utilizing neural network |
| CN107137225A (en) * | 2017-06-22 | 2017-09-08 | 山东中医药大学 | Personalized Head And Face acupuncture point identification model is set up, the method and system of positioning |
| CN108765546A (en) * | 2018-04-18 | 2018-11-06 | 北京奇虎科技有限公司 | Acupuncture point detection method and device |
| CN110232318A (en) * | 2019-05-06 | 2019-09-13 | 平安科技（深圳）有限公司 | Acupuncture point recognition methods, device, electronic equipment and storage medium |
| CN110464633A (en) * | 2019-06-17 | 2019-11-19 | 深圳壹账通智能科技有限公司 | Acupuncture point recognition methods, device, equipment and storage medium |
| CN210405493U (en) * | 2019-06-28 | 2020-04-24 | 西安工程大学 | Matrix type multi-view camera system |
| CN112184705A (en) * | 2020-10-28 | 2021-01-05 | 成都智数医联科技有限公司 | Human body acupuncture point identification, positioning and application system based on computer vision technology |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110188699A (en) * | 2019-05-31 | 2019-08-30 | 安徽柏络智能科技有限公司 | A kind of face identification method and system of binocular camera |
| CN111414812A (en) * | 2020-03-03 | 2020-07-14 | 平安科技（深圳）有限公司 | Human body attribute identification method, system, computer device and storage medium |
| CN113807207A (en) * | 2021-08-30 | 2021-12-17 | 中科尚易健康科技(北京)有限公司 | Human body meridian recognition method and device based on multiple cameras and human body meridian conditioning equipment |
-
2021
- 2021-08-30 CN CN202111004279.7A patent/CN113807207A/en active Pending
-
2022
- 2022-08-19 WO PCT/CN2022/113488 patent/WO2023030036A1/en unknown
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10124658A (en) * | 1996-05-22 | 1998-05-15 | Electron & Telecommun Res Inst | Method for correcting image distortion of camera by utilizing neural network |
| CN107137225A (en) * | 2017-06-22 | 2017-09-08 | 山东中医药大学 | Personalized Head And Face acupuncture point identification model is set up, the method and system of positioning |
| CN108765546A (en) * | 2018-04-18 | 2018-11-06 | 北京奇虎科技有限公司 | Acupuncture point detection method and device |
| CN110232318A (en) * | 2019-05-06 | 2019-09-13 | 平安科技（深圳）有限公司 | Acupuncture point recognition methods, device, electronic equipment and storage medium |
| CN110464633A (en) * | 2019-06-17 | 2019-11-19 | 深圳壹账通智能科技有限公司 | Acupuncture point recognition methods, device, equipment and storage medium |
| CN210405493U (en) * | 2019-06-28 | 2020-04-24 | 西安工程大学 | Matrix type multi-view camera system |
| CN112184705A (en) * | 2020-10-28 | 2021-01-05 | 成都智数医联科技有限公司 | Human body acupuncture point identification, positioning and application system based on computer vision technology |
Non-Patent Citations (1)
| Title |
|---|
| 马哲文 等: "基于视觉定位的按摩机器人穴位跟踪系统", 《机器人技术与应用》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023030036A1 (en) * | 2021-08-30 | 2023-03-09 | 中科尚易健康科技(北京)有限公司 | Multi-camera-based human body meridian identification method and apparatus, and human body meridian conditioning device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023030036A1 (en) | 2023-03-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10875187B2 (en) | Robotic arm camera system and method | |
| CN106679651B (en) | Sound localization method, device and electronic equipment | |
| CN110559077B (en) | Coordinate system registration method, robot control method, device, equipment and medium | |
| US20100249595A1 (en) | System and method for automatic calibration of tracked ultrasound | |
| CN109297413A (en) | A kind of large-size cylinder body Structural visual measurement method | |
| WO2023030036A1 (en) | Multi-camera-based human body meridian identification method and apparatus, and human body meridian conditioning device | |
| JP7241933B2 (en) | Image registration method and related model training method, device, apparatus | |
| CN109760107A (en) | A kind of robot localization Accuracy Assessment based on monocular vision | |
| CN115284292A (en) | Mechanical arm hand-eye calibration method and device based on laser camera | |
| CN113657184A (en) | Evaluation method and device for piano playing fingering | |
| CN110421565B (en) | Robot global positioning and measuring system and method for practical training | |
| CN113807204A (en) | Human body meridian recognition method and device, equipment and storage medium | |
| KR101683763B1 (en) | Augmented Reality Robot Simulation System And Method Using 360 Degree Camera | |
| CN115317097A (en) | Robot navigation control method and device, electronic equipment and storage medium | |
| EP4291127A1 (en) | Systems and methods for registration between patient space and image space using registration frame apertures | |
| CN113808184A (en) | Interpolation method and device based on point cloud identification, equipment and storage medium | |
| Sabatini et al. | Toward low-cost vision-based 2D localisation systems for applications in rehabilitation robotics | |
| CN111399634A (en) | Gesture-guided object recognition method and device | |
| CN111466948B (en) | Ultrasonic scanning method and ultrasonic scanning device | |
| CN114299120B (en) | Compensation method, registration method, and readable storage medium | |
| US9335167B2 (en) | Apparatus for measuring body size and method of doing the same | |
| CN214793027U (en) | Portable three-dimensional scanner | |
| CN117398252B (en) | Automatic positioning human back image acquisition method and massage system | |
| CN115615342B (en) | Luminosity stereoscopic vision system calibration method based on visual touch sensor | |
| US20220254040A1 (en) | Systems and methods for registration between patient space and image space using registration frame apertures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211217 |