KR20220150649A - Brain Visualization System using Holo Lens - Google Patents

Abstract

The present invention relates to a brain visualization system using a HOLO lens and, more specifically, to a brain visualization system using a HOLO lens, which comprises: a HOLO lens device which photographs a motion of a user to generate motion data, and outputs an image including a 3D subject on which the motion data is reflected based on holographic technology; a subject providing server into which the motion data is input from the HOLO lens device, and which generates the 3D subject on which the motion data is reflected; and a monitoring unit which outputs the image by being synchronized with the HOLO lens device such that a plurality persons except for a user wearing the HOLO lens device can monitor the image. The 3D subject is the brain of a person, and the subject providing server is connected to an Internet network and cumulatively stores brain data including the structure, the name, and the texture of the brain of the person. Therefore, the structure of the complex brain can be easily grasped by using the HOLO lens.

Description

Brain Visualization System using Holo Lens}

The present invention relates to a brain visualization system using a HoloLens, and more specifically, to use a HoloLens that outputs an image interacting with a user's environment based on a holographic technology, so as to easily grasp the complex structure of the brain. It is about a brain visualization system using a hololens that visualizes the brain.

Conventional medical students at medical educational institutions can practice on frozen human or animal brains when practicing brain anatomy. Medical students can actually check the theoretically learned structure of the brain through brain anatomy practice, which is a practice object, but there is a limit to repetitive learning due to limited practice opportunities and practice objects. In addition, the frozen brain cannot maintain its shape intact over time at room temperature, so medical students performing the practice have to feel uncomfortable or uncomfortable.

In addition, the medical staff explains the area of the disease to a person suspected of having a brain disease or a person with a brain disease through an image taken by MRI, etc. There are technical limitations that are difficult to explain and thus make it difficult for patients to clearly understand their disease.

To solve this problem, various techniques that can visualize the brain for brain dissection practice or diagnosis aid are being developed. Related Document 1 relates to a visualization system for deep brain stimulation, which includes a camera system, a display system, and an information analyzer to visualize the operation of a group of electrodes at a position corresponding to the actual position of the electrode group in the patient's brain. However, since it is displayed in 2D and cannot display the electrodes transmitted from the inner side of the brain, there is a limit to accurately identifying the part of the brain.

Related Document 2 relates to automatic 3D segmentation and reconstruction of cortical surfaces from MRI, and can perform automatic 3D image segmentation and reconstruction of organ structures, which are particularly well suited for use on cortical surfaces, but at each outer surface of the brain. There is a technical limitation in that it is impossible to separate or combine detailed components of the part and inner surface of the product, or to display the laminated cross-sectional area in detail.

KR 10-2020-0045474 KR 10-1966575

The present invention is to solve the above problems, and to enable medical students to practice repetitive brain anatomy, and to enable a suspected brain disease or a brain disease patient to understand according to an accurate description of the disease, based on holographic technology, the user's A HoloLens that includes a HoloLens device that outputs an image including a 3D object that reflects motion data, an object providing server that creates a 3D object, and a monitoring unit that allows multiple users other than users wearing the HoloLens device to monitor the image The purpose of this study is to obtain a brain visualization system using

In order to achieve the above object, the brain visualization system using the hololens of the present invention generates motion data by photographing the user's motion, and outputs an image including a 3D object reflecting the motion data based on holographic technology a HoloLens device; an object providing server receiving the motion data from the HoloLens device and generating a 3D object reflecting the motion data; and a monitoring unit that synchronizes with the HoloLens device and outputs the image so that a plurality of users other than users wearing the HoloLens device can monitor the image, wherein the 3D object is a human brain, and the object The provision server is characterized in that it is connected to an Internet network and accumulates cerebral data including the structure, name, and texture of the human cerebrum.

As described above, according to the present invention, by using a hololens that outputs images interacting with the user's environment based on holographic technology, the brain is visualized so that the complex structure of the brain can be easily grasped, so that medical students can repetitively It is possible to practice anatomy, and it can be used as a diagnosis aid device when the medical staff accurately explains the brain disease to a person suspected of having a brain disease or a person with a brain disease. It has the effect of enabling accurate understanding of brain diseases.

1 is a block diagram of a brain visualization system using the hololens of the present invention.
Figure 2 is a view showing that the components of the brain are separated (a) and coupled (b) from the separation and coupling module according to an embodiment of the present invention.
3 is a diagram showing that element information is provided from an element information providing module according to an embodiment of the present invention.
4 is a view showing cut surfaces according to movements (a)-(f) of a slide bar according to an embodiment of the present invention.
5 is a diagram showing that a motion menu (a) and an information menu (b) are activated according to an embodiment of the present invention.

The terms used in this specification have been selected from general terms that are currently widely used as much as possible while considering the functions in the present invention, but these may vary depending on the intention of a person skilled in the art, precedent, or the emergence of new technologies. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, not simply the name of the term.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this application, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. 1 is a block diagram of a brain visualization system using the hololens of the present invention.

Referring to FIG. 1 , the brain visualization system using the HoloLens of the present invention includes a HoloLens device 100, an object providing server 200, and a monitoring unit 300.

More specifically, the hololens device 100 generates motion data by capturing motions of a user, and outputs an image including a 3D object reflecting the motion data based on holographic technology.

In general, HoloLens outputs a scanned 3D image of a real object on a real screen, unlike Virtual Reality (VR) that shows a complete virtual screen or Augmented Reality (AR) that overlays a real environment. It is based on Mixed Reality (MR), which can freely manipulate 3D images.

That is, the 3D object of the present invention is most preferably a human cerebrum, and may be a 3D image obtained by scanning the cerebrum so that it can be used for brain anatomy practice by medical students or for assisting medical staff in diagnosing brain diseases.

In addition, the hololens device 100 includes a camera, so that motion data can be generated after recognizing various gestures taken by the user. For example, when a user gazes at a certain object and performs a gesture of pressing and releasing a finger in the air, the object may be activated as it is regarded as clicking the object.

Next, the object providing server 200 receives the motion data from the hololens device 100 and creates a 3D object reflecting the motion data. Here, the object providing server 200 is connected to the Internet network and accumulates cerebral data including the structure, name, and texture of the human cerebrum.

And the object providing server 200 separates and combines module 210, element information providing module 220, cut surface providing module 230 and brain disease information providing module 240 to generate a 3D object reflecting the motion data. can provide.

First, the separating and combining module 210 may set one or more components included in the 3D object based on the cerebral data, and separate or combine each preset component according to the motion data.

In other words, by using the cerebral data in the 3D image in which the human cerebrum is scanned, one or more components included in the cerebrum are assigned colors, materials, etc. to distinguish each component.

Figure 2 is a view showing that the components of the brain are separated (a) and coupled (b) from the separation and coupling module 210 according to an embodiment of the present invention. Referring to (a) of FIG. 2 , a state in which the 3D object, the human cerebrum, is coupled from the separate coupling module 210 is displayed. That is, the separation/combining module 210 may display the 3D objects combined in an initial state.

On the other hand, referring to (b) of FIG. 2 , when the motion data is input from the hololens device 100, the separation/combining module 210 connects the 3D object, the human cerebrum, to one preset one. The above components can be separated. On the other hand, when the motion data is input, the separation/combining module 210 may adjust the distance between components when separating one or more components, most preferably, the minimum distance may be in a state in which the 3D object is combined, , the maximum distance may be arbitrarily limited from the separation coupling module 210. That is, the distance between components of the 3D object may be adjusted between a minimum distance and a maximum distance according to the motion data.

Next, the element information providing module 220 may classify the elements by color and material based on the cerebral data, and provide element information for each element according to the motion data.

3 is a diagram showing that element information is provided from the element information providing module 220 according to an embodiment of the present invention. Referring to FIG. 3 , the element information providing module 220 may most preferably provide a Brodmann Area. In general, Brodmann's area was defined by German neurologist Brodmann by dividing it into 52 functional areas according to the characteristic structure and arrangement of brain cells.

That is, the element information providing module 220 may provide an inner surface and an outer surface for the cerebrum cut as a longitudinal section of the human cerebrum, and each Brodmann area is indicated by numbers on the inner and outer surfaces. and the name of each area can be provided in the form of a list.

Meanwhile, the element information providing module 220 may provide a navigator including the 3D object. The navigator may be provided in a regular hexahedron or rectangular parallelepiped shape around the 3D object, and may provide each vertex. This is because the user wearing the HoloLens device 100 makes various gestures in the air to move the 3D object when the 3D object is identified. If the navigator is not displayed, it is difficult for the user to check the reference point for the gesture. This is to solve problems that are difficult to control, such as moving, rotating, and resizing 3D objects.

Next, the cut plane providing module 230 may provide the cut planes stacked according to the movement data for moving the slide bar displayed in the image so that the cut plane of the human brain can be confirmed.

4 is a view showing cut surfaces according to movements (a)-(f) of a slide bar according to an embodiment of the present invention. Referring to (a) of FIG. 4 , the cut plane providing module 230 may previously store a photographed copy of the 3D object cut into a plurality of cross sections. Here, the number of sections of the slide bar may most preferably be the same as the number of photographed copies of the 3D object. In addition, the cutting plane providing module 230 may display only the navigator and the slide bar without stacking any photographed copies in an initial state.

4(b) to (f), when the slide bar is adjusted according to the motion data, the cross section providing module 230, which is previously stored in the section of the corresponding slide bar, is displayed as It can be provided to be displayed on top of the 3D object. Therefore, the cut plane providing module 230 can make the 3D object visible from the hololens device 100 as being stacked on a cross section in a state in which there is no 3D object.

Conversely, when the slide bar is adjusted so that the number of cross sections in which the 3D objects are stacked decreases according to the movement data, the cross section previously stored in the corresponding section of the slide bar is converted to the cross section of the 3D object. You can provide it to appear at the top. Accordingly, the cut plane providing module 230 has an effect of providing a user wearing the hololens device 100 with various shapes and structures that can be confirmed in a cross section of the human brain, which is a 3D object.

In other words, the present invention can provide an internal surface and an external surface for the cerebrum cut in the longitudinal section of the human cerebrum through the element information providing module 220, and through the cut surface providing module 230 By providing a cerebrum cut into a cross-section of the cerebrum, there is a remarkable effect of allowing the user to repeatedly learn and confirm the theoretically known part or the part that could only be confirmed in a small number of anatomical exercises without time and space limitations. In addition, rather than loading and outputting a photo format file stored in 2D at a specific point in time, it has a remarkable effect of allowing the user to learn and check as close to reality as possible by outputting it in 2D and 3D format at any point in time.

Next, the brain disease information providing module 240 interlocks with the external medical institution server 400 to receive clinical data related to brain disease, and display the brain disease on the 3D object based on the clinical data. .

That is, the medical institution server 400 can accumulate and store clinical data for a large number of people with brain disease, suspected brain disease, and people without brain disease. The clinical data may include MRI images of the brain, diagnostic charts of medical staff, and the like. Here, the brain disease information may most preferably include location information indicating a region where any brain disease occurs in the human cerebrum, and the name and symptom of the brain disease. Then, the hololens device 100 may receive the brain disease information from the brain disease information providing module 240 and display an arbitrary brain disease on a portion corresponding to the human cerebrum, which is the 3D object.

Accordingly, the present invention makes it easy for medical staff to explain the brain disease to patients with brain diseases, suspected brain diseases, or their guardians by using a diagnosis aid, and the patients who listen to the explanation are also aware of the brain diseases they suffer from or are suspected of. understanding can be enhanced. In addition, there is a remarkable effect of easily learning the region where any brain disease occurs for medical students studying the brain.

Next, the hololens device 100 may output a moving menu (Moving Active) and an information menu (Show Information) to the image in the form of a toggle button. The HoloLens device 100 may further output various other menus, for example, a home menu for returning to the main screen.

5 is a diagram showing that a motion menu (a) and an information menu (b) are activated according to an embodiment of the present invention. Referring to (a) of FIG. 5, first, when the movement menu (Moving Active) is activated by a user's movement, the object providing server 200 controls at least one of the movement, rotation, and resizing of the 3D object. A signal may be transmitted to the hololens device 100 so as to be possible.

On the other hand, when the moving menu (Moving Active) is activated by the user's movement, as the navigator is provided from the element information providing module 220 around the 3D object, the object providing server 200 includes the 3D object It is possible to provide the navigator that does. Here, the bounding box described in the drawings according to an embodiment of the present invention may most preferably be the navigator, and when the user's movement data activates or deactivates the toggle button type bounding box, the navigator may or may not be displayed. . In addition, the user may perform various gestures based on the navigator to control at least one of movement, rotation, and size adjustment of the 3D object. Therefore, by providing the navigator, the object providing server 200 has a remarkable effect of allowing the user to generate more accurate motion data.

5 (b), when the information menu (Show Information) is activated by a user's movement, the object providing server 200 fixes the 3D object and then activates each component so that the activation is possible. A signal may be transmitted to the hololens device 100 . In addition, when motion data for pressing a certain element is input, the object providing server 200 may provide a description of the corresponding element together with the 3D object in the image. This is to fix the 3D object in order to provide a description of the corresponding component, since there is a problem in that it is difficult to accurately convey information about the corresponding position when the 3D object moves.

That is, the present invention can provide not only element information provided in the form of a list provided from the element information providing module 220, but also the name of a component that the user wants to know directly from the 3D object in the form of a speech bubble on one side of the 3D object. .

Next, the monitoring unit 300 outputs the image in synchronization with the hololens device 100 so that a plurality of users other than the user wearing the hololens device 100 can monitor the image.

If the present invention is provided with only the holo-lens device 100 and the object providing server 200, there is a technical limitation that only one person wearing the holo-lens device 100 can check the 3D object. That is, the present invention can overcome technical limitations by providing the monitoring unit 300 so that not only the user wearing the HoloLens device 100 but also many others can monitor the image together.

Accordingly, the user wearing the HoloLens device 100 can control the 3D object and can monitor the synchronized 3D objects in real time, so that the user wearing the HoloLens device 100 and a plurality of viewers watching through the monitoring unit 300 can share the content in real time.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and/or the components of the system, structure, device, circuit, etc. described in are combined or combined in a different form than the described method, or in a different configuration. Appropriate results can be achieved even when substituted or substituted by elements or equivalents.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

100.. HoloLens device
200.. object provision server
210.. Separate coupling module
220.. Element information providing module
230.. Cutting plane providing module
240.. Brain disease information provision module
300.. monitoring unit
400.. Medical institution server

Claims (5)

a HoloLens device that generates motion data by photographing a user's motion and outputs an image including a 3D object to which the motion data is reflected based on holographic technology;
an object providing server receiving the motion data from the HoloLens device and generating a 3D object reflecting the motion data; and
a monitoring unit that synchronizes with the HoloLens device and outputs the image so that a plurality of users other than users wearing the HoloLens device can monitor the image;
The 3D object is a human cerebrum,
The brain visualization system using the HoloLens, characterized in that the object providing server is connected to the Internet network and accumulates cerebral data including the structure, name, and texture of the human cerebrum. According to claim 1,
The object providing server,
a separating/combining module that sets one or more components included in the 3D object based on the cerebral data and separates or combines each preset component according to the motion data; and
A brain visualization system using a HoloLens comprising: an element information providing module that classifies the components by color and material based on the cerebral data and provides element information on each component according to the motion data. According to claim 1,
The object providing server,
A brain visualization system using a HoloLens, further comprising: a cut plane providing module for providing the cut planes stacked according to the motion data that moves the slide bar displayed in the image so that the cut plane of the human brain can be identified. According to claim 1,
The object providing server,
A brain disease information providing module that receives clinical data related to brain disease in conjunction with an external medical institution server and provides brain disease information based on the clinical data so that any brain disease can be displayed on the 3D object; Brain visualization system using HoloLens. According to claim 2,
The hololens device,
A movement menu and an information menu may be output in the form of a toggle button on the image,
When the movement menu is activated by a user's movement, the object providing server transmits a signal to the HoloLens device to control at least one of movement, rotation, and resizing of the 3D object;
When the information menu is activated by a user's movement, the object providing server fixes the 3D object and transmits a signal to the HoloLens device so that each component can be activated. brain visualization system.

Images