KR102169613B1 - The method and apparatus for changing user interface based on user motion information - Google Patents

Abstract

Disclosed is a method for changing and providing a user interface (UI) used when examining an object through a medical device based on motion information of a user. According to an embodiment of the present invention, a method of acquiring motion information of a user, changing a UI using the acquired motion information of a user, and displaying a changed UI is disclosed.

Description

Method and apparatus for changing user interface (UTI) based on user's motion information {THE METHOD AND APPARATUS FOR CHANGING USER INTERFACE BASED ON USER MOTION INFORMATION}

The present invention relates to a method and apparatus for changing and providing a user interface based on motion information of a user.

The ultrasound diagnosis apparatus irradiates an ultrasound signal generated from a transducer of a probe to an object, receives information of an echo signal reflected from the object, and provides an internal portion of the object (e.g., a single layer of soft tissue or blood flow). Etc.) can be obtained. In particular, the ultrasound diagnosis apparatus may be used for medical purposes such as observing the inside of an object, detecting foreign substances, and measuring injury.

Such an ultrasound diagnosis apparatus has the advantage of being able to display information on an object in real time. In addition, the ultrasonic diagnostic device has the advantage of high stability because there is no exposure to X-rays, and other image diagnostics such as an X-ray diagnostic device, a computerized tomography (CT) scanner, a magnetic resonance image (MRI) device, and a nuclear medicine diagnostic device. It is widely used with devices.

The present invention relates to a method and apparatus for modifying and providing a user interface (UI) used for examination of an object through a medical device based on motion information of a user.

According to an embodiment of the present invention, a method for providing a user interface (UI) used for examination of an object through a medical device by changing based on motion information of a user is provided. It may include the step, changing the UI by using the obtained motion information of the user, and displaying the changed UI.

Motion information of a user according to an embodiment of the present invention may be obtained based on location information of a current probe.

The location information of the current probe according to an embodiment of the present invention may include at least one of a current indication direction of the probe, an inclination angle with respect to a predetermined reference point, and a height value.

The user motion information according to an embodiment of the present invention may be obtained based on biometric information including at least one of user's fingerprint information, user's iris information, and user's face information.

The UI according to an embodiment of the present invention may include at least one of a shortcut button, a switch, a keyboard, and a trackball indicating a function to be used when examining an object.

Changing the UI using the acquired motion information of the user according to an embodiment of the present invention may include changing at least one of the shape, size, and position of the UI according to the acquired motion information of the user. have.

According to an exemplary embodiment of the present invention, at least one of a shape, a size, and a location of the UI may be provided differently according to the frequency of a function used when examining an object.

The UI may be editable according to an external input signal according to an embodiment of the present invention.

The UI according to an embodiment of the present invention may be displayed through at least one of a display unit on which a captured image of an object is displayed and a control panel equipped with a display function.

The method may further include adjusting at least one of a height and an angle of the control panel according to motion information of a user according to an embodiment of the present invention.

The method may further include acquiring identification information of a user according to an embodiment of the present invention, and storing a UI changed according to the acquired identification information.

According to an embodiment of the present invention, an apparatus for providing a user interface (UI) used for examination of an object through a medical device by changing based on motion information of a user is a motion for obtaining motion information of a user. It may include an information acquisition unit, a UI changing unit that changes the UI using the acquired motion information of the user, and a display unit capable of displaying the changed UI.

The motion information acquisition unit according to an embodiment of the present invention further includes a sensing unit, and motion information of the user may be obtained based on the position information of a current probe obtained through the sensing unit.

The location information of the current probe according to an embodiment of the present invention may include at least one of a current indication direction of the probe, an inclination angle with respect to a predetermined reference point, and a height value.

User motion information according to an embodiment of the present invention may be obtained based on biometric information including at least one of a user's fingerprint information, a user's iris information, and a user's face information acquired through a sensing unit.

The UI according to an embodiment of the present invention may include at least one of a shortcut button, a switch, a keyboard, and a trackball indicating a function to be used when examining an object.

The UI changer according to an embodiment of the present invention may change at least one of the shape, size, and position of the UI according to the acquired motion information of the user.

In the UI according to an embodiment of the present invention, at least one of a shape, a size, and a location may be provided differently according to a frequency of a function used when an object is examined.

The apparatus according to an embodiment of the present invention further includes an external input receiving unit, and the UI may be edited by the UI changing unit according to an external input signal received through the external input receiving unit.

The apparatus according to an embodiment of the present invention further includes a control panel equipped with a display function, and the UI may be displayed through at least one of the display unit and the control panel.

The apparatus according to an embodiment of the present invention further includes a controller, and the controller may adjust at least one of a height and an angle of the control panel according to motion information of the user.

The apparatus according to an embodiment of the present invention may further include an identification information acquisition unit that obtains user identification information, and a storage unit that stores a UI changed according to the obtained identification information.

Meanwhile, according to an embodiment of the present invention, it is possible to provide a computer-readable recording medium in which a program for executing the above-described method on a computer is recorded.

According to the method and apparatus provided as an embodiment of the present invention, the user's motion information is acquired, and the user interface (UI) is changed and displayed based on the acquired motion information, thereby increasing the convenience of the user's use of medical devices. There is an effect that you can. In addition, by automatically controlling the location of a medical device based on user motion information, it is possible to reduce unnecessary time wasted in setting a photographing environment.

1A shows an example of a method for examining an object through a conventional medical device.
1B is a block diagram showing an exemplary configuration of an ultrasound diagnosis apparatus r1000 according to an embodiment of the present invention.
1C is a block diagram showing an exemplary configuration of a wireless probe r2000 according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of a method for providing a user interface (UI) used for examination of an object through a medical device according to an embodiment of the present invention by changing it based on motion information of a user.
3 is a flowchart illustrating a method for providing a user interface (UI) changed based on motion information of a user according to an embodiment of the present invention.
4 shows an example of obtaining motion information of a user according to an embodiment of the present invention.
5 shows an example of motion information of a user obtained according to another embodiment of the present invention.
6 shows an example of a user interface (UI) that can be modified and provided according to an embodiment of the present invention.
7 illustrates an aspect in which a user interface (UI) is changed and provided based on motion information of a user according to an embodiment of the present invention.
8 illustrates an aspect in which a user interface (UI) is changed and provided based on motion information of a user according to another embodiment of the present invention.
9A illustrates an example of editing a user interface (UI) according to an embodiment of the present invention.
9B shows an example of using a modified user interface (UI) according to an embodiment of the present invention.
10 illustrates an aspect in which a user interface (UI) is provided according to an embodiment of the present invention.
11 is a flowchart illustrating a method of controlling an operation of a medical device according to another embodiment of the present invention.
12 illustrates an example of operation control of a medical device according to another embodiment of the present invention.
13 is a flowchart illustrating a method of managing a changed UI according to another embodiment of the present invention.
14 illustrates an apparatus for changing and providing a user interface (UI) used when examining an object through a medical device according to an embodiment of the present invention based on motion information of a user.
15 illustrates a device further including a sensing unit according to an embodiment of the present invention.
16 illustrates a device further including a sensing unit, an external input receiving unit, and a control panel according to an embodiment of the present invention.
17 illustrates a device further including a sensing unit, an external input receiving unit, a control panel, and a control unit according to an embodiment of the present invention.
18 illustrates an apparatus for providing a user interface (UI) changed based on motion information of a user according to another embodiment of the present invention.

The terms used in the present specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected from general terms that are currently widely used while considering functions in the present invention, but this may vary depending on the intention or precedent of a technician working in the field, the emergence of new technologies, and the like. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall contents of the present invention, not a simple name of the term.

When a part of the specification is said to "include" a certain component, it means that other components may be further included rather than excluding other components unless otherwise stated. In addition, terms such as "... unit" and "module" described in the specification mean units that process at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. .

Throughout the specification, the term "ultrasonic image" may mean an image of an object acquired using ultrasound.

In addition, the subject may include a human or an animal, or a part of a human or animal. For example, the subject may include organs such as liver, heart, uterus, brain, breast, and abdomen, or blood vessels. In addition, the object may include a phantom, and the phantom may refer to a material having a volume very close to the density and effective atomic number of an organism.

In addition, throughout the specification, the "user" may be a medical expert, such as a doctor, a nurse, a clinical pathologist, a medical imaging expert, and the like, and may be a technician who repairs a medical device, but is not limited thereto.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are assigned to similar parts throughout the specification.

1A shows an example of a method for examining an object through a conventional medical device.

As shown in FIG. 1A, the user may examine the object 10 using the probe 120 of the ultrasound diagnosis apparatus. In other words, the user may obtain a photographed image of the object 10 through the display unit 110 by manipulating the probe 120 and the control panel 130. In order to accurately diagnose the object 10, a precise photographing image of the object 10 must be obtained beforehand.

A relatively precise photographed image of the object 10 may be obtained according to the user's skill in handling medical devices. However, even an experienced user may experience inconvenience in operation depending on the location of the medical device such as the user's posture when photographing a predetermined part of the object (e.g., liver, kidney, etc.), or it may be difficult to obtain a precise photographed image. have. For example, the ease of use of a medical device may be irrelevant to skill level depending on the photographing part of the object, the height at which the object is located, the user's preference for using the right or left hand, and the frequency of using a predetermined button in a predetermined examination item.

For example, when it is necessary to take a picture by moving the probe 120 in a wide range, it may be difficult to manipulate the control panel 130 according to the user's physical or behavioral characteristics. In other words, as the probe 120 is operated up, down, left, right, etc., the user's posture will also change, and in order to smoothly manipulate the control panel 130, the user may need to re-edit the posture. . In addition, the fixed height, position, etc. of the control panel 130 may make the user feel uncomfortable in operation.

In addition, when the user's gaze 140 changes, such as when the user is forced to look at the display unit 110 from the side according to the user's movement, it may be difficult to obtain or read a precise image of the object.

Therefore, compared to the prior art, it is necessary to increase the convenience of the user's operation of the medical device by automatically optimizing the photographing environment of the medical device according to the location where the user uses the medical device. In addition, the optimization of the photographing environment of such a medical device has an effect of enabling precise photographing of an object and shortening a photographing time.

Prior to describing a specific embodiment of the present invention, an exemplary configuration of an ultrasound diagnosis apparatus according to an embodiment of the present invention will be described below with reference to FIGS. 1B and 1C. For convenience of description, a reference mark (r) is added to each component of the device.

1B is a block diagram illustrating a configuration of an ultrasound diagnosis apparatus r1000 according to an exemplary embodiment of the present invention.

The ultrasound diagnosis apparatus r1000 according to an embodiment of the present invention includes a probe r20, an ultrasound transceiving unit r100, an image processing unit r200, a communication unit r300, a memory r400, an input device r500, and A control unit r600 may be included, and the various components described above may be connected to each other through a bus r700.

The ultrasound diagnosis apparatus r1000 may be implemented not only as a cart type but also as a portable type. Examples of the portable ultrasound diagnosis apparatus may include a PACS viewer, a smart phone, a laptop computer, a PDA, and a tablet PC, but are not limited thereto.

The probe r20 transmits an ultrasonic signal to the object r10 according to a driving signal applied from the ultrasonic transceiving unit r100 and receives an echo signal reflected from the object r10. The probe r20 includes a plurality of transducers, and the plurality of transducers vibrate according to the transmitted electrical signal and generate ultrasonic waves that are acoustic energy. In addition, the probe r20 may be connected to the main body of the ultrasound diagnosis apparatus r1000 by wire or wirelessly, and the ultrasound diagnosis apparatus r1000 may include a plurality of probes r20 according to implementation types.

The transmission unit r110 supplies a driving signal to the probe r20 and includes a pulse generator r112, a transmission delay unit r114, and a pulser r116.

The pulse generation unit r112 generates a pulse for forming a transmission ultrasonic wave according to a predetermined pulse repetition frequency (PRF), and the transmission delay unit r114 determines transmission directionality. The delay time for this is applied to the pulse. Each pulse to which the delay time is applied corresponds to a plurality of piezoelectric vibrators included in the probe r20, respectively.

The pulser r116 applies a driving signal (or driving pulse) to the probe r20 at a timing corresponding to each pulse to which a delay time is applied.

The receiving unit r120 generates ultrasonic data by processing the echo signal received from the probe r20, and an amplifier r122, an analog digital converter (ADC) (r124), a reception delay unit (r126), and It may include a summing unit (r128).

The amplifier r122 amplifies the echo signal for each channel, and the ADC r124 converts the amplified echo signal to analog-digital.

The reception delay unit r126 applies a delay time for determining reception directionality to the digitally converted echo signal, and the summing unit r128 sums the echo signals processed by the reception delay unit r166. Generate ultrasound data.

The image processing unit r200 generates and displays an ultrasound image through a scan conversion process for ultrasound data generated by the ultrasound transmission/reception unit r100.

On the other hand, the ultrasound image is not only a gray scale ultrasound image that scans the object according to the A mode, B mode, and M mode, but also the motion of the object as a Doppler image. Can be indicated. The Doppler image may include a blood flow Doppler image representing blood flow (or also referred to as a color Doppler image), a tissue Doppler image representing tissue movement, and a spectral Doppler image representing the movement speed of the object as a waveform. have.

The B mode processing unit r212 extracts and processes a B mode component from the ultrasound data.

The image generator r220 may generate an ultrasound image in which the intensity of a signal is expressed as brightness based on the B mode component extracted by the B mode processor r212.

Likewise, the Doppler processing unit r214 may extract a Doppler component from ultrasound data, and the image generator r220 may generate a Doppler image representing a motion of an object in color or waveform based on the extracted Doppler component.

The image generator r220 according to an embodiment may generate a 3D ultrasound image through a volume rendering process for volume data, and also generate an elastic image obtained by imaging the degree of deformation of the object r10 according to pressure. May be. Furthermore, the image generator r220 may express various additional information on the ultrasound image as text or graphics. Meanwhile, the generated ultrasound image may be stored in the memory r400.

The display unit r230 displays and outputs the generated ultrasound image. The display unit r230 may display not only an ultrasound image, but also various information processed by the ultrasound diagnosis apparatus r1000 on a screen through a graphical user interface (GUI). Meanwhile, the ultrasound diagnosis apparatus r1000 may include two or more display units r230 according to implementation types.

The communication unit r300 is connected to the network r30 by wire or wirelessly to communicate with an external device or server. The communication unit r300 may exchange data with a hospital server connected through a medical image information system (PACS, Picture Archiving and Communication System) or other medical devices in the hospital. In addition, the communication unit r300 may perform data communication according to a standard for digital imaging and communication in medicine (DICOM).

The communication unit r300 may transmit/receive data related to diagnosis of the object such as ultrasound image, ultrasound data, Doppler data, etc. of the object through the network r30, and medical images captured by other medical devices such as CT, MRI, and X-ray. It can also send and receive. Further, the communication unit r300 may receive information on a diagnosis history or treatment schedule of a patient from the server and use it for diagnosis of an object. Furthermore, the communication unit r300 may perform data communication with a portable terminal of a doctor or patient, as well as a server or medical device in a hospital.

The communication unit r300 may be connected to the network r30 by wire or wirelessly to exchange data with the server r32, the medical device r34, or the portable terminal r36. The communication unit r300 may include one or more components that enable communication with an external device, for example, a short-range communication module r310, a wired communication module r320, and a mobile communication module r330. I can.

The short-range communication module r310 may mean a module for short-range communication within a predetermined distance. Short-range communication technology according to an embodiment of the present invention includes wireless LAN, Wi-Fi, Bluetooth, zigbee, Wi-Fi Direct (WFD), ultra wideband (UWB), infrared communication ( IrDA, infrared data association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), etc. may be, but is not limited thereto.

The wired communication module r320 refers to a module for communication using an electrical signal or an optical signal, and wired communication technology according to an embodiment includes a pair cable, a coaxial cable, an optical fiber cable, an Ethernet cable, etc. May be included.

The mobile communication module r330 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include a voice call signal, a video call signal, or various types of data according to transmission/reception of text/multimedia messages.

The memory r400 stores various pieces of information processed by the ultrasound diagnosis apparatus r1000. For example, the memory r400 may store medical data related to diagnosis of an object, such as input/output ultrasound data and ultrasound images, and may store an algorithm or program performed in the ultrasound diagnosis apparatus r1000.

The memory r400 may be implemented as various types of storage media such as a flash memory, a hard disk, and an EEPROM. In addition, the ultrasound diagnosis apparatus r1000 may operate a web storage or a cloud server that performs a storage function of the memory r400 on the web.

The input device r500 refers to a means for receiving data for controlling the ultrasound diagnosis apparatus r1000 from a user. The input device r500 may include a hardware configuration such as a keypad, a mouse, a touch panel, a touch screen, a trackball, a jog switch, etc., but is not limited thereto, and an electrocardiogram measurement module, a respiration measurement module, a voice recognition sensor, a gesture recognition sensor , Fingerprint recognition sensor, iris recognition sensor, depth sensor, distance sensor, etc. may further include various input means.

The controller r600 may generally control the operation of the ultrasound diagnosis apparatus r1000. That is, the control unit r600 controls the operation between the probe r20, the ultrasonic transceiving unit r100, the image processing unit r200, the communication unit r300, the memory r400, and the input device r500 shown in FIG. 1 can do.

Some or all of the probe (r20), the ultrasonic transceiving unit (r100), the image processing unit (r200), the communication unit (r300), the memory (r400), the input device (r500) and the control unit (r600) can be operated by a software module. However, the present invention is not limited thereto, and some of the above-described components may be operated by hardware.

In addition, at least some of the ultrasound transceiving unit r100, the image processing unit r200, and the communication unit r300 may be included in the control unit r600, but the implementation is not limited thereto.

1C is a block diagram showing a configuration of a wireless probe r2000 according to an embodiment of the present invention.

The wireless probe r2000 includes a plurality of transducers as described with reference to FIG. 1B, and may include some or all of the configuration of the ultrasonic transceiving unit r100 of FIG. 1B according to an implementation form.

The wireless probe r2000 according to the embodiment illustrated in FIG. 1C includes a transmitter r2100, a transducer r2200, and a receiver r2300, and each configuration has been described in 1, so detailed descriptions are omitted. do. Meanwhile, the wireless probe r2000 may selectively include a reception delay unit r2330 and an summing unit r2340 according to the implementation type.

The wireless probe r2000 may transmit an ultrasound signal to the object r10 and receive an echo signal, generate ultrasound data, and wirelessly transmit the ultrasound data to the ultrasound diagnosis apparatus r1000 of FIG. 1.

FIG. 2 is a schematic diagram of a method for providing a user interface (UI) used for examination of an object through a medical device according to an embodiment of the present invention by changing it based on motion information of a user.

As shown in FIG. 2A, it is possible to determine whether the user uses the probe 1200 with the left hand or the probe 1200 with the right hand by detecting the movement direction of the connection line of the probe 1200. For example, if the user is left-handed, the probe 1200 may be used with the left hand. Similarly, if the user is right-handed, the probe 1200 can be used with the right hand.

As described above, according to a user's use of the probe 1200, a user interface (UI) may be provided differently on the control panel.

For example, as shown in FIG. 2B, when the user is a right-handed user, it may be convenient for the user to position the object on his right side and to manipulate the probe using his right hand. Therefore, when the user is a right-handed user, buttons, trackballs, etc. that are included on the control panel and used for shooting need to be provided as the UI pattern 131 for the right hand.

In addition, when the user is a left-handed user, it may be convenient for the user to position the object on his left side and operate the probe using his left hand. In other words, when the user is a left-handed user, a left-handed UI pattern 133 may be provided to the user on the control panel.

In addition, the UI pattern 131 for the right hand and the UI pattern 133 for the left hand on the control panel may be switched based on the user's motion information including the current position of the probe 1200 and the position change.

3 is a flowchart illustrating a method for providing a user interface (UI) changed based on motion information of a user according to an embodiment of the present invention.

According to an embodiment of the present invention, a method for providing a user interface (UI) used when examining an object through a medical device is changed based on motion information of a user, comprising: acquiring motion information of the user (S100), changing the UI by using the acquired motion information of the user (S200) and displaying the changed UI (S300).

According to an embodiment of the present invention, motion information of a user may be obtained (S100).

Using the acquired motion information, a user interface (UI) indicating a button to be used for photographing, a trackball, etc. may be changed (S200). Such changes in the user interface (UI) may include optimizing a medical device photographing environment so that a user can easily manipulate the medical device.

Changes to the user interface (UI), etc., as described above, vary the layout of the UI according to the preference of the user's left or right hand, or the layout and size of the UI depending on the frequency of use of the button used for shooting. Can be provided by adjusting.

The UI changed according to an embodiment of the present invention may be provided to a user through a display unit or the like (S300).

Motion information of a user according to an embodiment of the present invention may be obtained from a location of a probe and biometric information of a user. In this regard, it will be described later with reference to FIGS. 4 and 5.

4 shows an example of obtaining motion information of a user according to an embodiment of the present invention.

Motion information of a user according to an embodiment of the present invention may be obtained based on location information of the current probe 1200.

The location information of the current probe 1200 according to an embodiment of the present invention may include at least one of a current indication direction of the probe 1200, an inclination angle with respect to a predetermined reference point, and a height value.

For example, position information of the current probe 1200 may be obtained according to the direction indicated by the probe 1200. For example, through a predetermined sensor 121 embedded in the probe 1200 (eg, included at the end of the probe for connection to the ultrasonic device) or embedded in the ultrasonic device to which the probe 1200 is to be connected. Current indication direction information of the probe 1200 may be obtained.

The predetermined sensor 121 may be configured to determine a direction in which the probe 1200 is directed from a reference line (eg, the center line ML of the sensor 121) according to which direction the connection line of the probe 1200 is directed. have.

For example, the indication direction of the probe 1200 may be determined through the sensor 121 or the like in the same manner as the toggle switch principle. As shown in FIG. 4A, if the probe 1200 is deflected to the right from the center line ML, the current position information of the probe 1200 is of the ultrasonic device including the display unit 2300 and the control panel 2500. It can be determined that you are pointing to the right. In other words, information that the probe 1200 is located on the right side of the ultrasound apparatus may be obtained.

In addition, as shown in FIG. 4A, if the probe 1200 is deflected to the left from the center line ML, the current position information of the probe 1200 is ultrasonic waves including the display unit 2300 and the control panel 2500. It can be determined that you are pointing to the left side of the device. In other words, information that the probe 1200 is located on the left side of the ultrasound device may be obtained.

In addition, when the probe 1200 according to an embodiment of the present invention is wireless, the predetermined sensor 121 may be configured to sense the coordinates of the GPS (Global Positioning System) of the wireless probe through wireless communication.

In addition, according to an embodiment of the present invention, the location information of the current probe 1200 may include at least one of an inclination angle and a height value for a predetermined reference point.

As shown in FIG. 4B, the probe 1200 may include a sensor 123 for obtaining tilt information or height information. The sensor 123 may include a gyro sensor, a height sensor, or the like.

For example, the location information of the current probe 1200 may be obtained as information on an inclination angle for a predetermined reference point. The predetermined reference point may include a ground on which the medical device is located, an initial position of the patient table or probe 1200 (eg, a position attached to the ultrasound device, etc.).

The user's current motion may be estimated according to the inclination angle information. For example, it may be estimated that the degree of movement of the probe 1200 gradually increases as the width of change of the inclination angle increases, and the range of the movement angle of the user operating the probe 1200 based on the estimated movement of the probe 1200 Motion information on the back may be obtained.

For example, the ground (or patient table) and the probe 1200 may form an angle of up to 90 degrees (eg, when the probe 1200 is positioned perpendicular to the ground). The user may move the probe 1200 in a predetermined direction and angle as photographing proceeds. An angle between the probe 1200 and the ground may be included within a predetermined range (eg, 0 degrees to 90 degrees). For example, when a user wants to photograph liver ultrasound of an object, the probe 1200 may be parallel to the ground (for example, the probe 1200 and the ground are at an angle of 0 degrees) or vertical (eg, probe 1200) according to user manipulation. (1200) and the ground at an angle of 90 degrees) can be moved within a range.

In other words, when the probe 1200 is moved while the probe 1200 is parallel to the ground and the probe 1200 and the ground are perpendicular to each other, the width of the change in the inclination angle can be maximum, and the The motion degree may be estimated to be the maximum, and motion information indicating that the motion degree of the user operating the probe 1200 is also maximum may be obtained based on the estimated motion of the probe 1200.

In addition, similarly, an inclination angle from the position of the initial state in which the probe 1200 was attached to the ultrasound device may be obtained as position information of the current probe 1200.

For example, in the sensor 123 according to an embodiment of the present invention, the orientation toward the right or east side based on the sensor 123 is expressed as a positive angle, and the orientation toward the left or west side is a negative angle. It may be configured to be expressed, but is not necessarily limited thereto.

Accordingly, when the probe 1200 is present while being deflected to the right from the initial position, a positive inclination angle may be sensed on the sensor 123, and the current probe 1200 is the inclination angle sensed on the right side of the sensor 123 Information that it exists at the location of the province can be obtained. In addition, when the probe 1200 is currently deflected to the left from the initial position, a negative tilt angle may be sensed on the sensor 123, and the current probe 1200 is the tilt angle sensed on the left side of the sensor 123 Information that it exists at the location of may be obtained.

Also, the location information of the current probe 1200 may be obtained as height value information for a predetermined reference point. The predetermined reference point may include the ground, the patient's table, or the initial position of the probe 1200 (eg, a position attached to the ultrasound device, etc.). In this case, the predetermined sensor 123 may be a height sensor.

For example, the sensor 123 may sense information that the probe 1200 is currently positioned at a height of about 120 cm from the ground. In addition, the sensor 123 may sense that the probe 1200 is currently positioned about 5 cm lower than the initial state attached to the ultrasonic device, for example. In other words, the height value information for the predetermined reference point may be obtained as position information of the current probe 1200.

Motion information indicating a change in height of a posture of a user operating the probe 1200 may be obtained based on height value information (eg, a change in height, etc.) of the probe 1200 according to an embodiment of the present invention. have.

5 shows an example of motion information of a user obtained according to another embodiment of the present invention.

The user motion information according to an embodiment of the present invention may be obtained based on biometric information including at least one of user's fingerprint information, user's iris information, and user's face information.

User motion information according to an embodiment of the present invention may be obtained based on the user's fingerprint information.

For example, based on the user's fingerprint information 11 acquired through the sensor 125 included in the probe 1200, it is possible to determine whether the user currently grips the probe 1200 with the left or right hand. have.

In other words, from the user's fingerprint information 11 acquired through the sensor 125 included in the probe 1200, motion information on whether the user uses the probe 1200 with the left hand or the right hand can be obtained. have.

For example, if the obtained user's fingerprint information 11 is a fingerprint for the thumb or index finger of the user's right hand, it is determined that the user is currently holding the probe 1200 with the right hand, and the use of the probe 1200 with the user's right hand. Motion information can be obtained.

In addition, if the obtained user's fingerprint information 11 is a fingerprint of the user's left thumb or index finger, it is determined that the user is currently holding the probe 1200 with his left hand, and the use of the probe 1200 with the user's left hand. Motion information can be obtained.

In addition, according to an embodiment of the present invention, based on the user's fingerprint information 11 acquired through the sensor 125 included in the probe 1200, it is possible to identify who is the user currently using the probe. . In this case, the user's fingerprint information 11 may be utilized as user identification information (eg, ID, etc.).

User motion information according to an embodiment of the present invention may be obtained based on at least one of the user's iris information 13 and the user's face information 15.

As shown in FIG. 5, at least one of the user's iris information 13 and the user's face information 15 may be obtained through a sensor 111 provided on the ultrasonic device. The sensor 111 may be located near the display unit 2300 of the ultrasonic device, but is not limited thereto.

In addition, the sensor 111 may perform iris recognition and face recognition functions at the same time, or may be implemented as individual sensors having an iris recognition function and a face recognition function.

According to an embodiment of the present invention, iris information 13 of a user may be obtained through the sensor 111. The user's iris information 13 may include user identification information on who the current user is, and information on the current location of the current user's iris. For example, by reading the user's iris through the sensor 111, information on who the current user is may be obtained.

In addition, a current gaze of the user may be determined using the current location information of the iris obtained through the sensor 111 and motion information of the user may be obtained based on the current gaze of the user. In other words, information about the user's current posture may be obtained depending on whether the iris is positioned to be deflected to the left or to the right of the user's eye, but is not limited thereto.

For example, in the case where the iris of the user's eye is generally deflected to the left, it may be determined that the user's upper body is oriented toward the right. In other words, it may be determined that the user is currently operating the probe 1200 or the like with the right hand.

Similarly, when the iris of the user's eye is positioned to be generally biased to the right, it may be determined that the user's upper body is pointing to the left. It may be determined that the user is currently operating the probe 1200 or the like with his left hand.

According to an embodiment of the present invention, the user's facial information 15 may be obtained through the sensor 111. The user's facial information 15 may include user identification information on who the current user is, and information on the current facial direction of the user. For example, by reading the user's facial feature points and facial contours through the sensor 111, information on who the current user is may be obtained.

In addition, the user's main face may be determined by using the user's face direction information acquired through the sensor 111, and motion information of the user may be obtained based on the user's main face. In other words, the current main face of the user may be determined based on the user's face area, and information on the current posture of the user may be obtained from the determined main face.

For example, by comparing the area of the user's right face and the left face, if the area of the left face is wider, it may be determined that the upper body of the user is oriented toward the right. In other words, it may be determined that the user is currently operating the probe 1200 or the like with the right hand.

Similarly, when the area of the user's right face and the left face are compared and the area of the right face is larger, it may be determined that the user's upper body is oriented toward the left. In other words, it may be determined that the user is currently operating the probe 1200 or the like with his left hand.

In addition, motion information of the user may be obtained using the user's iris information 13 and the user's facial information 15. For example, if the area of the user's right and left face is compared, but it is not possible to clearly distinguish which area is larger, the user's motion is additionally considered as in the above-described method. Information can also be obtained.

6 shows an example of a user interface (UI) that can be modified and provided according to an embodiment of the present invention.

As shown in FIG. 6, according to an embodiment of the present invention, a user interface (UI) (eg, 611 to 615) for displaying operation functions in a medical device may be displayed and provided.

The UI according to an embodiment of the present invention may include at least one of a shortcut button, a switch, a keyboard, and a trackball indicating a function to be used when examining an object.

A user interface (UI) according to an embodiment of the present invention includes a keyboard 611 type UI capable of inputting letters, numbers, etc. in relation to photographing of an object, image enlargement/reduction, resolution adjustment, 2D image and A UI in the form of at least one button or switch 613 representing a predetermined function such as switching between 3D images, a UI in the form of a trackball 615, and the like may be included.

At least one of the aforementioned UI in the form of a keyboard 611, a UI in the form of at least one button or switch 613, and a UI in the form of a trackball 615 may be provided as a virtual UI layout.

In other words, the user interface (UI) according to an embodiment of the present invention does not always have to exist in a physically constant form, but in the form of letters, numbers, images, etc. on the display unit 2300 or the control panel 2500 It may exist as a virtual (virtual) to represent a predetermined function.

7 illustrates an aspect in which a user interface (UI) is changed and provided based on motion information of a user according to an embodiment of the present invention. For convenience of explanation, a user interface (UI) is simplified and shown as shown in FIG. 7.

The step of changing the UI using the motion information of the user obtained according to an embodiment of the present invention (S200) includes changing at least one of the shape, size, and position of the UI according to the obtained motion information of the user. Can include.

As shown in FIG. 7A, the shape of the UI may be changed according to the acquired motion information of the user.

For example, when the movement range of the probe 1200 is very wide, that is, when the movement of the user increases, the user's touch to the user interface (UI) 611 to 615 may be inconvenient. In other words, the user's hand may not reach the user interface (UI), or the user may feel difficult to manipulate the UI.

In this case, as in the exemplary embodiment of the present invention, the user's convenience may be increased by changing the shape of the user interface (UI). For example, the UI in the form of a keyboard 611 of FIG. 6, a UI in the form of a button or switch 613, etc. may be provided by changing the shape of the UI (1 to 3 of 131) of FIG. 7 have. For example, if the user uses button 2 more often than button 1 or button 3 during the photographing process, the shape of the lower side may be changed to a triangle shape with the longest bottom side to increase the probability of accessing the button 2 by the user. In other words, you can change the area of button 2 to be larger than button 1 or button 3.

In addition, according to an embodiment of the present invention, the positions of buttons 1 to 3 may be changed. For example, the UI can be changed to be arranged in the order of button 3, button 1, and button 2. In this case, button 1 may be the button with the highest frequency of use, and button 1 may form a triangle shape with the longest bottom side, but the shape of the UI before and after changes such as the button is not necessarily limited thereto.

In addition, as shown in FIG. 7B, the size of the UI may be changed according to the acquired motion information of the user.

As described above, when the user's movement increases, the size of the UI pattern 131 is entirely or partially increased or decreased, thereby changing the size of the UI so that the user can easily access the UI such as the button, etc. )can do.

For example, by increasing the size of buttons 1 to 3, it is possible to increase the user's accessibility of the buttons. Also, similarly, by changing the positions of buttons 1 to 3 to be close to the user, it is possible to increase the accessibility of the user's button.

In addition, as shown in FIG. 7C, different patterns of UI may be provided according to the user's physical characteristics.

For example, when the user is a right-handed user, a right-handed UI pattern 131 may be provided. In addition, when the user is a left-handed user, a left-handed UI pattern 133 may be provided. The right-handed UI pattern 131 and the left-handed UI pattern 133 may be symmetrical to each other, and, as described above, may be interchangeable based on user motion information.

8 illustrates an aspect in which a user interface (UI) is changed and provided based on motion information of a user according to another embodiment of the present invention.

According to an embodiment of the present invention, at least one of a shape, a size, and a position of the UI may be provided differently according to the frequency of a function used when examining an object.

For example, the UI pattern 131 of FIG. 7 that simplified the UI in the form of a keyboard 611 of FIG. 6, and the UI in the form of a button or switch 613 is shown in FIGS. 8B and 8C. The UI may be provided (137 or 139) by changing at least one of a shape, a size, and a location according to the frequency of the function used at the time. For convenience of explanation, it is assumed that the UI pattern 131 of FIG. 8A is a basic pattern that can be provided by a medical device.

For example, a trackball is not used in a predetermined examination item, and a function corresponding to the second button among the button-type UIs simplified to 1 to 3 may be used most. In this case, as shown in FIG. 8B, the circular UI of the trackball function is omitted from the basic pattern 131, and the size of the UI of the second button may be changed to be the largest.

In other words, by omitting a UI that is not used by a user or having a low frequency of use, and providing a UI with a high frequency of use in a large size, it is possible to increase the accessibility and convenience of a user who operates a medical device to the UI.

As another example, if a user mainly uses a trackball in a predetermined examination item and uses the third button (for example, an enlargement/reduction function, etc.) of a button-type UI most often, as shown in FIG. 8C, the basic pattern 131 In ), the size of button 3 among the circular UI and button-type UI corresponding to the trackball can be greatly changed and provided (139).

Also, similar to the shape of a user's hand, the UI may be arranged and provided in a radial shape like the pattern 139 in FIG. 8C, but the present invention is not limited thereto.

In addition, since the frequency of functions used may be different according to the type of inspection, different patterns of UI can be provided according to the type of inspection.

For example, the frequency of use of functions provided by the ultrasound apparatus may differ according to each type of examination, such as an echocardiography, a liver ultrasound, an abdominal ultrasound, a Pelvic Ultrasonography, and a Doppler ultrasound. For example, in echocardiography, an image enlargement/reduction function and a trackball function are frequently used, whereas in a liver ultrasound examination, an image enlargement/reduction function and a resolution control function may be used more than a trackball function.

In other words, in the echocardiography, a UI that performs an image enlargement/reduction function and a UI that performs a trackball function are relatively larger than those of other UIs, and may be positioned in the center of the control panel 2500.

In contrast, in the liver ultrasound examination, the size of the UI of the image enlargement/reduction function and the UI of the trackball function can be reduced or omitted, and the UI performing the resolution adjustment function in the center of the control panel 2500 is relatively large. You can change the UI pattern so that it is arranged in size.

9A illustrates an example of editing a user interface (UI) according to an embodiment of the present invention.

The UI may be editable according to an external input signal according to an embodiment of the present invention.

For example, UI can be added, deleted, position, and size changed according to user input. In addition, the UI may change language, font, and UI display color according to user input.

For example, the location or size of the button-type UI may be changed according to a user input.

As shown in FIG. 9, the location or size of the UI may be changed based on a user's predetermined input to the UI. A user's predetermined input according to an embodiment of the present invention may include at least one of clicking at least once on the UI and pressing for a predetermined time. In addition, such a predetermined input may include a case where a plurality of inputs are simultaneously input.

For example, as a user input for button 1 of the button UI, an edit start signal is applied by pressing for about 1 to 1.5 seconds, and a drag and drop 21 signal is applied to the button 1 to a target position. You can change the position of the No. 1 button.

The drag-and-drop 21 signal may be a continuous user input signal. In other words, for example, the user input signal of the drag-and-drop 21 for button 1 may be a signal that is continuously applied. For example, the drag-and-drop 21 signal may be applied with a single touch, but is not limited thereto.

Further, for example, after receiving the editing start signal, the first button may be enlarged or reduced by receiving a drag signal in a predetermined direction with respect to the boundary line of the first button.

In addition, for example, after receiving the editing start signal, the corresponding UI may be deleted by clicking the UI once more.

In addition, for example, the UI may be added by receiving a UI addition start signal through a pressing 23 applied by the user to an empty space other than the UI for several seconds (eg, about 1 to 2 seconds, etc.).

In addition, for example, an existing UI or a newly added UI is based on an external input signal including multiple clicks on the UI or input of a predetermined pattern (eg, star, triangle, rectangle, etc.). You can reset the function of In other words, functions applicable to a corresponding UI per a predetermined pattern or click (eg, switching between a 2D image and a 3D image, resolution adjustment, etc.) may be changed and displayed on the corresponding UI.

In addition, according to another embodiment of the present invention, functions applicable to an existing UI or a newly added UI may be provided in the form of a preset table.

For example, such a table may be provided in a pop-up form, or may be provided in a form displayed in a blank space of the control panel 2500 or the like. When functions applicable to the UI are provided, functions for the existing UI or newly added UI may be reset according to the user's matching input (eg, sequential click, drag and drop, etc.), but is not limited thereto. .

The order of matching between the UI and the function is a method of first selecting the function to be applied according to the user input and then selecting the UI to which the selected function is applied, or selecting the UI to reset the function first, and then selecting the function to be applied to the selected UI. You can follow any of the methods.

Similarly, a language, font, color, etc. displayed on an existing UI or a newly added UI may be changed and displayed. In other words, by receiving an external input signal such as a signal for selecting a nationality of the user, it is possible to change characters, numbers, etc. displayed on the UI to suit the user's language.

In addition, the language displayed on the UI may be automatically changed according to the nationality included in the user profile including the user's identification information.

In addition, by receiving a signal for changing the UI display color to a color selected from a color table set in advance and available as an external input signal, the color displayed on the UI may be changed.

9B shows an example of using a modified user interface (UI) according to an embodiment of the present invention.

According to an embodiment of the present invention, when a user interface (UI) is changed based on motion information, when the user does not sufficiently recognize the change of the UI, the user may be confused or malfunction in using the changed UI. Accordingly, by indicating the user's UI use state with respect to the changed UI according to an embodiment of the present invention, the aforementioned confusion or malfunction of the user can be prevented.

Provision of the user's UI usage status may be performed in real time. In addition, the user's UI use state may be provided as at least one of an image including letters and numbers, and sounds such as voice.

For example, a UI used by the user to perform a predetermined function may be displayed to the user for a predetermined time through the display unit 2300 or the like. Through this display, the user can accurately recognize what UI he or she wants to use.

9B, if the user presses the second button (for example, a 2D to 3D switching function) for the changed UI 139 (91), the function of the second button for a predetermined time The information 93 may appear on the upper left of the display unit 2300 or the like. For example, when the user presses button 2 (91), the function information 93 of "2D→3D" may be displayed and disappeared on the display unit 2300 for about 1 second or 1.5 seconds, but the implementation form is not It is not necessarily limited thereto.

In addition, as shown in FIG. 9B, in response to the user's pressing 91 of the second button, the function of the second button (for example, a 2D to 3D switching function) is performed, and a 3D image 115 of the object Can be obtained.

In addition, the user's UI use state may be provided in the form of sound such as voice. As in the above-described example, when the user presses the second button, the user's UI use state may be provided to the user in the form of a voice message, such as "2D to 3D conversion".

10 illustrates an aspect in which a user interface (UI) is provided according to an embodiment of the present invention.

The UI according to an exemplary embodiment of the present invention may be displayed through at least one of a display unit 2300 on which a captured image of an object is displayed and a control panel 2500 equipped with a display function.

A need for a portable medical device is emerging, and due to the trend of miniaturization of medical devices, a screen for displaying an image of an object and a screen for providing a control panel may coexist on one touch screen. In other words, as illustrated in FIG. 10A, the image 113 of the object may be provided on the same screen (eg, the display unit 2300) as the UI pattern 131.

However, the use of such an integrated display may make it difficult to read a photographed image of an object due to foreign substances such as dust and a user's fingerprint.

According to an embodiment of the present invention, as shown in FIG. 10B, a UI that can be changed and provided may be provided through a control panel 2500 equipped with a display function other than a display unit of a captured image. For example, the captured image may be provided to the user through the display unit 2300, and the predetermined UI pattern 131 may be provided to the user separately from the captured image through the control panel 2500 equipped with a display function. have.

11 is a flowchart illustrating a method of controlling an operation of a medical device according to another embodiment of the present invention. 12 illustrates an example of operation control of a medical device according to another embodiment of the present invention.

The method according to an embodiment of the present invention may further include adjusting at least one of a height and an angle of the control panel according to motion information of the user (S400).

In order to maximize the convenience of the user's medical device, the physical position of a predetermined component (eg, the display unit 2300, the control panel 2500, etc.) of the medical device may be adjusted based on motion information reflecting the user's posture. .

For example, the position of a predetermined component (eg, the display unit 2300, the control panel 2500, etc.) of the medical device may be automatically adjusted depending on which hand the user holds the probe before photographing the object.

For example, when the user holds the probe with the right hand, there is a very high probability that the user's upper body changes to the right side when shooting starts. Accordingly, the display unit 2300, the control panel 2500, and the like may be moved up, down, left, right, horizontal or vertical, or at a predetermined angle toward the user according to motion information obtained from the user's movement. In other words, the display unit 2300, the control panel 2500, etc. may be moved up, down, left, right, horizontally or vertically, or at a predetermined angle so that the display unit 2300 and the control panel 2500 face the user (or are located in front of the user as much as possible). .

In addition, as described above, based on motion information that can be obtained by moving the user who is photographing the object, the display unit 2300, the control panel 2500, etc. may face the user (or be positioned in front of the user as much as possible). ) It can be moved up, down, left, right, horizontal or vertical, at a predetermined angle.

As illustrated in FIG. 12, at least one of the height and angle of the control panel 2500 may be adjusted according to the movement of the probe 1200. In other words, as the positions of certain components of the medical device (eg, the display unit 2300, the control panel 2500, etc.) may be moved, the user's convenience of operating the medical device may be further increased.

13 is a flowchart illustrating a method of managing a changed UI according to another embodiment of the present invention.

The method according to an embodiment of the present invention may further include the step of acquiring the user's identification information (S500), and the step of storing the UI changed according to the acquired identification information (S600), but are limited thereto. no. For example, the step of acquiring user identification information (S500) may be performed before the step (S100) of acquiring motion information of the user.

In addition, the step of acquiring user identification information (S500) may be performed simultaneously with the step (S100) of acquiring motion information of the user.

For example, as described above, the user's fingerprint information, iris information, facial information, and the like may be acquired, and a user profile including the user's biometric information may be acquired. In other words, the user identification information may be included in the user profile together with the user biometric information, and this user profile may be built in advance in the form of a database.

For example, a UI pattern that is preferred for each user or a UI changed according to an embodiment of the present invention is stored according to user identification information (eg, ID, etc.), and the user later You can also load and use a saved UI pattern.

In other words, since the physical characteristics and behavioral characteristics of the user may be different, the appropriate UI pattern or previously changed UI pattern, etc., according to the user identification information is databaseized, and the user's identification information when the user operates the medical device. By loading the UI pattern corresponding to the to set the shooting environment of the medical device, it is possible to shorten the shooting time.

14 illustrates an apparatus for changing and providing a user interface (UI) used when examining an object through a medical device according to an embodiment of the present invention based on motion information of a user.

According to an embodiment of the present invention, an apparatus 2000 for changing and providing a user interface (UI) used for examination of an object through a medical device based on motion information of a user, comprises: It may include a motion information acquisition unit 2100 to be acquired, a UI changer 2200 to change a UI using the acquired motion information of a user, and a display unit 2300 capable of displaying the changed UI.

The motion information acquisition unit 2100 may acquire motion information indicating whether the user uses the probe 1200 with the left hand or the probe 1200 with the right hand.

For example, a user who mainly uses his left hand places an object (eg, a patient) on his or her left hand and proceeds with image capture. In addition, a user who mainly uses his right hand will place an object on his right side and take an image. In other words, a left-handed person can hold the probe 1200 with their left hand and perform image capturing, and a right-handed person can hold the probe 1200 with their right hand and perform image capturing.

A user of a medical device, particularly an ultrasound diagnostic device, manipulates the control panel 2500 as well as the probe 1200 to obtain an image of an object. Therefore, in order to increase the convenience of use of the medical device user, a user interface (UI) or the like may be provided in different forms on the control panel according to the user's use of the probe 1200.

As described above, when the user is a right-handed user, it may be convenient for the user to position the object on his right side and manipulate the probe using the right hand, so that the button UI on the control panel, the trackball UI, etc. If provided as a pattern (131 in FIG. 2B), the user can feel the convenience of using the medical device.

In addition, similarly, if the user is left-handed, it may be convenient for the user to position the object on his left side and to operate the probe using the left hand. Can be provided as a left-handed UI pattern (133 in FIG. 2B).

The UI pattern 131 for the right hand and the UI pattern 133 for the left hand on the control panel may be switched to each other based on motion information of the user.

For example, when a user A and a user B exist as a user who uses a medical device, a user A who is a right-handed user wants to use the medical device, the right hand on the control panel 2500 according to an embodiment of the present invention. A UI pattern (131 in FIG. 2B) may be provided. In contrast, when a left-handed user B wants to use a medical device, a UI pattern for a left hand on the control panel 2500 according to an embodiment of the present invention (FIG. 133 of 2b may be provided.

In other words, according to an embodiment of the present invention, each user can flexibly use the medical device without being limited by his or her physical or behavioral characteristics, and by providing a UI of an appropriate pattern according to the user, There is an effect that user convenience can be increased.

Changing the user interface (UI), etc., based on the motion information obtained according to an embodiment of the present invention, adaptively adjusts the photographing environment of the medical device to the user so that the user can conveniently operate the medical device. May include. As described above, depending on whether the user is an excellent user or a left-handed user, the layout of the UI such as the control panel 2500 is different, the layout, size, etc. of the UI are different depending on the type of examination, or the frequency of use of buttons used for shooting. According to the UI layout, size, etc. can be adjusted and provided.

15 illustrates a device further including a sensing unit according to an embodiment of the present invention.

The motion information acquisition unit 2100 according to an embodiment of the present invention may further include a sensing unit 2110.

Motion information of a user according to an embodiment of the present invention may be obtained based on position information of a current probe obtained through the sensing unit 2110.

The location information of the current probe according to an embodiment of the present invention may include at least one of a current indication direction of the probe, an inclination angle with respect to a predetermined reference point, and a height value.

For example, the location information of the current probe 1200 may be obtained based on the current indication direction of the probe 1200. For example, the probe 1200 may be included in the probe 1200 for connection to the ultrasonic device (eg, included at the end of the probe) or through a predetermined sensor 121 embedded in the ultrasonic device to which the probe 1200 is to be connected. ) Of the current indication direction information may be obtained.

The predetermined sensor 121 may be configured to determine a direction in which the probe 1200 is directed from a reference line (eg, the center line ML of the sensor 121) according to which direction the connection line of the probe 1200 is directed. have. In other words, the predetermined sensor 121 according to an embodiment of the present invention may operate similarly to the operating principle of the toggle switch.

Referring again to FIG. 4A of the present application, if the probe 1200 is deflected to the right from the center line ML, it may be determined that the current position information of the probe 1200 indicates the right side of the ultrasound device. For example, information indicating that the probe 1200 is located on the right side of the ultrasound device may be obtained.

Also, if the probe 1200 is deflected to the left from the center line ML, it may be determined that the position information of the current probe 1200 indicates the left side of the ultrasound device. In other words, information that the probe 1200 is located on the left side of the ultrasound device may be obtained.

In addition, when the probe 1200 according to an embodiment of the present invention is wireless, the predetermined sensor 121 may be configured to sense the coordinates of the GPS (Global Positioning System) of the wireless probe through wireless communication.

In addition, according to an embodiment of the present invention, the location information of the current probe 1200 may include at least one of an inclination angle and a height value for a predetermined reference point.

As shown in FIG. 4B, the probe 1200 may include a sensor 123 for obtaining tilt information or height information. The sensor 123 may include a gyro sensor, a height sensor, or the like.

For example, the location information of the current probe 1200 may be obtained as information on an inclination angle for a predetermined reference point. The predetermined reference point may include a ground on which the medical device is located, an initial position of the patient table or probe 1200 (eg, a position attached to the ultrasound device, etc.).

The user's current motion may be estimated according to the inclination angle information. For example, it may be estimated that the degree of movement of the probe 1200 gradually increases as the width of change of the inclination angle increases, and the range of the movement angle of the user operating the probe 1200 based on the estimated movement of the probe 1200 Motion information on the back may be obtained.

For example, the ground (or patient table) and the probe 1200 may form an angle of up to 90 degrees (eg, when the probe 1200 is positioned perpendicular to the ground). The user may move the probe 1200 in a predetermined direction and angle as photographing proceeds. An angle between the probe 1200 and the ground may be included within a predetermined range (eg, 0 degrees to 90 degrees). For example, when a user wants to photograph liver ultrasound of an object, the probe 1200 may be parallel to the ground (for example, the probe 1200 and the ground are at an angle of 0 degrees) or vertical (eg, probe 1200) according to user manipulation. (1200) and the ground at an angle of 90 degrees) can be moved within a range.

In other words, when the probe 1200 is moved while the probe 1200 is parallel to the ground and the probe 1200 and the ground are perpendicular to each other, the width of the change in the inclination angle can be maximum, and the The motion degree may be estimated to be the maximum, and motion information indicating that the motion degree of the user operating the probe 1200 is also maximum may be obtained based on the estimated motion of the probe 1200.

In addition, similarly, an inclination angle from the position of the initial state in which the probe 1200 was attached to the ultrasound device may be obtained as position information of the current probe 1200.

For example, in the sensor 123 according to an embodiment of the present invention, the orientation toward the right or east side based on the sensor 123 is expressed as a positive angle, and the orientation toward the left or west side is a negative angle. It may be configured to be expressed, but is not necessarily limited thereto.

Accordingly, when the probe 1200 is present while being deflected to the right from the initial position, a positive inclination angle may be sensed on the sensor 123, and the current probe 1200 is the inclination angle sensed on the right side of the sensor 123 Information that it exists at the location of the province can be obtained. In addition, when the probe 1200 is currently deflected to the left from the initial position, a negative tilt angle may be sensed on the sensor 123, and the current probe 1200 is the tilt angle sensed on the left side of the sensor 123 Information that it exists at the location of may be obtained.

Also, the location information of the current probe 1200 may be obtained as height value information for a predetermined reference point. The predetermined reference point may include the ground, the patient's table, or the initial position of the probe 1200 (eg, a position attached to the ultrasound device, etc.). In this case, the predetermined sensor 123 may be a height sensor.

For example, the sensor 123 may sense information that the probe 1200 is currently positioned at a height of about 120 cm from the ground.

In addition, the sensor 123 may sense that the probe 1200 is currently positioned about 5 cm lower than the initial state attached to the ultrasonic device, for example. In other words, the height value information for the predetermined reference point may be obtained as position information of the current probe 1200.

Motion information indicating a change in height of a posture of a user operating the probe 1200 may be obtained based on height value information (eg, a change in height, etc.) of the probe 1200 according to an embodiment of the present invention. have.

User motion information according to an embodiment of the present invention may be obtained based on biometric information including at least one of user's fingerprint information, user's iris information, and user's face information acquired through the sensing unit 2110. have.

Referring again to FIG. 5 of the present application, based on the user's fingerprint information 11 acquired through the sensor 125 included in the probe 1200, the user currently selects the probe 1200 with either the left or the right hand. It can be determined whether or not it was gripped.

In other words, from the user's fingerprint information 11 acquired through the sensor 125 included in the probe 1200, motion information on whether the user uses the probe 1200 with the left hand or the right hand can be obtained. have.

For example, if the obtained user's fingerprint information 11 is a fingerprint for the thumb or index finger of the user's right hand, it is determined that the user is currently holding the probe 1200 with the right hand, and the use of the probe 1200 with the user's right hand. Motion information can be obtained.

In addition, if the obtained user's fingerprint information 11 is a fingerprint of the user's left thumb or index finger, it is determined that the user is currently holding the probe 1200 with his left hand, and the use of the probe 1200 with the user's left hand. Motion information can be obtained.

In addition, according to an embodiment of the present invention, based on the user's fingerprint information 11 acquired through the sensor 125 included in the probe 1200, it is possible to identify who is the user currently using the probe. . In this case, the user's fingerprint information 11 may be utilized as user identification information (eg, ID, etc.).

In addition, user motion information according to an embodiment of the present invention may be obtained based on at least one of the user's iris information 13 and the user's face information 15.

Referring again to FIG. 5 of the present application, at least one of the user's iris information 13 and the user's face information 15 may be obtained through a sensor 111 provided on an ultrasonic device. The sensor 111 may be located near the display unit 2300 of the ultrasonic device, but is not limited thereto.

In addition, the sensor 111 may perform iris recognition and face recognition functions at the same time, or may be implemented as individual sensors having an iris recognition function and a face recognition function.

According to an embodiment of the present invention, iris information 13 of a user may be obtained through the sensor 111. The user's iris information 13 may include user identification information on who the current user is, and information on the current location of the current user's iris. For example, by reading the user's iris through the sensor 111, information on who the current user is may be obtained.

In addition, a current gaze of the user may be determined using the current location information of the iris obtained through the sensor 111 and motion information of the user may be obtained based on the current gaze of the user. In other words, information on the current posture of the user may be obtained according to whether the iris is more deflected to the left or the right of the user's eye, but is not limited thereto.

For example, in the case where the iris is generally deflected to the left in the user's eye, it may be determined that the upper body of the user is oriented toward the right. In other words, it may be determined that the user is currently operating the probe 1200 or the like with the right hand.

Similarly, when the iris of the user's eye is positioned to be generally biased to the right, it may be determined that the user's upper body is pointing to the left. It may be determined that the user is currently operating the probe 1200 or the like with his left hand.

According to an embodiment of the present invention, the user's facial information 15 may be obtained through the sensor 111. The user's facial information 15 may include user identification information on who the current user is, and information on the current facial direction of the user. For example, by reading the user's facial feature points and facial contours through the sensor 111, information on who the current user is may be obtained.

In addition, the user's main face may be determined by using the user's face direction information acquired through the sensor 111, and motion information of the user may be obtained based on the user's main face. In other words, the current main face of the user may be determined based on the user's face area, and information on the current posture of the user may be obtained from the determined main face.

For example, by comparing the area of the user's right face and the left face, if the area of the left face is wider, it may be determined that the upper body of the user is oriented toward the right. In other words, it may be determined that the user is currently operating the probe 1200 or the like with the right hand.

Similarly, when the area of the user's right face and the left face are compared and the area of the right face is larger, it may be determined that the user's upper body is oriented toward the left. In other words, it may be determined that the user is currently operating the probe 1200 or the like with his left hand.

In addition, motion information of the user may be obtained using the user's iris information 13 and the user's facial information 15. For example, if the area of the user's right and left face is compared, but it is not possible to clearly distinguish which area is larger, the user's motion is additionally considered as in the above-described method. Information can also be obtained.

The UI according to an embodiment of the present invention may include at least one of a shortcut button, a switch, a keyboard, and a trackball indicating a function to be used when examining an object.

Referring back to FIG. 6 of the present application, a user interface (UI) for displaying operation functions in a medical device according to an embodiment of the present invention includes a keyboard 611, a button or switch 613, a trackball 615, and the like. Can include.

A user interface (UI) according to an embodiment of the present invention includes a keyboard 611 type UI capable of inputting letters, numbers, etc. in relation to photographing of an object, image enlargement/reduction, resolution adjustment, 2D image and A UI in the form of at least one button or switch 613 representing a predetermined function such as switching between 3D images, a UI in the form of a trackball 615, and the like may be included.

At least one of the aforementioned UI in the form of a keyboard 611, a UI in the form of at least one button or switch 613, and a UI in the form of a trackball 615 may be provided as a virtual UI layout.

In other words, the user interface (UI) according to an embodiment of the present invention does not always have to exist in a physically constant form, but in the form of letters, numbers, images, etc. on the display unit 2300 or the control panel 2500 It may exist as a virtual (virtual) to represent a predetermined function.

The UI changer 2200 according to an embodiment of the present invention may change at least one of a shape, a size, and a location of the UI according to the acquired motion information of the user.

Referring again to FIG. 7A of the present invention, the shape of the UI may be changed according to the acquired motion information of the user.

For example, when the movement range of the probe 1200 is very wide, that is, when the movement of the user increases, the user's touch to the user interface (UI) 611 to 615 may be inconvenient. In other words, the user's hand may not reach the user interface (UI), or the user may feel difficult to manipulate the UI.

In this case, as in the exemplary embodiment of the present invention, the user's convenience may be increased by changing the shape of the user interface (UI). For example, if the user uses button 2 more often than button 1 or button 3 during the photographing process, the shape of the lower side may be changed to a triangle shape with the longest bottom side to increase the probability of accessing the button 2 by the user. In other words, you can change the area of button 2 to be larger than button 1 or button 3.

In addition, according to an embodiment of the present invention, the positions of buttons 1 to 3 may be changed. For example, the UI can be changed to be arranged in the order of button 3, button 1, and button 2. In this case, button 1 may be the button with the highest frequency of use, and button 1 may form a triangle shape with the longest bottom side, but the shape of the UI before and after changes such as the button is not necessarily limited thereto.

Further, referring again to FIG. 7B of the present invention, the size of the UI may be changed according to the acquired motion information of the user.

As described above, when the user's movement increases, the size of the UI pattern 131 is entirely or partially increased or decreased, thereby changing the size of the UI so that the user can easily access the UI such as the button, etc. )can do.

For example, by increasing the size of buttons 1 to 3, it is possible to increase the user's accessibility of the buttons. Also, similarly, by changing the positions of buttons 1 to 3 to be close to the user, it is possible to increase the accessibility of the user's button.

Referring again to FIG. 7C of the present application, a different pattern of UI may be provided based on a user's physical characteristics.

For excellent users, a right-handed UI pattern 131 may be provided. In addition, for a left-handed user, a left-handed UI pattern 133 may be provided. The right-handed UI pattern 131 and the left-handed UI pattern 133 may be symmetrical to each other, and, as described above, may be interchangeable based on user motion information.

In the UI according to an embodiment of the present invention, at least one of a shape, a size, and a location may be provided differently according to a frequency of a function used when an object is examined.

Referring back to FIG. 8 of the present application, the UI pattern 131 is provided by changing at least one of shape, size, and position according to the frequency of functions used during examination of the object as shown in FIGS. 8B and 8C. (137 or 139). For convenience of explanation, it is assumed that the UI pattern 131 of FIG. 8A is a basic pattern that can be provided by a medical device.

For example, a trackball is not used in a predetermined examination item, and a function corresponding to the second button among the button-type UIs simplified to 1 to 3 may be used most. In this case, as shown in FIG. 8B, the circular UI of the trackball function is omitted from the basic pattern 131, and the size of the UI of the second button may be changed to be the largest.

In other words, by omitting a UI that is not used by a user or having a low frequency of use, and providing a UI with a high frequency of use in a large size, it is possible to increase the accessibility and convenience of a user who operates a medical device to the UI.

As another example, if a user mainly uses a trackball in a predetermined examination item and uses the third button (for example, an enlargement/reduction function, etc.) of a button-type UI most often, as shown in FIG. 8C, the basic pattern 131 In ), the size of button 3 among the circular UI and button-type UI corresponding to the trackball can be greatly changed and provided (139).

Also, similar to the shape of a user's hand, the UI may be arranged and provided in a radial shape like the pattern 139 in FIG. 8C, but the present invention is not limited thereto.

In addition, since the frequency of functions used may be different according to the type of inspection, different patterns of UI can be provided according to the type of inspection.

For example, the frequency of use of functions provided by the ultrasound apparatus may be different according to each type of examination, such as echocardiography, liver ultrasound, abdominal ultrasound, gynecological ultrasound, and Doppler ultrasound. For example, in echocardiography, an image enlargement/reduction function and a trackball function are frequently used, whereas in a liver ultrasound examination, an image enlargement/reduction function and a resolution control function may be used more than a trackball function.

In other words, in the echocardiography, a UI that performs an image enlargement/reduction function and a UI that performs a trackball function are relatively larger than those of other UIs, and may be positioned in the center of the control panel 2500.

In contrast, in the liver ultrasound examination, the size of the UI of the image enlargement/reduction function and the UI of the trackball function can be reduced or omitted, and the UI performing the resolution adjustment function in the center of the control panel 2500 is relatively large. You can change the UI pattern so that it is arranged in size.

16 illustrates a device further including a sensing unit, an external input receiving unit, and a control panel according to an embodiment of the present invention.

The device 2000 according to an embodiment of the present invention may further include an external input receiving unit 2400.

The UI according to an embodiment of the present invention may be edited by the UI changing unit 2200 according to an external input signal received through the external input receiving unit 2400.

The UI according to an embodiment of the present invention can be added, deleted, position, and size changed according to, for example, a user input. In addition, the UI may change language, font, and UI display color according to user input.

For example, the location or size of the button-type UI may be changed according to a user input.

Referring again to FIG. 9 of the present application, the location or size of the UI may be changed based on a user's predetermined input to the UI. A user's predetermined input according to an embodiment of the present invention may include at least one of clicking at least once on the UI and pressing for a predetermined time. In addition, such a predetermined input may include a case where a plurality of inputs are simultaneously input.

For example, as a user input for button 1 of a button-type UI, an editing start signal is applied by pressing for about 1 to 1.5 seconds, and drag and drop 21 signals are applied to button 1 to a target position. You can change the position of the No. 1 button.

The drag-and-drop 21 signal may be a continuous user input signal. In other words, for example, the user input signal of the drag-and-drop 21 for button 1 may be a signal that is continuously applied. For example, the drag-and-drop 21 signal may be applied with a single touch, but is not limited thereto.

Further, for example, after receiving the editing start signal, the first button may be enlarged or reduced by receiving a drag signal in a predetermined direction with respect to the boundary line of the first button.

In addition, for example, after receiving the editing start signal, the corresponding UI may be deleted by clicking the UI once more.

In addition, for example, the UI may be added by receiving a UI addition start signal through a pressing 23 applied by the user to an empty space other than the UI for several seconds (eg, about 1 to 2 seconds, etc.).

In addition, for example, an existing UI or a newly added UI is based on an external input signal including multiple clicks on the UI or input of a predetermined pattern (eg, star, triangle, rectangle, etc.). You can reset the function of In other words, functions applicable to the UI per predetermined pattern or click (eg, switching between a 2D image and a 3D image, resolution adjustment, etc.) may be changed and displayed on the corresponding UI.

In addition, according to another embodiment of the present invention, functions applicable to an existing UI or a newly added UI may be provided in the form of a preset table.

For example, such a table may be provided in a pop-up form, or may be provided in a form displayed in a blank space of the control panel 2500 or the like. When functions applicable to the UI are provided, functions for the existing UI or newly added UI may be reset according to the user's matching input (eg, sequential click, drag and drop, etc.), but is not limited thereto. . The order of matching between the UI and the function is a method of first selecting the function to be applied according to the user input, then selecting the UI to which the selected function is applied, or selecting the UI to reset the function first, and then selecting the function to be applied to the selected UI. You can follow any of the methods.

Similarly, a language, font, color, etc. displayed on an existing UI or a newly added UI may be changed and displayed. In other words, by receiving an external input signal such as a signal for selecting a nationality of the user, it is possible to change characters, numbers, etc. displayed on the UI to suit the user's language. In addition, the language displayed on the UI may be automatically changed according to the nationality included in the user profile including the user's identification information.

In addition, by receiving a signal for changing the UI display color to a color selected from a color table set in advance and available as an external input signal, the color displayed on the UI may be changed.

The apparatus 2000 according to an embodiment of the present invention may further include a control panel 2500 equipped with a display function.

The UI according to an embodiment of the present invention may be displayed through at least one of the display unit 2300 and the control panel 2500.

The UI according to an exemplary embodiment of the present invention may be displayed through at least one of a display unit 2300 on which a captured image of an object is displayed and a control panel 2500 equipped with a display function.

A need for a portable medical device is emerging, and due to the trend of miniaturization of medical devices, a screen for displaying an image of an object and a screen for providing a control panel may coexist on one touch screen. In other words, referring again to FIG. 10A of the present application, the image 113 of the object may be provided on the same screen (eg, the display unit 2300) as the UI pattern 131.

However, the use of such an integrated display may make it difficult to read a photographed image of an object due to foreign substances such as dust and a user's fingerprint.

Referring again to FIG. 10B of the present application, a UI that is modified and provided according to an embodiment of the present invention may be provided through a control panel 2500 equipped with a display function other than a display unit of a captured image. For example, the captured image may be provided to the user through the display unit 2300, and the predetermined UI pattern 131 may be provided to the user separately from the captured image through the control panel 2500 equipped with a display function. have.

17 illustrates a device further including a sensing unit, an external input receiving unit, a control panel, and a control unit according to an embodiment of the present invention.

The device 2000 according to an embodiment of the present invention may further include a control unit 2600. The controller 2600 according to an embodiment of the present invention may adjust at least one of a height and an angle of the control panel 2500 according to motion information of a user.

In order to maximize the convenience of the user's medical device, the physical position of a predetermined component (eg, the display unit 2300, the control panel 2500, etc.) of the medical device may be adjusted based on motion information reflecting the user's posture. .

For example, the position of a predetermined component (eg, the display unit 2300, the control panel 2500, etc.) of the medical device may be automatically adjusted depending on which hand the user holds the probe before photographing the object.

For example, when the user holds the probe with the right hand, there is a very high probability that the user's upper body changes to the right side when shooting starts. Accordingly, the display unit 2300, the control panel 2500, and the like may be moved up, down, left, right, horizontal or vertical, or at a predetermined angle toward the user according to motion information obtained from the user's movement. In other words, the display unit 2300, the control panel 2500, etc. may be moved up, down, left, right, horizontally or vertically, or at a predetermined angle so that the display unit 2300 and the control panel 2500 face the user (or are located in front of the user as much as possible). .

In addition, as described above, based on motion information that can be obtained by moving the user who is photographing the object, the display unit 2300, the control panel 2500, etc. may face the user (or be positioned in front of the user as much as possible). ) It can be moved up, down, left, right, horizontal or vertical, at a predetermined angle. Referring again to FIG. 12 of the present application, at least one of the height and angle of the control panel 2500 may be adjusted according to the movement of the probe 1200. In other words, as the positions of certain components of the medical device (eg, the display unit 2300, the control panel 2500, etc.) may be moved, the user's convenience of operating the medical device may be further increased.

18 illustrates an apparatus for providing a user interface (UI) changed based on motion information of a user according to another embodiment of the present invention.

The apparatus 2000 according to an embodiment of the present invention includes a motion information acquisition unit 2100, a UI change unit 2200, a display unit 2300, an external input receiving unit 2400, a control panel 2500, and a control unit ( 2600), and may further include an identification information acquisition unit 2700 for acquiring user identification information, and a storage unit 2800 for storing a UI changed according to the acquired identification information.

According to an embodiment of the present invention, the identification information acquisition unit 2700 may acquire user identification information. The obtained identification information may be stored in the storage unit together with the changed UI, but is not limited thereto.

According to an embodiment of the present invention, obtaining user identification information may be performed prior to obtaining motion information of the user.

In addition, the acquisition of user identification information may be performed simultaneously with motion information of the user. For example, as described above, the user's fingerprint information, iris information, facial information, and the like may be acquired, and a user profile including the user's biometric information may be acquired. In other words, the user identification information may be included in the user profile together with the user biometric information, and this user profile may be built in advance in the form of a database.

For example, a UI pattern that is preferred for each user or a UI changed according to an embodiment of the present invention is stored according to user identification information (eg, ID, etc.), and the user later You can also load and use a saved UI pattern.

In other words, since the physical characteristics and behavioral characteristics of the user may be different, the appropriate UI pattern or previously changed UI pattern, etc., according to the user identification information is databaseized, and the user's identification information when the user operates the medical device. By loading the UI pattern corresponding to the to set the shooting environment of the medical device, it is possible to shorten the shooting time.

In relation to the apparatus according to an embodiment of the present invention, the contents of the above-described method may be applied. Accordingly, description of the same contents as those of the above-described method in relation to the apparatus is omitted.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium.

Such computer-readable recording media include magnetic storage media (e.g., ROM, floppy disk, hard disk, etc.), optical reading media (e.g., CD-ROM, DVD, etc.), and carrier waves (e.g., Internet). It includes storage media such as (transfer).

So far, the present invention has been looked at around its preferred embodiments. Those of ordinary skill in the art to which the present invention pertains will be able to understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative point of view rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (10)

In the ultrasound diagnosis apparatus,
A control panel including a touch screen;
A memory for storing a UI pattern including at least one of a shape, a size, and a location for displaying a plurality of user interfaces (UIs) on the touch screen; And
At least one processor configured to execute computer-readable instructions;
Including,
The at least one processor,
Receive user input for selecting the type of test,
Change at least one of the shape, size, and location of the plurality of UIs based on the type of the test to provide a function corresponding to the type of test selected by the received user input, and the changed plurality of UIs Display on the touch screen,
Change the position of at least one UI among the plurality of UIs displayed on the touch screen, based on the drag-and-drop input received through the touch screen,
The plurality of UIs are operable to control functions of the ultrasound diagnosis apparatus,
The UI pattern includes information on at least one of a shape, size, and location of the plurality of UIs matched to each of a plurality of functions provided according to the type of inspection,
The at least one processor resets a first function matched with a first UI among the plurality of UIs to a second function based on a touch and drag-and-drop input input through the touch screen.
The method of claim 1,
The at least one processor may include setting information of the plurality of UIs displayed on the touch screen including the at least one UI whose location is changed, as a UI layout individually set by a user. Stored in the memory of the ultrasound diagnostic device.
The method of claim 1,
The at least one processor displays an ultrasound image on the touch screen.
The method of claim 1,
The plurality of UIs displayed on the touch screen includes a button-type UI.
The method of claim 4,
The button-type UI operates to freeze the displayed ultrasound image by a user.
The method of claim 1,
The plurality of UIs displayed on the touch screen include a keyboard type UI capable of receiving an input from a user to input at least one of letters and numbers.
The method of claim 1,
The plurality of UIs displayed on the touch screen includes a switch-type UI that enlarges or reduces an ultrasound image.
The method of claim 1,
The plurality of UIs displayed on the touch screen include a switch type UI for switching an ultrasound image between a 2D ultrasound image and a 3D ultrasound image.
The method of claim 1,
A display device that displays an ultrasound image;
The ultrasound diagnosis apparatus further comprising.
The method of claim 1,
Each of the plurality of UIs displayed on the touch screen is a graphic UI.

Images