JP2014032620A - Medical work support system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve efficiency of work relating to an operation panel of a medical appliance.SOLUTION: A medical work support system according to an embodiment includes a terminal device, a recognition unit, and a function display unit. The terminal device includes a camera for capturing an image of a periphery and a display on which the image captured by the camera is displayed. When an image of an operation panel of a medical appliance having a plurality of operation elements arranged thereon is captured, the recognition unit recognizes the operation elements in the image of the panel displayed on the display. The function display unit specifies functions assigned to the operation elements recognized by the recognition unit, in accordance with function assignment information showing assignment of functions to respective operation elements arranged on the operation panel and displays information indicative of the specified functions, on the display in association with areas of the operation elements included in the image of the operation panel.

Description

  Embodiments described herein relate generally to a medical work support system that supports various work related to an operation panel of a medical device.

  Conventionally, in the medical field, medical devices such as various modalities for capturing an internal image of a subject such as an ultrasonic diagnostic apparatus, an X-ray diagnostic apparatus, an X-ray CT (Computer Tomography) apparatus, and an MRI (Magnetic Resonance Imaging) apparatus. Equipment is being used. Many medical devices include an operation panel on which operation elements such as buttons, knobs, and switches are arranged, and operate according to user operations on the operation panel.

  The operation panel provided in the medical device is required to have high operability in order to enable rapid treatment / diagnosis. Also, regular maintenance is indispensable so that the operation panel does not malfunction.

  When a large number of operation elements are arranged on the operation panel of the medical device, it is difficult to grasp the correspondence between each operation element and the function executed by the medical device when each operation element is operated. Further, if many operation elements are arranged, the time required for maintenance becomes enormous. In addition to these, various problems have arisen in connection with the operation panel of the medical device, which contributes to a reduction in the efficiency of various operations related to the operation panel.

  The problem to be solved by the present invention is to improve the efficiency of work related to an operation panel of a medical device.

  A medical work support system according to an embodiment includes a terminal device, a recognition unit, and a function display unit. The terminal device includes a camera that captures the surroundings and a display that displays an image captured by the camera. The recognizing unit recognizes the operation element from an image of the panel displayed on the display when an operation panel of a medical device in which a plurality of operation elements is arranged is imaged. The function display unit specifies the function assigned to the operation element recognized by the recognition unit from the function assignment information indicating the assignment of the function to each of the operation elements arranged on the operation panel. Information indicating the function is displayed on the display in association with the region of the operation element included in the image of the operation panel.

1 is a configuration diagram of a medical work support system according to an embodiment. The block diagram which shows the principal part structure of the ultrasound diagnosing device which concerns on the same embodiment. The figure which shows the example of a data structure of the model information file which concerns on the embodiment. The figure which shows the data structure example of the function allocation file which concerns on the same embodiment. The figure which shows the data structure example of the operation frequency file which concerns on the same embodiment. The block diagram which shows the principal part structure of the server which concerns on the same embodiment. The block diagram which shows the principal part structure of the terminal device which concerns on the same embodiment. The flowchart of the process at the time of starting concerning the embodiment. 6 is a flowchart of shutdown processing according to the embodiment. The flowchart of the process at the time of the first access which concerns on the same embodiment. The flowchart of the process in the function display mode which concerns on the same embodiment. The figure for demonstrating the function display mode which concerns on the embodiment. The flowchart of the process in the help mode which concerns on the same embodiment. The figure for demonstrating the help mode which concerns on the embodiment. The flowchart of the process in the usage frequency mode which concerns on the embodiment. The figure for demonstrating the function display mode which concerns on the embodiment. 6 is a flowchart of processing in a default display mode according to the embodiment. The figure for demonstrating the default display mode which concerns on the same embodiment. The flowchart of the process in the customization mode which concerns on the same embodiment. The figure for demonstrating the customization mode which concerns on the embodiment. The flowchart of the process in the remote operation mode which concerns on the same embodiment. The figure for demonstrating the remote operation mode which concerns on the same embodiment.

  An embodiment will be described with reference to the drawings.

  In the present embodiment, a medical work support system that supports work related to an operation panel of an ultrasonic diagnostic apparatus that is a kind of medical equipment is illustrated. However, the medical work support system may support work related to an operation panel included in another medical device such as an X-ray CT apparatus, an MRI apparatus, or a radiotherapy apparatus.

(System configuration)
FIG. 1 is a configuration diagram of a medical work support system 1 according to the present embodiment.
The system 1 includes, for example, a plurality of ultrasonic diagnostic apparatuses US1 to US4 arranged in a hospital, a server 2 installed in or outside the hospital, and a terminal device 3 that is, for example, a smartphone or a tablet terminal. The ultrasonic diagnostic apparatuses US1 to US4, the server 2, and the terminal apparatus 3 are connected for communication via, for example, a LAN (Local Area Network) installed in a hospital or the Internet.

  An apparatus code “US001” is assigned to the ultrasonic diagnostic apparatus US1, an apparatus code “US002” is assigned to the ultrasonic diagnostic apparatus US2, and an apparatus code “US003” is assigned to the ultrasonic diagnostic apparatus US3. A device code “US004” is assigned to the diagnostic device US4. In the following description, when the ultrasonic diagnostic apparatuses US1 to US4 are not particularly distinguished, they are simply referred to as an ultrasonic diagnostic apparatus US. The number of ultrasonic diagnostic apparatuses US included in the medical work support system 1 is not limited to four.

  The housing surface of the ultrasonic diagnostic apparatus US is attached with a two-dimensional code C that encodes the MAC address of the own apparatus, a memory address in which information of the own apparatus (a database DB described later) is stored in the server 2, and the like. Yes. As the two-dimensional code C, for example, a QR code (registered trademark) can be adopted.

  FIG. 2 is a block diagram showing a main configuration of the ultrasonic diagnostic apparatus US. The ultrasonic diagnostic apparatus US includes a controller 10 that functions as a control center, a memory 11, a communication unit 12, an ultrasonic transmission / reception unit 13, an ultrasonic probe 14, a display 15, and an operation panel 16. The memory 11, the communication unit 12, the ultrasonic transmission / reception unit 13, the display 15 and the operation panel 16 are connected to the controller 10 via a bus line 17 constituted by an address bus and a data bus. The ultrasonic probe 14 is connected to the ultrasonic transmission / reception unit 13.

  The controller 10 includes a CPU (Central Processing Unit), a chip set, and the like. The memory 11 is composed of a read only memory (ROM), a random access memory (RAM), or an auxiliary storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The memory 11 stores a computer program or the like for causing the controller 10 to execute. The communication unit 12 communicates with the server 2 and the terminal device 3 via a network such as a LAN or the Internet.

  The ultrasonic transmission / reception unit 13 supplies a drive signal to the ultrasonic probe 14 to generate an ultrasonic wave, processes a reception signal output according to the reflected wave received by the ultrasonic probe 14, and generates a B-mode image, color An ultrasonic image such as a Doppler image or a pulse Doppler image is generated. The ultrasonic probe 14 includes a plurality of piezoelectric vibrators. Each piezoelectric vibrator generates an ultrasonic wave based on a drive signal supplied from the ultrasonic transmission / reception unit 13. Each piezoelectric vibrator outputs a reception signal to the ultrasonic transmission / reception unit 13 when receiving a reflected wave from the subject.

  The display 15 displays an ultrasonic image or the like generated by the ultrasonic transmission / reception unit 13.

  The operation panel 16 includes a plurality of buttons (operation elements) to which predetermined functions are assigned. By operating these buttons, patient information, ultrasonic scan conditions, an instruction to start ultrasonic image generation, an instruction to print an ultrasonic image, an instruction to transfer an ultrasonic image to another device, and the like are input. A button identifier that is not duplicated is assigned to each button arranged on the operation panel 16.

  The memory 11 according to the present embodiment includes a model information file F1 indicating model information of the ultrasonic diagnostic apparatus US, a function assignment file F2, and an operation count file F3 indicating the cumulative operation count of each button provided on the operation panel 16. Is remembered.

  An example of the data structure of the model information file F1 is shown in FIG. The model information file F1 includes the apparatus code and model code of the own apparatus (ultrasound diagnostic apparatus US), an operation panel configuration diagram, default function allocation information and the like. The model code is identification information assigned to each model of the ultrasonic diagnostic apparatus US. The ultrasonic diagnostic apparatus US having the same model code has the same layout of the operation panel 16. The operation panel configuration diagram is image data in which button patterns simulating the shape of buttons arranged on the operation panel 16 of the own apparatus are arranged according to the actual layout of each button (see FIGS. 20 and 22). . The button identifier described above is assigned to each button pattern included in the image data. The default function assignment information is information indicating the default function assigned to each button of the operation panel 16, and specifically, the default function of the button indicated by the identifier is described with respect to the above button identifier. Has an associated data structure.

  An example of the data structure of the function assignment file F2 is shown in FIG. The function allocation file F2 corresponds to the button identifiers (B001, B002, B003...) Of the buttons provided on the operation panel 16 of the own device, and the functions (F001, F002) to be executed by the controller 10 when each button is operated. , F003...).

  An example of the data structure of the operation frequency file F3 is shown in FIG. The operation frequency file F3 associates the number of times each button has been operated (cumulative operation frequency) with the identifier (B001, B002, B003...) Of each button provided on the operation panel 16 of the own device. It is a file.

  Each of the ultrasonic diagnostic apparatuses US1 to US4 has the configuration illustrated in FIG. 2, and the files F1 to F3 illustrated in FIGS. 3 to 5 are stored in the memory 11.

  FIG. 6 is a block diagram illustrating a main configuration of the server 2. The server 2 includes a controller 20 that functions as a control center, a memory 21, and a communication unit 22. The memory 21 and the communication unit 22 are connected to the controller 20 via a bus line 23 constituted by an address bus and a data bus.

  The controller 20 includes a CPU, a chip set, and the like. The memory 21 is configured by a ROM, a RAM, or an auxiliary storage device such as an HDD or an SSD. The memory 21 stores a computer program or the like for causing the controller 20 to execute. The communication unit 22 communicates with the ultrasonic diagnostic apparatus US and the terminal device 3 via a network such as a LAN or the Internet.

  The memory 21 according to the present embodiment corresponds to the database DB1 corresponding to the ultrasonic diagnostic apparatus US1, the database DB2 corresponding to the ultrasonic diagnostic apparatus US2, the database DB3 corresponding to the ultrasonic diagnostic apparatus US3, and the ultrasonic diagnostic apparatus US4. The database DB4 is stored. In the following description, when the databases DB1 to DB4 are not particularly distinguished, they are simply referred to as a database DB.

  The database DB includes a model information file F1, a function assignment file F2, and an operation frequency file F3 related to the corresponding ultrasonic diagnostic apparatus US. Hereinafter, the model information file F1, the function allocation file F2, and the operation frequency file F3 included in the databases DB1 to DB4 are referred to as a model information file F1 ′, a function allocation file F2 ′, and an operation frequency file F3 ′, respectively. The contents of the model information file F1 ′, the function assignment file F2 ′ and the operation frequency file F3 ′ included in each of the databases DB1 to DB4 are model information files F1 stored in the memory 11 of the corresponding ultrasonic diagnostic apparatuses US1 to US4, The contents of the function assignment file F2 and the operation frequency file F3 may be completely the same or different.

  FIG. 7 is a block diagram showing a main configuration of the terminal device 3. The terminal device 3 includes a controller 30 that functions as a control center, a memory 31, a communication unit 32, a camera unit 33, a display 34, a touch panel 35, a speaker unit 36, and a microphone unit 37. The memory 31, the communication unit 32, the camera unit 33, the display 34, the touch panel 35, the speaker unit 36, and the microphone unit 37 are connected to the controller 30 via a bus line 38 constituted by an address bus and a data bus.

  The controller 30 is composed of a CPU, a chip set, and the like. The memory 31 is configured by a ROM, a RAM, or the like. The memory 31 stores a computer program or the like for causing the controller 30 to execute. The communication unit 32 communicates with the ultrasonic diagnostic apparatus US and the server 2 via a network such as a LAN or the Internet. The camera unit 33 includes a lens that collects light from a direction opposite to the direction in which the display surface of the display 34 is directed, a CCD (Charge Coupled Device) element that converts the light collected by the lens into an electric signal, and the CCD. An AD converter that AD-converts an electrical signal output from the element to generate image data is provided. That is, the camera unit 33 captures a direction opposite to the direction in which the display surface of the display 34 is directed, and generates image data. The display 34 displays image data and the like generated by the camera unit 33. The touch panel 35 detects a touch operation with a user's finger or stylus on the display surface of the display 34. The speaker unit 36 generates sound. The microphone unit 37 collects sound and generates a sound signal. For example, when the terminal device 3 is a smartphone, the speaker unit 36 and the microphone unit 37 may be used for the purpose of calling with other devices.

  The memory 31 according to the present embodiment stores a help information file F4 provided for each model code (T001, T002, T003...) Of the ultrasonic diagnostic apparatus US. Each help information file F4 includes help information for explaining functions assigned to buttons of the operation panel 16 of the ultrasonic diagnostic apparatus US of the corresponding model. Each help information file F4 is downloaded from the server 2 or the like via the communication unit 32, for example, and stored in the memory 31.

  The controller 30 according to the present embodiment executes the computer program stored in the memory 31 to thereby detect the detection unit 100, the recognition unit 101, the function display unit 102, the help information display unit 103, the operation frequency display unit 104, the default display. Functions as the unit 105, the editing unit 106, and the remote operation unit 107 are realized. Details of these parts will be made clear in the following description.

(Operation of ultrasonic diagnostic equipment)
The operation of the ultrasonic diagnostic apparatus US will be described. Here, the characteristic operation of the ultrasonic diagnostic apparatus US in the present embodiment will be mainly described, and the description of the general operation at the time of performing the ultrasonic diagnosis will be omitted.

  When the ultrasonic diagnostic apparatus US is activated, the controller 10 executes the activation process shown in the flowchart of FIG. In this startup process, first, the controller 10 starts up the ultrasonic diagnostic apparatus US by performing processing such as diagnosis and initialization of each device, startup of the operating system, establishment of communication with the server 2, and the like (step S1). S101).

  After step S101, the controller 10 accesses the server 2 via the communication unit 12, and acquires the function assignment file F2 ′ from the database DB corresponding to the own device (step S102). The controller 10 temporarily stores the acquired function assignment file F2 ′ in the memory 11.

  After step S102, the controller 10 compares the function assignment file F2 stored in the memory 11 from the beginning of the start-up process with the function assignment file F2 ′ temporarily stored in the memory 11, and both files F2, F2 are compared. It is determined whether or not there is a difference between the contents of '(step S103).

  If there is a difference (Yes in Step S103), the controller 10 updates (overwrites) the function assignment file F2 stored in the memory 11 with the contents of the function assignment file F2 ′ (Step S104). The start-up process is completed through step S104. On the other hand, if there is no difference (No in step S103), the startup process is terminated without passing through step S104.

  After such startup processing, the operation of the ultrasound diagnostic apparatus US shifts to normal processing. In this normal processing, when each button on the operation panel 16 is operated, the controller 10 specifies a function associated with the identifier of the operated button from the function assignment file F2 stored in the memory 11, Perform the specified function.

  Further, each time a certain function is executed, the controller 10 identifies the identifier of the button to which the function is assigned from the function assignment file F2, and increments the cumulative operation count associated with the specified identifier in the operation count file F3. Increment.

  Subsequently, a shutdown process of the ultrasonic diagnostic apparatus US will be described. When the shutdown of the ultrasonic diagnostic apparatus US is instructed by the user operating the operation panel 16 or the like, the controller 10 executes a shutdown process shown in the flowchart of FIG.

  In the shutdown process, first, the controller 10 transmits the operation frequency file F3 stored in the memory 11 to the server 2 through the communication unit 12 together with the identification code of the own device (step S201). Thereafter, the controller 10 executes processing such as shutting down the operating system or stopping power supply to each unit in the own device (step S202). The shutdown process is completed by step S202.

  Even after the shutdown process, the contents of the model information file F1, the function assignment file F2, and the operation frequency file F3 stored in the memory 11 of the ultrasonic diagnostic apparatus US are retained.

  When the communication unit 22 of the server 2 receives the operation frequency file F3 and the identification code transmitted in step S201, the controller 20 of the server 2 performs an operation included in the database DB of the ultrasonic diagnostic apparatus US corresponding to the identification code. The frequency file F3 ′ is updated (overwritten) with the operation frequency file F3.

(Operation of terminal equipment)
Next, the operation of the terminal device 3 will be described.
The terminal device 3 is preinstalled with an application for supporting work related to the operation panel 16 of the ultrasonic diagnostic apparatus US (hereinafter referred to as a medical work support application). When the user instructs the execution of the medical work support application by operating the touch panel 35 or the like, the controller 30 of the terminal device 3 executes the first access process shown in the flowchart of FIG.

  In the initial access process, the controller 30 first detects access destination information (step S301). The access destination information includes, for example, the MAC address of the ultrasonic diagnostic apparatus US including the operation panel 16 to be worked, and the memory address of the database DB corresponding to the ultrasonic diagnostic apparatus US. In particular, in this embodiment, the controller 30 decodes the two-dimensional code based on the image data obtained by imaging the two-dimensional code attached to the ultrasonic diagnostic apparatus US by the camera unit 33, thereby accessing the access destination information. Is detected. However, the method for acquiring the access destination information is not limited to this. For example, the access destination information may be input by operating the GUI displayed on the display 34 via the touch panel 35.

  After step S301, the controller 30 determines whether or not the ultrasonic diagnostic apparatus US indicated by the detected MAC address is activated (power ON) (step S302). In this process, for example, the controller 30 attempts to establish a communication link with the ultrasonic diagnostic apparatus US indicated by the detected MAC address, and the ultrasonic diagnostic apparatus US is activated when the communication link is established. If the communication link cannot be established, it is determined that the ultrasonic diagnostic apparatus is not activated.

  When it is determined that the ultrasonic diagnostic apparatus US is activated (Yes in step S302), the controller 30 accesses the ultrasonic diagnostic apparatus US and stores the model information file F1 and the function assignment file F2 stored in the memory 11. Is acquired (step S303). The controller 30 stores the acquired model information file F1 and function assignment file F2 in the memory 31.

On the other hand, if it is determined that the ultrasonic diagnostic apparatus US is not activated (no in step S302), the controller 30 accesses the server 2 and includes the model included in the database DB corresponding to the memory address detected in step S301. The information file F1 ′ and the function assignment file F2 ′ are acquired (step S304). The controller 30 stores the acquired model information file F1 ′ and function assignment file F2 ′ in the memory 31.
After step S303 or step S304, the initial access process is completed. In particular, step S301 in the initial access process is the operation as the detection unit 100.

  After the initial access process, the controller 30 operates according to various operation modes provided by the medical work support application. Particularly in the present embodiment, a function display mode, a help mode, a use frequency mode, a default display mode, a customization mode, and a remote operation mode are prepared as the operation modes. Details of these modes will be described below.

[Function display mode]
When the process at the time of the first access is completed, the operation mode of the medical work support application is switched to the function display mode. The function display mode is a mode for displaying the function assigned to each button of the operation panel 16 on the image of the operation panel 16 displayed on the display 34 when the operation panel 16 is imaged by the camera unit 33.

  FIG. 11 is a flowchart of processing executed by the controller 30 in the function display mode. FIG. 12 is a schematic diagram for explaining the function display mode, and shows the operation support target operation panel 16 and the terminal device 3 that is faced with the lens of the camera unit 33 facing the operation panel 16 side. ing.

  In the function display mode, first, the controller 30 tries to recognize the button from the image of the operation panel 16 displayed on the display 34 (step S401). For this recognition, for example, a well-known image analysis method such as extracting an edge (a portion with a large change in pixel value) from the image displayed on the display 34 and specifying the shape of the button can be used. When a plurality of buttons are included in the image displayed on the display 34, the controller 30 recognizes all these buttons.

  After step S401, the controller 30 specifies the function assigned to each recognized button (step S402). Specifically, the controller 30 recognizes the shape of each button recognized in step S401 and the positional relationship between the buttons, and the operation included in the model information file F1 or F1 ′ saved in the memory 31 during the first access process. By comparing the shape of each button pattern arranged in the panel configuration diagram and the positional relationship between each pattern, it can be determined which button pattern corresponding to each recognized button corresponds to each button pattern arranged in the operation panel configuration diagram. Identify. As described above, since the button identifier is assigned to each button arranged in the operation panel configuration diagram, the correspondence between each button recognized in step S401 and each button pattern arranged in the operation panel configuration diagram is specified. If possible, the button identifier of each recognized button is determined. The controller 30 identifies the function assigned to the button identifier thus found from the function assignment file F2 or F2 ′ saved in the memory 31 in the initial access process.

  After step S402, the controller 30 attaches a marker to the button area recognized in step S401 in the image displayed on the display 34, and associates the function of each button identified in step S402 with each button area. Are displayed (step S403). In the example of FIG. 12, a marker M surrounding the shape of each button displayed on the display 34 is attached. In addition, a character string (“capture”, “server transfer”, “color print”, “monochrome print”, “M mode”...) Indicating the function assigned to each button is displayed immediately below each button. The function assigned to each button is displayed in association with each button. However, these character strings may be arranged directly above each button or in the marker M.

  After step S403, the controller 30 determines whether the end of the function display mode is instructed (step S404). As shown in FIG. 12, the display screen of the display 34 includes a help key 200 for designating a help mode, a usage frequency key 201 for designating a usage frequency mode, and a default display for designating a default display mode. A key 202, a customization key 203 for designating a customization mode, and a remote operation key 204 for designating a remote operation mode are included. The controller 30 determines that the end of the function display mode is instructed when the customize key 203 or the remote operation key 204 among these keys is operated, or when the end of the medical work support application is instructed.

  When it is determined that the end of the function display mode is not instructed (no in step S404), the operation of the controller 30 returns to step S401. That is, the controller 30 repeats steps S401 to S404 until the end of the function display mode is instructed. If the user moves the terminal device 3 while holding it on the operation panel 16 during this time, the button displayed on the display screen of the display 34 is recognized at each position, and the marker M and the function are displayed in the area of the recognized button. The

  Eventually, when the controller 30 determines that the end of the function display mode has been instructed (step S404: yes), the processing shown in the flowchart ends.

  Of the processes shown in the flowchart, step S401 is the operation as the recognition unit 101, and steps S402 and S403 are the operation as the function display unit 102.

[Help mode]
When the user operates the help key 200 shown in FIG. 12, the operation mode of the medical work support application is switched to the help mode. The help mode is a mode in which the description of the function assigned to the button to which the marker M is attached is displayed on the display 34 in the function display mode. The help mode is realized by the controller 30 operating as the help information display unit 103.

  FIG. 13 is a flowchart of processing executed by the controller 30 in the help mode. The controller 30 executes the process according to the flowchart in parallel with the process according to the function display mode. FIG. 14 is a schematic diagram for explaining the help mode, and shows the operation panel 16 that is a work target and the terminal device 3 that is faced with the lens of the camera unit 33 facing the operation panel 16 side. .

  In the help mode, first, the controller 30 determines whether or not the button area (area surrounded by the marker M) included in the image of the operation panel 16 displayed on the display 34 has been operated (step S501).

  If it is determined that one of the button areas has been operated (step S501: yes), the controller 30 displays help information describing the function of the button corresponding to the operated area on the display 34 (step S501). S502). Specifically, the controller 30 first acquires the model code of the ultrasonic diagnostic apparatus US with reference to the model information file F1 or F1 ′ stored in the memory 31 in the first access process, and uses the acquired model code as the acquired model code. The corresponding help information file F4 is specified from the memory 31. Then, the controller 30 reads the help information explaining the function of the button corresponding to the operated area from the identified help information file F4, and puts the read help information into the area of the button operated by the user's finger or the like. Display in association. FIG. 14 shows an example in which the button area to which the “monochrome print” function is assigned is operated and “print selected image in black and white” is displayed as the help information.

  After step S502, the controller 30 determines whether the end of the help mode is instructed (step S503). When the help mode is executed, as shown in FIG. 14, the help key 200 becomes in-execution notation (shown with diagonal lines), and a mode (MODE) key 205 is newly arranged. The controller 30 determines that the end of the help mode is instructed when the mode key 205 is operated, or when the end of the medical work support application is instructed.

  When it is determined that the end of the help mode is not instructed (no in step S503), the operation of the controller 30 returns to step S501. That is, the controller 30 repeats steps S501 to S503 until the end of the help mode is instructed. During this time, when the user operates an area of another button displayed on the display 34, help information for the button is displayed.

  Eventually, when the controller 30 determines that the end of the help mode has been instructed (Yes in step S503), the processing shown in the flowchart ends. When the help mode is ended because the mode key 205 is operated, the display screen of the display 34 returns to the state shown in FIG.

[Usage frequency mode]
When the user operates the usage frequency key 201 shown in FIG. 12, the operation mode of the medical work support application is switched to the usage frequency mode. The usage frequency mode is a mode for displaying the cumulative number of operations of each button on the operation panel 16 on the display 34. The usage frequency mode is realized by the controller 30 operating as the operation frequency display unit 104.

  FIG. 15 is a flowchart of processing executed by the controller 30 in the usage frequency mode. The controller 30 executes the process according to the flowchart in parallel with the process according to the function display mode. FIG. 16 is a schematic diagram for explaining the function display mode, and shows the operation support target operation panel 16 and the terminal device 3 that is faced with the lens of the camera unit 33 facing the operation panel 16 side. ing.

  In the usage frequency mode, the controller 30 first determines whether or not the ultrasonic diagnostic apparatus US indicated by the MAC address detected in step S301 is activated (power ON) (step S601). This determination method is the same as that in step S302.

  When it is determined that the ultrasonic diagnostic apparatus US is activated (Yes in step S601), the controller 30 accesses the ultrasonic diagnostic apparatus US via the communication unit 32 and stores the operation count file stored in the memory 11. F3 is acquired (step S602). The controller 30 stores the acquired operation frequency file F3 in the memory 31.

  On the other hand, if it is determined that the ultrasonic diagnostic apparatus US is not activated (no in step S601), the controller 30 accesses the server 2 via the communication unit 32 and corresponds to the memory address detected in step S301. The operation frequency file F3 ′ included in the database DB is acquired (step S603). The controller 30 stores the acquired operation frequency file F3 ′ in the memory 31.

  After step S602 or S603, the controller 30 specifies the cumulative number of button operations recognized from the image displayed on the display 34 (step S604). Specifically, the controller 30 searches the operation frequency file F3 or F3 ′ stored in the memory 31 with the button identifier of each button recognized in step S401 of the function display mode being executed in parallel. Specify the cumulative number of operations for these buttons.

  After step S604, the controller 30 compares the cumulative number of operations of each identified button with a predetermined threshold value (step S605). This threshold value represents the recommended replacement time of the button, and may be set to about 100,000 times, for example. This threshold value may be determined for each type of button.

  After step S605, the controller 30 determines whether or not there is a button having the number of operations> threshold (step S606). If there is a corresponding button (yes in step S606), the controller 30 highlights the area of the button (area surrounded by the marker M), and informs these buttons that the replacement time has arrived. Is displayed (step S607). FIG. 16 shows an example in which the cumulative number of operations of buttons to which the functions of “capture” and “monochrome print” are assigned exceeds a threshold value (100,000 times). In this example, these button areas are shaded to express highlighting. Specifically, the cumulative number of operations can be increased by changing the coloring or pattern in the marker M or the coloring of the marker M itself. What is necessary is just to highlight the area | region of the button which exceeded the threshold value. FIG. 16 shows an example in which a message “100,000 replacement times” is displayed in association with the button areas to which the functions of “capture” and “monochrome print” are assigned.

  After step S607, the controller 30 determines whether or not the button area included in the image of the operation panel 16 displayed on the display 34 has been operated (step S608). If it is determined that one of the button areas has been operated (Yes in step S608), the controller 30 displays the cumulative number of operations specified in step S504 for the button corresponding to the operated area on the display 34 ( Step S609). In FIG. 16, the button area to which the “M mode” function is assigned is operated by the user's finger, and the cumulative operation count “5,823 times” of the button is displayed in association with the button area. Show.

  After step S609 or when it is determined in step S608 that no button area has been operated (no in step S608), the controller 30 determines whether or not the end of the use frequency mode has been instructed ( Step S610). When the usage frequency mode is executed, the usage frequency key 201 becomes in-execution notation (shown by hatching) as shown in FIG. 16, and the mode key 205 is arranged. The controller 30 determines that the end of the use frequency mode is instructed when the mode key 205 is operated or when the end of the medical work support application is instructed.

  When it is determined that the end of the usage frequency mode is not instructed (no in step S610), the operation of the controller 30 returns to step S604. That is, the controller 30 repeats steps S604 to S610 until the end of the usage frequency mode is instructed. During this time, when the user moves the terminal device 3 while holding it on the operation panel 16, the buttons displayed on the display screen of the display 34 are recognized at each position, and the cumulative use count of the recognized buttons is compared with the threshold value. When the cumulative use count> threshold, the button area is highlighted and a message is displayed informing that the replacement time has arrived in association with the button area. Further, when the user operates an area of another button displayed on the display 34, the cumulative number of times that the button is used is displayed.

  Eventually, when the controller 30 determines that the end of the usage frequency mode has been instructed (Yes in step S610), the processing shown in the flowchart ends. When the usage frequency mode is ended by operating the mode key 205, the display screen of the display 34 returns to the state shown in FIG.

[Default display mode]
When the user operates the default display key 202 shown in FIG. 12, the operation mode of the medical work support application is switched to the default display mode. The default display mode is a mode for displaying the default function when the function assigned to the button recognized in the function display mode is different from the default function assigned to the button. The default display mode is realized by the controller 30 operating as the default display unit 105.

  FIG. 17 is a flowchart of processing executed by the controller 30 in the default display mode. The controller 30 executes the process according to the flowchart in parallel with the process according to the function display mode. FIG. 18 is a schematic diagram for explaining the default display mode, showing the operation panel 16 to be worked on and the terminal device 3 that is faced with the lens of the camera unit 33 facing the operation panel 16 side. Yes.

  In the default display mode, first, the controller 30 specifies a default function for each button recognized in step S401 in the function display mode from the image displayed on the display 34 (step S701). The default function can be specified by searching the default function assignment information of the model information file F1 or F1 ′ stored in the memory 31 at the time of the first access process based on the button identifier of each button.

  After step S701, the controller 30 identifies the current function identified in step S402 and the current function identified in step S701 for each button recognized in step S401 of the function display mode from the image displayed on the display 34. The default function is compared (step S702). After this comparison, the controller 30 determines whether or not there is a button with different current function and default function (step S703).

  If it is determined that there is a button with a different function from the default function (Yes in step S703), the controller 30 highlights the area of the button (area surrounded by the marker M) and The default function is displayed in association with the button area (step S704). In FIG. 18, since the default function of the button to which the “capture” function is assigned is “quick scan”, the button is highlighted, and the character string “quick scan” is directly above the button. An example in which is displayed. In this example, highlighting is expressed by adding a diagonal line to a button area to which the “capture” function is assigned. Specifically, the coloring or pattern in the marker M or the coloring of the marker M itself is changed. For example, it is sufficient to highlight a button area having a different default function from the current function.

  After step S704 or when it is determined in step S703 that there is no button whose current function is different from the default function (no in step S703), the controller 30 determines whether or not the end of the default display mode has been instructed Determination is made (step S705). When the default display mode is executed, as shown in FIG. 18, the default display key 202 is in execution (shown with diagonal lines), and the mode key 205 is arranged. The controller 30 determines that the end of the default display mode is instructed when the mode key 205 is operated or when the end of the medical work support application is instructed.

  When it is determined that the end of the default display mode is not instructed (no in step S705), the operation of the controller 30 returns to step S701. That is, the controller 30 repeats steps S701 to S705 until the end of the default display mode is instructed. During this time, when the user moves the terminal device 3 while holding it on the operation panel 16, the buttons displayed on the display screen of the display 34 are recognized at each position, and the current function assigned to the recognized button and the default are assigned. When the functions are compared and the two are different, the button area is highlighted and the default function is displayed in association with the button area.

  Eventually, when the controller 30 determines that the end of the default display mode has been instructed (step S705: yes), the processing shown in the flowchart ends. When the default display mode is ended because the mode key 205 is operated, the display screen of the display 34 returns to the state shown in FIG.

[Customize mode]
When the user operates the customization key 203 shown in FIG. 12, the operation mode of the medical work support application is switched to the customization mode. The customization mode is a mode for customizing (editing) the function assigned to each button of the operation panel 16. The customization mode is realized by the controller 30 operating as the editing unit 106.

  FIG. 19 is a flowchart of processing executed by the controller 30 in the customization mode. Note that the controller 30 ends the processing related to the function display mode as described above when the customize key 203 is operated.

  In the customization mode, first, the controller 30 displays the operation panel configuration diagram included in the model information file F1 or F1 ′ saved in the memory 31 in the initial access process on the display 34 (step S801).

  FIG. 20 is a schematic diagram illustrating an example of the terminal device 3 in which the operation panel configuration diagram is displayed on the display 34. The operation panel configuration diagram is image data in which button patterns simulating buttons arranged on the operation panel 16 as described above are arranged according to the actual layout of each button. At the time of executing the customization mode, the customization key 203 becomes in-execution notation (represented by hatching), and the mode key 205 is arranged. Further below that, a list 210 of functions to be assigned to each button, a scroll bar 211 for scrolling the list 210, and an enter key 212 are arranged. The functions arranged in the list 210 are all the functions described in the default function assignment information of the model information file F1 or F1 ′ saved in the memory 31 in the first access process, for example.

  After step S801, the controller 30 assigns a function included in the list 210 to each button in accordance with an operation on the display 34 (step S802). For example, the controller 30 assigns the selected function to a button corresponding to the selected button pattern by the user selecting any one of the functions included in the list 210 and the button pattern. Further, when the user touches an area of any function included in the list 210 and drags and drops it to any button pattern as it is, the function is assigned to the button corresponding to the button pattern. This makes it easier to assign functions.

  Note that “assignment” in step S802 is the function assigned to the button identifier of the assignment destination button in the function assignment file F2 or F2 ′ saved in the memory 31 at the time of the first access processing. It means to rewrite with.

  While accepting such assignment work, the controller 30 waits for an operation of the enter key 212 (no in step S803). When the enter key 212 is eventually operated (yes in step S803), the controller 30 determines whether or not the ultrasonic diagnostic apparatus US indicated by the MAC address detected in step S301 is activated (powered on). (Step S804). This determination method is the same as that in step S302.

  If it is determined that the ultrasonic diagnostic apparatus US is activated (yes in step S804), the controller 30 uses the function assignment file F2 or F2 ′ stored in the memory 31 to perform the ultrasonic diagnosis via the communication unit 32. The data is transmitted to the device US (step S805). When the function assignment file F2 or F2 ′ is received, the controller 10 of the ultrasonic diagnostic apparatus US updates the function assignment file F2 stored in the memory 11 with the received function assignment file F2 or F2 ′ ( Overwrite.

  On the other hand, if it is determined that the ultrasonic diagnostic apparatus US has not been activated (no in step S804), the controller 30 stores the function assignment file F2 or F2 ′ stored in the memory 31 and the identification code of the ultrasonic diagnostic apparatus US. Are transmitted to the server 2 via the communication unit 32 (step S806). When the function assignment file F2 or F2 ′ and the identification code are received, the controller 20 of the server 2 receives the function assignment file F2 ′ of the database DB corresponding to the identification code stored in the memory 21 and the received function assignment file F2 ′. Update (overwrite) in file F2 or F2 '.

  When the ultrasound diagnostic apparatus US executes the above-described shutdown process after step S805 is performed, the function assignment file F2 ′ included in the database DB of the ultrasound diagnostic apparatus US stored in the memory 21 of the server 2 is stored. The function assignment file F2 after the update is updated. When the ultrasonic diagnostic apparatus US executes the above-described startup process after step S806 is executed, the function assignment file F2 stored in the memory 11 of the ultrasonic diagnostic apparatus US is updated with the updated function assignment. It is updated in the file F2 ′. In this way, regardless of which of steps S805 and S806 is executed, both the function assignment file F2 stored in the ultrasound diagnostic apparatus US and the function assignment file F2 ′ stored in the server 2 are the contents after customization. Updated.

  The processing shown in the flowchart is completed through step S805 or S806. Thereafter, the display screen of the display 34 returns to the state shown in FIG. 12, and the controller 30 resumes the processing related to the function display mode.

[Remote operation mode]
When the user operates the remote operation key 204 shown in FIG. 12, the operation mode of the medical work support application is switched to the remote operation mode. The remote operation mode is a mode in which the ultrasonic diagnostic apparatus US is remotely operated from the terminal device 3 without using the operation panel 16. The remote operation mode is realized by the controller 30 operating as the remote operation unit 107.

  FIG. 21 is a flowchart of processing executed by the controller 30 in the remote operation mode. Note that the controller 30 ends the processing related to the function display mode as described above when the remote operation key 204 is operated.

  In the remote operation mode, the controller 30 first determines whether or not the ultrasonic diagnostic apparatus US indicated by the MAC address detected in step S301 is activated (power ON) (step S901). This determination method is the same as that in step S302.

  When it is determined that the ultrasonic diagnostic apparatus US is activated (Yes in step S901), the controller 30 is a configuration diagram of the operation panel included in the model information file F1 or F1 ′ stored in the memory 31 in the first access process. Is displayed on the display 34 (step S902).

  FIG. 22 is a schematic diagram illustrating an example of the terminal device 3 in which the operation panel configuration diagram is displayed on the display 34 in step S902. When the remote operation mode is executed, the remote operation key 204 is displayed as being executed (shown with diagonal lines), and the mode key 205 is arranged.

  After step S902, the controller 30 determines whether or not the button pattern in the operation panel configuration diagram has been operated by the user (step S903). If it is determined that any one of the button patterns has been operated (step S903: yes), the controller 30 specifies the function assigned to the button corresponding to the button pattern (step S904). Specifically, the controller 30 is assigned to the button corresponding to the button pattern by searching the function assignment file F2 or F2 ′ stored in the memory 31 with the button identifier assigned to the operated button pattern. Identify the function.

  After step S904, the controller 30 transmits the specified function execution instruction to the ultrasonic diagnostic apparatus US indicated by the MAC address detected in step S301 (step S905). The controller 10 of the ultrasonic diagnostic apparatus US that has received the execution instruction executes a function related to the instruction.

  If it is determined that the button pattern is not operated after step S905 or in step S903 (no in step S903), the controller 30 determines whether or not the end of the remote operation mode is instructed (step S906). The controller 30 determines that the end of the default display mode is instructed when the mode key 205 is operated, or when the end of the medical work support application is instructed.

  When it is determined that the end of the remote operation mode is not instructed (no in step S906), the operation of the controller 30 returns to step S903. That is, the controller 30 repeats steps S903 to S906 until the end of the remote operation mode is instructed. When the user operates any button pattern during this period, an instruction to execute the function assigned to the button corresponding to the operated button pattern is transmitted to the ultrasound diagnostic apparatus US, and the function is performed by the ultrasound diagnostic apparatus US. Is executed.

  Eventually, when the controller 30 determines that the end of the remote operation mode has been instructed (step S906: yes), the processing shown in the flowchart ends. When the remote operation mode is terminated due to the operation of the mode key 205, the display screen of the display 34 returns to the state shown in FIG. 12, and the controller 30 resumes the processing related to the function display mode.

  If the controller 30 determines in step S901 that the ultrasonic diagnostic apparatus US has not been activated (no in step S901), the ultrasonic diagnostic apparatus US cannot be remotely operated. Therefore, the process shown in the flowchart is completed without going through steps S902 to S906. Also in this case, the display screen of the display 34 returns to the state shown in FIG. 12, and the controller 30 resumes the processing related to the function display mode.

The operation of the medical work support system described above will be described.
By using the function display mode, the user can easily grasp the functions of the buttons arranged on the operation panel 16. In particular, when the function of the button is customized, the correspondence between the button and the function tends to be unclear, and conventionally, a sticker that describes the function is pasted on the customized button. When the seal is used in this way, it is necessary to replace the seal every time customization is performed. However, if the function display mode is used, there is no need to rely on such a seal.

  By using the help mode, the user can easily know the details of the functions of the buttons arranged on the operation panel 16 without referring to a manual or the like.

  By using the usage frequency mode, the user can easily know the cumulative number of operations of the buttons arranged on the operation panel 16. Conventionally, the number of button operations has been investigated by referring to the log of the ultrasonic diagnostic apparatus US during maintenance work, but such troublesome work is not necessary. Further, since the button whose cumulative operation count exceeds the threshold value is highlighted, it is easy to grasp the replacement time.

  By using the default display mode, it is possible to easily grasp the default functions of the buttons arranged on the operation panel 16. Therefore, for example, when it is desired to grasp the customization status of the operation panel 16 during maintenance, the working efficiency is significantly increased by using the mode.

  By using the customization mode, the functions of the buttons arranged on the operation panel 16 can be easily customized. In particular, the customization work is performed by a touch operation on the operation panel configuration diagram displayed on the display 34 or the list 210, so that the work is facilitated.

  By using the remote operation mode, it is possible to operate the ultrasonic diagnostic apparatus US without using the operation panel 16. In particular, since an operation panel configuration diagram in which button patterns are arranged simulating actual button shapes and layouts is used, the ultrasonic diagnostic apparatus US can be remotely operated as if operating the operation panel 16.

  Furthermore, the function display mode, the help mode, the usage frequency mode, the default display mode, and the customization mode can be executed regardless of whether or not the ultrasonic diagnostic apparatus US is activated. Therefore, it is not necessary to activate the ultrasonic diagnostic apparatus US for maintenance work or the like.

  As described above, according to the medical work support system according to the present embodiment, various operations related to the operation panel 16 are made efficient. In addition, various suitable actions can be obtained from the configuration disclosed in the present embodiment.

(Modification)
The configuration disclosed in the above embodiment can be variously modified.
For example, in the above embodiment, the case where the operation element arranged on the operation panel is a button is illustrated. However, the operation element may be another operation element such as a switch, a knob, or a trackball.

  In the above embodiment, the medical work support application is installed in the terminal device 3, and the detection unit 100, the recognition unit 101, the function display unit 102, the operation count display unit 104, the help information display unit 103, and the default display unit 105. The case where the functions of the editing unit 106, the remote operation unit 107, and the like are realized by the controller 30 of the terminal device 3 is illustrated. However, all or some of these units may be realized by a device other than the terminal device 3, for example, the ultrasonic diagnostic device US or the server 2. In this case, the terminal device 3 is provided with a browser function, receives processing results from the ultrasound diagnostic apparatus US and the server 2 and displays them on the display 34, and inputs detected by the touch panel 35 to the ultrasound diagnostic apparatus US. And send it to the server 2. It is also possible to construct a medical work support system using the cloud.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Medical work support system, US ... Ultrasonic diagnostic apparatus, 2 ... Server, 3 ... Terminal device, 34 ... Display, M ... Marker, 200 ... Help key, 201 ... Usage frequency key, 202 ... Default display key, 203 ... Customize key, 204, remote operation key, F1, model information file, F2, function allocation file, F3, operation frequency file, F4, help information file, DB, database.

Claims (12)

A terminal device including a camera that captures the surroundings, and a display that displays an image captured by the camera;
A recognition unit for recognizing the operation element from an image of the panel displayed on the display when an operation panel of a medical device in which a plurality of operation elements is arranged is imaged;
The function assigned to the operation element recognized by the recognition unit is identified from the function assignment information indicating the assignment of the function to each of the operation elements arranged on the operation panel, and the information indicating the identified function is A function display unit for displaying on the display in association with the region of the operation element included in the image of the operation panel;
A medical work support system comprising:   2. A help information display unit that displays help information for explaining a function assigned to the operation element included in the image of the operation panel on the display in association with a region of the operation element. Medical work support system described in 1.   From the operation count information indicating the cumulative number of times each of the operation elements has been operated in the past, the cumulative number of times that the operation element recognized by the recognition unit has been operated in the past is specified, and information regarding the specified cumulative count is 2. The medical work support system according to claim 1, further comprising an operation frequency display unit that displays on the display in association with a region of the operation element included in the image of the operation panel.   The medical work support system according to claim 1, further comprising an editing unit that edits the function assignment information by an operation on a display screen of the display.   The operation panel configuration diagram in which patterns corresponding to the operation elements of the operation panel are arranged in accordance with the layout of the operation elements on the operation panel is displayed on the display, and the operation panel configuration diagram includes The medical work support system according to claim 1, further comprising a remote operation unit that instructs the medical device to execute a function assigned to the operation element corresponding to the pattern when the pattern is operated. . The medical work support system includes a plurality of the medical devices that can communicate with the terminal device,
Each of the medical devices stores the function allocation information related to its own operation panel,
The terminal device further includes a detection unit that detects identification information of the medical device,
The function display unit identifies a function assigned to the operation element recognized by the recognition unit from the function assignment information stored in the medical device corresponding to the identification information detected by the detection unit. The medical work support system according to claim 1. The medical work support system includes a server capable of communicating with the terminal device,
The server stores each of the function allocation information related to an operation panel of each medical device,
The function display unit stores the function assigned to the operation element recognized by the recognition unit stored in the medical device when the medical device corresponding to the identification information detected by the detection unit is activated. The function assigned to the operation element recognized by the recognizing unit is identified from the function assignment information related to the operation panel of the medical device stored in the server when the medical device is not activated. The medical work support system according to claim 6, wherein the medical work support system is specified. Each medical device is provided with a code symbol representing its own identification information,
The said detection part detects the said identification information by decoding the said code symbol based on the image obtained when the said code symbol is imaged with the said camera, The Claim 6 or 7 characterized by the above-mentioned. Medical work support system. A terminal device including a camera that captures the surroundings, and a display that displays an image captured by the camera;
A recognition unit for recognizing the operation element from an image of the panel displayed on the display when an operation panel of a medical device in which a plurality of operation elements is arranged is imaged;
The cumulative number of times that the operation element recognized by the recognition unit has been operated in the past is identified from the operation count information indicating the cumulative number of times each of the operation elements arranged in the operation panel has been operated in the past, and is specified. An operation count display unit for displaying information on the accumulated number of times in association with a region of the operation element included in the image of the operation panel and displaying on the display;
A medical work support system comprising: The medical work support system includes a plurality of the medical devices that can communicate with the terminal device,
Each of the medical devices stores the operation frequency information regarding the operation panel of the own device,
The terminal device further includes a detection unit that detects identification information of the medical device,
The operation number display unit specifies the cumulative number of times the operation element recognized by the recognition unit has been operated in the past from the operation number information stored in the medical device corresponding to the identification information detected by the detection unit. The medical work support system according to claim 9. The medical work support system includes a server capable of communicating with the terminal device,
The server stores each of the operation number information related to an operation panel of each medical device,
When the medical device corresponding to the identification information detected by the detection unit is activated, the operation number display unit displays the cumulative number of times the operation element recognized by the recognition unit has been operated in the past. The operation of the medical device stored in the server stores the cumulative number of times that the operation element recognized by the recognition unit has been operated in the past when the medical device is not activated, specified from the stored operation frequency information The medical work support system according to claim 10, wherein the medical work support system is specified from the operation frequency information regarding the panel. Each medical device is provided with a code symbol representing its own identification information,
12. The detection unit according to claim 10, wherein the detection unit detects the identification information by decoding the code symbol based on an image obtained when the code symbol is captured by the camera. Medical work support system.

Images