JP2019165922A - Imaging support program, information processing device, and imaging support method - Google Patents

Abstract

To facilitate acquisition of images with uniform imaging conditions.SOLUTION: A storage part 11 stores a captured image 30 of a predetermined site of a user's body captured in the past. A processing part 12 causes a captured image 41 captured by an imaging device 13 to be displayed on a screen 40 of a display device 14 upon receipt of an instruction to start capturing an image of the predetermined site of the user. The processing part 12 causes an auxiliary image 42 indicating a display position of the predetermined site on the screen 40 at the time when the captured image 30 was captured in the past to be overlapped with the captured image 41 and displayed on the basis of the captured image 30 stored in the storage part 11.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging support program, an information processing apparatus, and an imaging support method.

  A system for performing medical support using information processing technology has been considered. For example, the server receives health status information via the Internet, searches the information stored in the server based on the health status information for a suspicious or cautionary disease name, and transmits the disease description information to the user's personal computer. There is a proposal for a health information management system. There is also a proposal of a skin disease analysis system that predicts a corresponding disease name from accumulated skin disease data based on a photograph of an affected area taken with a digital camera and the content of symptoms, and outputs a countermeasure.

JP 2003-22325 A JP 2005-58398 A

  A user may take a picture of a predetermined part of the body multiple times with a camera or the like at intervals of time, and observe the progress of symptoms such as illness or injury at the part. In this case, in each imaging, if the imaging conditions (for example, the angle and composition at the time of imaging) are different, the progress of symptoms may not be properly grasped.

  In one aspect, an object of the present invention is to facilitate acquisition of an image with uniform imaging conditions.

  In one aspect, an imaging support program is provided. When the imaging support program receives an instruction to start imaging of a predetermined part of the user's body, the imaging support program displays the first image captured by the imaging apparatus on the screen of the display device, and the first part of the user's part captured in the past is displayed. Based on the second image, the computer is caused to execute a process of superimposing and displaying the auxiliary image indicating the display position of the part on the screen when the second image has been captured in the past on the first image.

  In one aspect, it becomes easy to acquire images with uniform imaging conditions.

It is a figure for demonstrating the process of the information processing apparatus of 1st Embodiment. It is a figure which shows the example of the medical assistance system of 2nd Embodiment. It is a block diagram which shows the hardware example of a terminal device. It is a block diagram which shows the hardware example of a server. It is a block diagram which shows the function example of a terminal device. It is a block diagram which shows the function example of a server. It is a figure which shows the example of a disease name master. It is a figure which shows the example of a patient table. It is a figure which shows the example of an affected part table. It is a figure which shows the example of an image table. It is a figure which shows the example of a transparency table. It is a figure which shows the example (the 1) of the screen transition of a terminal device. It is a figure which shows the example (the 2) of the screen transition of a terminal device. It is a figure which shows the example (the 3) of the screen transition of a terminal device. It is a figure which shows the example (the 4) of the screen transition of a terminal device. It is a figure which shows the example (the 5) of the screen transition of a terminal device. It is a figure which shows the example (the 6) of the screen transition of a terminal device. It is a figure which shows the example (the 7) of the screen transition of a terminal device. It is a flowchart which shows the process example of an affected part new registration. It is a flowchart which shows the example of a process of the existing affected part imaging. It is a flowchart which shows the example of a process of an affected part observation support. It is a flowchart which shows the process example of a transparency provision.

Hereinafter, the present embodiment will be described with reference to the drawings.
[First Embodiment]
A first embodiment will be described.

FIG. 1 is a diagram for explaining processing of the information processing apparatus according to the first embodiment.
The information processing apparatus 10 supports imaging of a predetermined part of the body by the user. The information processing apparatus 10 includes a storage unit 11, a processing unit 12, an imaging device 13, and a display device 14.

  The storage unit 11 may be a volatile storage device such as a RAM (Random Access Memory) or a non-volatile storage device such as an HDD (Hard Disk Drive) or a flash memory. The processing unit 12 may include a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), and the like. The processing unit 12 may be a processor that executes a program. The “processor” may include a set of multiple processors (multiprocessor).

The imaging device 13 captures an image with an image sensor in accordance with an instruction from the processing unit 12 and outputs the image to the processing unit 12. The display device 14 displays an image captured by the imaging device 13.
The memory | storage part 11 memorize | stores the 1st image of the predetermined site | part of the user's body imaged in the past. As an example, consider that the user observes the affected part 22 on the finger 21 of the hand 20. The storage unit 11 stores the captured image 30 of the finger 21 as the first image. The captured image 30 is an image of the finger 21 captured in the past by the imaging device 13.

  When the processing unit 12 receives an instruction to start imaging of a predetermined part of the user, the processing unit 12 displays a second image captured by the imaging device 13 on the screen 40 of the display device 14. The screen 40 is an area where an image is displayed on the display device 14. Based on the first image stored in the storage unit 11, the processing unit 12 displays an auxiliary image indicating the display position of the corresponding part on the screen 40 when the first image has been captured in the past as the second image. Superimposed on the display.

  For example, the processing unit 12 receives an instruction to start imaging of the finger 21. Then, the processing unit 12 displays the captured image 41 currently being captured by the imaging device 13 on the screen 40 as a second image. Then, based on the captured image 30, the processing unit 12 superimposes and displays the auxiliary image 42 indicating the display position of the finger 21 on the screen 40 when the captured image 30 has been captured in the past on the captured image 41.

  The auxiliary image 42 is an image obtained by processing the captured image 30 captured in the past. For example, the auxiliary image 42 is an image obtained by performing a transparent process on the captured image 30 so that the captured image 41 can be seen through behind the auxiliary image 42. For example, the processing unit 12 generates the auxiliary image 42 by setting an α value representing the transparency (or transparency) of the captured image 30 to a predetermined value. The auxiliary image 42 thinly represents the palm of the left hand and indicates that the palm side of the finger 21 is directed toward the imaging device 13. The auxiliary image 42 is an image showing the outline of the finger 21 extracted from the captured image 30 by image processing (for example, the outer edge or joint of the finger 21 or an affected part imaged in the past so that the orientation with respect to the imaging device 13 can be seen. Image including the contour of That is, the auxiliary image 42 can also represent the orientation of the finger 21 with respect to the imaging device 13 when the captured image 30 has been captured in the past. The information processing apparatus 10 accepts registration of the names of the parts of the captured image such as the palm of the left hand and the back of the left hand when the captured image 30 is captured, and the registered part when the auxiliary image 42 is displayed. May be displayed on the screen.

  According to the information processing apparatus 10, when an instruction to start imaging of a predetermined part of the user's body is received, the captured image 41 captured by the imaging apparatus 13 is displayed on the screen 40 of the display apparatus 14. Then, based on the captured image 30 of the corresponding part of the user captured in the past, the display position of the part on the screen 40 and the orientation of the part with respect to the imaging device 13 when the captured image 30 has been captured in the past. The auxiliary image 42 shown is superimposed on the captured image 41 and displayed.

  Accordingly, when the user observes the progress of a symptom such as illness or injury (for example, the affected part 22 developed in the finger 21) by himself / herself, it is easy to acquire an image with uniform imaging conditions. it can. For example, the user adjusts the position of the imaging device 13 (or the information processing device 10) so that the auxiliary image 42 and the image of the finger 21 included in the captured image 41 are aligned. When the auxiliary image 42 and the image of the finger 21 included in the captured image 41 are combined, the user inputs an instruction to save the captured image 41 at that time to the information processing apparatus 10. For example, the processing unit 12 may cause the screen 40 to display an input unit for receiving an input of the storage instruction. Alternatively, the information processing apparatus 10 may include an input device for receiving an input of the storage instruction.

  In this way, the user can cause the information processing apparatus 10 to acquire the captured image 41 of the finger 21 under the same conditions as the captured image 30 captured in the past. The user compares the captured image 30 with the captured image 41 to determine the degree of development / healing or healing of the affected area between the past time point when the captured image 30 was captured and the current time point when the captured image 41 was captured. It becomes possible to properly grasp the condition of symptoms such as condition.

Below, a more specific system is illustrated and the function of the information processing apparatus 10 is demonstrated in detail.
[Second Embodiment]
Next, a second embodiment will be described.

FIG. 2 is a diagram illustrating an example of a medical support system according to the second embodiment.
The medical support system according to the second embodiment includes a terminal device 100, a server 200, and a terminal device 300. Server 200 and terminal device 300 are connected to network 50. The network 50 is, for example, the Internet. An access point 51 is connected to the network 50. The access point 51 is a relay device that relays wireless communication, such as a wireless router or a wireless base station. The terminal device 100 is connected to the network 50 via the access point 51 and communicates with the server 200.

  The terminal device 100 is a client computer used by a user. For example, the terminal device 100 is a smart device such as a smartphone or a tablet terminal. The terminal device 100 has a function of capturing an image and a function of displaying an image, and is used for observing the progress of a disease symptom in a predetermined part of the user's body. However, the terminal device 100 may be used for observing the progress of an injury or observing the progress of an injured or ill-treated site (for example, the shape change or pigmentation status of a laser-treated site on the skin).

  In cooperation with the server 200, the terminal device 100 can also present to the user advice from a specialist (for example, a doctor) such as a disease or injury that uses the terminal device 300. The terminal device 100 provides a function for supporting the follow-up observation of a predetermined part by the user. Here, the terminal device 100 is an example of the information processing apparatus 10 according to the first embodiment.

  The server 200 is a server computer that cooperates with the terminal devices 100 and 300. The server 200 receives and stores an image captured by the terminal device 100. The server 200 provides an image to the terminal device 300 and receives information on advice from an expert. The server 200 transmits the received advice information to the terminal device 300.

  The terminal device 300 is a client computer used by an expert. The terminal device 300 displays the image of the affected part of the patient received from the server 200. In addition, the terminal device 300 receives input of advice from an expert, and transmits information on the advice to the server 200.

Here, the medical support system according to the second embodiment may include a terminal device used by a user other than the terminal device 100.
FIG. 3 is a block diagram illustrating a hardware example of the terminal device.

  The terminal device 100 includes a CPU 101, a RAM 102, a flash memory 103, a camera 104, a display 105, a touch panel 106, a media reader 107, and a communication interface 108.

  The CPU 101 corresponds to the processing unit 12 of the first embodiment. The RAM 102 or the flash memory 103 corresponds to the storage unit 11 of the first embodiment. The camera 104 corresponds to the imaging device 13 of the first embodiment. The display 105 corresponds to the display device 14 of the first embodiment.

  The CPU 101 is a processor that executes program instructions. The CPU 101 loads at least a part of the program and data stored in the flash memory 103 into the RAM 102 and executes the program. Note that the CPU 101 may include a plurality of processor cores. The terminal device 100 may have a plurality of processors. The processes described below may be executed in parallel using a plurality of processors or processor cores. A set of processors may be referred to as “multiprocessor” or simply “processor”.

  The RAM 102 is a volatile semiconductor memory that temporarily stores programs executed by the CPU 101 and data used by the CPU 101 for calculations. Note that the terminal device 100 may include a type of memory other than the RAM, or may include a plurality of memories.

  The flash memory 103 is a non-volatile storage device that stores software programs such as an OS (Operating System), middleware, and application software, and data. The terminal device 100 may include a plurality of nonvolatile storage devices.

  The camera 104 is an imaging device mounted on the terminal device 100. The camera 104 includes an image sensor such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 104 generates still image data or moving image data of a scene in front of the lens of the camera 104 according to a command from the CPU 101. The camera 104 is provided on the back surface of the terminal device 100 (the surface opposite to the display surface of the display 105). The terminal device 100 may include another camera on the front surface (the same surface as the display surface of the display 105) of the terminal device 100 (in this case, for example, the user places the other camera on the affected area of his / her face). Can be used for imaging).

  The display 105 outputs an image according to a command from the CPU 101. As the display 105, a liquid crystal display (LCD: Liquid Crystal Display), an organic EL (OEL: Organic Electro-Luminescence) display, or the like can be used. The display 105 displays an image captured by the camera 104.

  The touch panel 106 is a pointing device that outputs a position touched by the user to the CPU 101. The touch panel 106 is provided so as to overlap the display area of the display 105. The user can perform a touch operation on the touch panel 106 while visually confirming the display on the display 105.

  The medium reader 107 is a reading device that reads programs and data recorded on the recording medium 111. As the recording medium 111, for example, a semiconductor memory can be used. For example, the medium reader 107 copies a program or data read from the recording medium 111 to another recording medium such as the RAM 102 or the flash memory 103. The read program is executed by the CPU 101, for example. The recording medium 111 may be a portable recording medium and may be used for distributing programs and data. In addition, the recording medium 111 and the flash memory 103 may be referred to as computer-readable recording media.

  The communication interface 108 is an interface that is connected to the network 50 via the access point 51 and communicates with the server 200 via the network 50. The communication interface 108 is connected to the access point 51 wirelessly. However, the communication interface 108 may be connected to the access point 51 by wire.

Note that the terminal device 300 is also realized using the same hardware as the terminal device 100.
FIG. 4 is a block diagram illustrating a hardware example of the server.
The server 200 includes a CPU 201, a RAM 202, an HDD 203, an image signal processing unit 204, an input signal processing unit 205, a medium reader 206, and a communication interface 207.

  The CPU 201 is a processor that executes program instructions. The CPU 201 loads at least a part of the program and data stored in the HDD 203 into the RAM 202 and executes the program. Note that the CPU 201 may include a plurality of processor cores. The server 200 may have a plurality of processors.

  The RAM 202 is a volatile semiconductor memory that temporarily stores programs executed by the CPU 201 and data used by the CPU 201 for calculation. Note that the server 200 may include a type of memory other than the RAM, or may include a plurality of memories.

  The HDD 203 is a non-volatile storage device that stores software programs such as an OS, middleware, and application software, and data. The server 200 may include other types of storage devices such as flash memory and SSD, and may include a plurality of nonvolatile storage devices.

  The image signal processing unit 204 outputs an image to the display 211 connected to the server 200 in accordance with a command from the CPU 201. As the display 211, any type of display such as a CRT (Cathode Ray Tube) display, LCD, plasma display, OEL display, or the like can be used.

  The input signal processing unit 205 acquires an input signal from the input device 212 connected to the server 200 and outputs it to the CPU 201. As the input device 212, a pointing device such as a mouse, a touch panel, a touch pad, a trackball, a keyboard, a remote controller, a button switch, or the like can be used. A plurality of types of input devices may be connected to the server 200.

  The medium reader 206 is a reading device that reads programs and data recorded on the recording medium 213. As the recording medium 213, for example, a magnetic disk, an optical disk, a magneto-optical disk (MO), a semiconductor memory, or the like can be used. Magnetic disks include flexible disks (FD: Flexible Disk) and HDDs. The optical disc includes a CD (Compact Disc) and a DVD (Digital Versatile Disc).

  For example, the medium reader 206 copies the program and data read from the recording medium 213 to another recording medium such as the RAM 202 and the HDD 203. The read program is executed by the CPU 201, for example. The recording medium 213 may be a portable recording medium and may be used for distributing programs and data. Further, the recording medium 213 and the HDD 203 may be referred to as computer-readable recording media.

  The communication interface 207 is an interface that is connected to the network 50 and communicates with other computers including the terminal devices 100 and 300 via the network 50. The communication interface 207 is connected to a communication device such as a switch or a router with a cable, for example.

FIG. 5 is a block diagram illustrating an example of functions of the terminal device.
The terminal device 100 includes a storage unit 120, a UI (User Interface) unit 130, a communication processing unit 140, and a display control unit 150. The storage unit 120 is realized using a storage area of the RAM 102 or the flash memory 103. The UI unit 130, the communication processing unit 140, and the display control unit 150 are realized by the CPU 101 executing a program stored in the RAM 102.

  The storage unit 120 stores a disease name master, a patient table, an affected part table, and an image table. The disease name master is a table in which a disease name is associated with a standard period (referred to as a standard treatment period) required for treatment of a disease having the disease name. The disease name master is acquired from the server 200 and stored in the storage unit 120. The patient table is a table that manages information related to a user (referred to as a patient) who is an object of observation of progress by the terminal device 100. In addition to the user himself / herself, other users related to the user (for example, family members, friends, other nursing subjects) may be registered as patients in the patient table. The affected part table is a table for managing information related to the affected part of the patient. The image table is a table for managing an image obtained by capturing an affected area (referred to as an affected area image).

  The storage unit 120 stores the whole body and each part of the body (ear, forehead, cheek, nose, chin, neck, shoulder, arm, elbow, hand, finger of the hand, wrist, heel, chest, waist, abdomen, thigh, A model image representing a three-dimensional model of a knee, a shin, an ankle, a foot, a toe, a heel, a sole, and the like is stored in advance. The storage unit 120 stores a two-dimensional schema image of each part in advance.

  The UI unit 130 causes the display 105 to display a GUI (Graphical User Interface) that receives input from the patient. The UI unit 130 may display expert advice acquired from the server 200 by the communication processing unit 140 on the display 105. Also, when the UI unit 130 enters the imaging mode of the camera 104, the UI unit 130 displays an image captured by the camera 104 on the display 105 according to the control of the display control unit 150.

  The communication processing unit 140 communicates with the server 200. The communication processing unit 140 transmits the affected part image stored in the storage unit 120 to the server 200. In addition, the communication processing unit 140 receives expert advice from the server 200. Furthermore, the communication processing unit 140 receives information on the transparency according to the part to be imaged from the server 200. The transparency information is used for image processing by the image processing unit 152.

  The display control unit 150 causes the display 105 to display a GUI that supports imaging of the affected part by the patient. The display control unit 150 includes a camera image acquisition unit 151 and an image processing unit 152.

  The camera image acquisition unit 151 acquires an image captured by the camera 104 (referred to as a camera image) from the camera 104 and stores it in a predetermined storage area (referred to as a buffer) of the RAM 102. The camera image is an image currently captured by the camera 104.

  The image processing unit 152 superimposes the guide image on the camera image and displays it on the display 105. The guide image is an image that supports imaging of a predetermined part by the patient. When registering a certain affected part in the terminal device 100 for the first time, the image processing unit 152 generates a guide image based on the schema image of the corresponding part stored in the storage unit 120. In addition, when newly imaging an affected part (existing affected part) that is currently being observed, the image processing unit 152 obtains an affected part image captured in the past for the affected part from the storage unit 120, and based on the affected part image A guide image is generated. The guide image is an image in which a schema image or a diseased part image captured in the past is in a translucent state. For example, the image processing unit 152 can change the setting of the α value indicating the transparency of the affected part image to make the schema image or the affected part image translucent. The image processing unit 152 acquires an initial value (recommended value) of transparency applied to the region to be imaged from the server 200 via the communication processing unit 140.

  The image processing unit 152 may receive a change in the transparency of the guide image being displayed by the user. In that case, the transparency of the guide image being displayed is changed to the designated transparency. When the current affected part image is acquired for the region to be imaged, the image processing unit 152 sends the information on the region to be imaged this time and the information on the transparency used this time via the communication processing unit 140. To the server 200. Here, “the affected part image is acquired” means that the camera image at a certain timing stored in the buffer is stored as an affected part image in the nonvolatile area of the storage unit 120 and the affected part image is registered in the image table. Indicates.

FIG. 6 is a block diagram illustrating an exemplary function of the server.
The server 200 includes a storage unit 220, a communication processing unit 230, and a transparency calculation unit 240. The storage unit 220 is realized using a storage area of the RAM 202 or the HDD 203. The communication processing unit 230 and the transparency calculation unit 240 are realized by the CPU 201 executing a program stored in the RAM 202.

The memory | storage part 220 memorize | stores the information of the transmittance | permeability for every site | part used by the patient. In addition, the storage unit 220 stores the affected part image acquired from the terminal device 100.
The communication processing unit 230 communicates with the terminal devices 100 and 300. The communication processing unit 230 receives the affected part image from the terminal device 100 and stores it in the storage unit 220. The communication processing unit 230 transmits the affected part image and the advice request to the terminal device 300. In response to the advice request, the communication processing unit 230 receives the advice from the terminal device 300 and stores it in the storage unit 220. The communication processing unit 230 transmits advice to the terminal device 100.

  In addition, the communication processing unit 230 receives information indicating a part to be imaged by the terminal device 100. Then, the communication processing unit 230 instructs the transparency calculation unit 240 to calculate a recommended value of transparency according to the region to be imaged, and acquires the calculated recommended value. The communication processing unit 230 transmits the calculated recommended value of transparency to the terminal device 100. Further, the communication processing unit 230 acquires information on the part that is the current imaging target and information on the transparency used in the current imaging from the terminal device 100, and associates the information on the part with the information on the transparency. In addition, it is stored in the storage unit 220.

  When the transparency calculation unit 240 receives an instruction for calculating the transparency from the communication processing unit 230, the transparency calculation unit 240 recommends the transparency for the site to be imaged this time based on the transparency information for each site accumulated in the storage unit 220. The value is calculated and provided to the communication processing unit 230. The transmittance calculation unit 240 uses a statistical value (for example, an average value or a percentile value) of the transmittance of the corresponding part accumulated in the storage unit 220 as a recommended value of the transmittance of the part. In addition, when the record of the transmittance | permeability with respect to the applicable site | part is not recorded on the memory | storage part 220, the transmittance | permeability calculation part 240 makes the recommended value of the transmittance | permeability of the said part the predetermined value (for example, 50%).

  Note that the transmittance calculation unit 240 calculates a statistical value (for example, an average value or a percentile value) of the transmittance for the results of the transmittance used for all parts, not for each part, and calculates the statistical value. May be a recommended value of transparency.

Next, an example of data stored in the terminal device 100 and the server 200 will be described.
FIG. 7 is a diagram illustrating an example of a disease name master.
The disease name master 121 is stored in the storage unit 120. The disease name master 121 includes items of a disease name and a standard treatment period. A disease name is registered in the disease name item. In the standard treatment period item, a standard period required for treatment of a disease is registered. For example, in the standard treatment period, the number of days corresponding to the past performance of the same disease is set in advance.

  For example, in the disease name master 121, a record having a disease name “IBO” and a standard treatment period “7 days” is registered. This record indicates that the standard number of days until the disease with the disease name “Ibo” is cured is 7 days.

  The standard treatment period may be determined by the server 200. For example, the server 200 obtains the number of days required for treatment for each disease from a plurality of terminal devices used by the patient, determines its statistical value (for example, the average or percentile value) as the standard treatment period, You may provide to each terminal device.

FIG. 8 is a diagram illustrating an example of a patient table.
The patient table 122 is stored in the storage unit 120. The patient table 122 includes items of patient ID (IDentifier) and date of birth. Patient identification information (patient ID) is registered in the item of patient ID. The birth date of the patient is registered in the birth date item.

  For example, in the patient table 122, a record having a patient ID “U1” and a date of birth “19900101” is registered. This record indicates that the date of birth of the patient with the patient ID “U1” is January 1, 1990.

FIG. 9 is a diagram illustrating an example of an affected area table.
The affected part table 123 is stored in the storage unit 120. The affected area table 123 includes items of patient ID, affected area ID, site name, disease name, drug name, onset date, and in-treatment flag. A patient ID is registered in the item of patient ID. In the item of affected area ID, identification information (affected area ID) of the affected area is registered. In the part name item, the name of the part including the affected part is registered. The disease name of the affected area is registered in the disease name item. The name of the medicine used for treatment is registered in the medicine name item. If there is no drug in use, the drug name item is not set. In the onset date item, the onset date of the disease is registered. A flag indicating whether or not the affected part is being treated is registered in the item of “in-treatment flag”. The in-treatment flag “TRUE” indicates that treatment is in progress. The in-treatment flag “FALSE” indicates that the treatment is not being performed, that is, the treatment has been completed. The default value of the in-treatment flag is “TRUE”.

  For example, in the affected area table 123, the patient ID is “U1”, the affected area ID is “001”, the site name is “left index finger”, the disease name is “wart”, the drug name is “medicine A”, and the onset date is “2018 / 2/22 "and a record that the treatment flag is" TRUE "are registered. In this record, the patient with the patient ID “U1” has developed warts on the left index finger, the affected part corresponding to the wart is managed with the affected part ID “001”, and the drug name “medicine A” is used to treat the wart. It indicates that the drug is being used and the wart onset date is February 22, 2018, and is still being treated.

FIG. 10 is a diagram illustrating an example of an image table.
The image table 124 is stored in the storage unit 120. The image table 124 includes items of patient ID, affected area ID, imaging date and time, and image file. A patient ID is registered in the item of patient ID. The affected area ID is registered in the affected area ID item. The date and time when the affected part image was acquired is registered in the item of imaging date and time. In the item of the image file, a data body of the affected part image or a path name indicating the data body of the affected part image is registered.

  For example, in the image table 124, a record with a patient ID “U1”, an affected part ID “001”, an imaging date “2018/3/1”, and an image file “image file F1” is registered. This record indicates the affected part image “image file F1” acquired on March 1, 2018 for the affected part with the affected part ID “001” of the patient with the patient ID “U1”. In the image table 124, a plurality of affected part images can be registered for a certain affected part.

FIG. 11 is a diagram illustrating an example of a transparency table.
The transparency table 221 is stored in the storage unit 220. The transparency table 221 includes items of part name and transparency. In the part name item, the name of the imaged part is registered. The transparency used at the time of imaging is registered in the item of transparency. For example, the transmittance is expressed as a percentage. More specifically, a transmittance of 0% indicates a completely opaque state. A transmittance of 100% indicates a completely transparent state. By setting the transparency to a value between 0 and 100%, the transparency of the guide image is adjusted.

  For example, in the transparency table 221, a record having a part name “left index finger” and a transparency “40%” is registered. This record indicates that imaging is performed with the transparency of the guide image set to 40% when imaging a site of the site name “left hand index finger” of a patient. As described above, in the transparency table 221, the information on the transparency actually used at the time of imaging is stored in association with the part name of the part to be imaged.

Next, an example of screen transition of the terminal device 100 will be described.
FIG. 12 is a diagram illustrating an example (part 1) of the screen transition of the terminal device.
When a predetermined application that supports follow-up observation is activated, the UI unit 130 displays a menu screen 500 on the display 105. When a plurality of patients are managed in the terminal device 100, the UI unit 130 displays a patient selection screen for selecting a patient before the menu screen 500. Then, the UI unit 130 displays a menu screen 500 corresponding to the patient selected on the patient selection screen.

  The menu screen 500 is a screen that presents a list of support services provided to a patient (or a user other than a patient who performs imaging of an affected part of the patient). The support service includes support for new registration of an affected area, support for imaging an existing affected area, and support for observation of an affected area. The menu screen 500 includes an affected area new registration button 501, an existing affected area imaging button 502, and an affected area observation button 503.

  An affected area new registration button 501 is a button for activating a new affected area registration process. The existing affected area imaging button 502 is a button for starting processing of existing affected area imaging. The affected area observation button 503 is a button for starting the affected area observation support process. The existing affected part imaging button 502 and the affected part observation button 503 are not displayed when there is no registered affected part for the corresponding patient. The patient can select the button by touching (or tapping) the position of a desired button on the touch panel 106, and can cause the terminal device 100 to start processing corresponding to the button. The menu screen 500 may further include an end button for ending the application.

  First, a case where the new affected area registration button 501 is selected will be described. When a patient newly registers an affected area, the patient selects an affected area new registration button 501. When the UI unit 130 accepts a selection for the new lesion registration button 501, the UI unit 130 displays the input screen 510 on the display 105. The input screen 510 includes a disease name input form 511, a drug name input form 512, a days input form 513, and an onset site button 514.

  The disease name input form 511 is a form for inputting a disease name estimated by the patient for the affected part. The UI unit 130 supplements a disease name registered in advance in the disease name master 121 in accordance with the text input in the disease name input form 511, and supports input of the disease name. Further, the UI unit 130 may support input of a disease name registered in advance in the disease name master 121 by using the disease name input form 511 in a pull-down format.

The medicine name input form 512 is a form for inputting a medicine name of a medicine used for treatment of the affected part.
The number-of-days input form 513 is a form for inputting the number of days since the onset of illness. The UI unit 130 sets the date on which the disease is onset as a date that has been traced back by the number of days input in the day input form 513 from the current day.

  The onset site button 514 is a button for starting the site selection screen 520. The UI unit 130 allows the user to select the onset site button 514 when input to the disease name input form 511, the drug name input form 512, and the number of days input form 513 is completed, and in other cases, the onset site button 514 Cannot be selected.

FIG. 13 is a diagram illustrating an example (part 2) of the screen transition of the terminal device.
When the onset site button 514 on the input screen 510 is selected, the UI unit 130 displays the site selection screen 520 on the display 105. The part selection screen 520 includes a whole body image 521 representing the whole body of a person. The whole body image 521 is drawn based on the model image of the whole body three-dimensional model stored in the storage unit 120 in advance. The UI unit 130 rotates the whole-body image 521 in response to a predetermined drag operation on the touch panel 106 and causes the display 105 to display a user's desired location (for example, a side surface or a back side of the whole-body image 521). The rotation axis of the image rotation is, for example, a horizontal axis with respect to the screen of the display 105 (for example, an axis passing through the midpoint of the vertical side of the rectangular screen area) and a vertical axis (for example, a rectangular It can be considered that the axis passes through the midpoint of the horizontal side of the screen area.

  When the portion 521a of the whole body image 521 is selected, the UI unit 130 causes the display 105 to display a part selection screen 530. Here, as an example, it is assumed that the part 521a is the left arm part of the whole body image 521. In this case, the part selection screen 530 includes an arm image 531 of the left arm. The arm image 531 is drawn based on the model image of the three-dimensional model of the left arm stored in advance in the storage unit 120, as with the whole body image 521. The UI unit 130 rotates the arm image 531 in accordance with a predetermined drag operation on the touch panel 106 and causes the display 105 to display a user's desired location (for example, the inside or outside of the arm).

  Furthermore, when the part 531 a of the arm image 531 is selected, the UI unit 130 displays a part selection screen 540 on the display 105. Here, as an example, it is assumed that the portion 531 a is a hand portion of the arm image 531. In this case, the part selection screen 540 includes a left hand image 541. The hand image 541 is drawn based on the model image of the three-dimensional model of the left hand stored in advance in the storage unit 120, like the whole body image 521 and the arm image 531. The UI unit 130 rotates the hand image 541 in response to a predetermined drag operation on the touch panel 106, and causes the display 105 to display a user's desired location (for example, the palm side or the back of the hand).

Then, when the part 541a of the hand image 541 is selected, the UI unit 130 specifies the part 541a as a part to be imaged this time, and notifies the display control unit 150 of the specified part.
Then, the display control unit 150 generates the imaging support screen 550 using the schema image stored in advance in the storage unit 120 in association with the identification information of the portion 541a and displays the imaging support screen 550 on the display 105. For one part, a schema image when the part is viewed from a plurality of viewpoints is stored in advance in the storage unit 120 for each viewpoint. The display control unit 150 generates the imaging support screen 550 using the schema image corresponding to the display of the corresponding part when the part is selected on the part selection screen 540.

  Here, as an example, it is assumed that the portion 541a is a left index finger portion viewed from the palm side. In this case, the display control unit 150 obtains a schema image of the left index finger viewed from the palm side from the storage unit 120, generates a guide image 551 corresponding to the left index finger part based on the schema image, and supports imaging Include in screen 550. The display position of the guide image 551 is fixed in the imaging support screen 550. The imaging support screen 550 includes a camera image as the background of the guide image 551 (however, the camera image is not shown in FIG. 13). The guide image 551 is subjected to a transparent process. The patient can see the background camera image through the guide image 551. At this time, the display control unit 150 sets the transparency of the guide image 551 as the recommended value acquired from the server 200. The display control unit 150 can acquire a recommended value of transparency according to the region to be imaged from the server 200.

  In addition, the imaging support screen 550 includes a transparency changing unit 552 for accepting a change in the transparency of the guide image 551. The transparency changing unit 552 is, for example, a slide bar. For example, the display control unit 150 changes the transparency of the guide image 551 in accordance with the change in the position of the bar in the slide bar (for example, the transparency is increased when the bar is moved upward by the patient, For example, if the bar is moved downwards by, the transparency is reduced). However, the transparency changing unit 552 may be an input form of another format such as an input form for inputting a transparency value.

FIG. 14 is a diagram illustrating an example (part 3) of the screen transition of the terminal device.
The patient points the camera 104 of the terminal device 100 toward the affected area and images a part including the affected area. For example, when the patient newly starts observing the affected part 62 that has developed in the finger 61 of his / her hand 60, the terminal device 100 selects the finger 61 as a region to be imaged. Then, the display control unit 150 displays the imaging support screen 550 on the display 105. The imaging support screen 550 includes a camera image 553 that is captured by the camera 104. When the lens of the camera 104 is pointed at the finger 61, the camera image 553 includes an image of the finger 61 or the affected part 62 formed on the finger 61.

  Then, the display control unit 150 displays the guide image 551 superimposed on the camera image 553. The patient can see the camera image 553 through the guide image 551. Further, the display control unit 150 changes the transparency of the guide image 551 in accordance with the change of the transparency using the transparency changing unit 552. For example, increasing the transparency of the guide image 551 makes it easier to see the image of the finger 61 and the affected part 62 in the camera image 553. In addition, for example, by reducing the transparency of the guide image 551, the guide image 551 can be easily seen. Thus, the patient images the finger 61 including the affected part 62 in accordance with the guide image 551. For example, the patient inputs the affected part image acquisition instruction to the terminal device 100 with the image of the finger 61 in the camera image 553 being matched with the image of the finger corresponding to the finger 61 in the guide image 551. The imaging support screen 550 may include a button that receives an input of the acquisition instruction. Alternatively, the terminal device 100 may include the button on the side surface of the casing of the terminal device 100 or the like.

  When the display control unit 150 receives an instruction to acquire an affected part image, the display control unit 150 displays the affected part image display screen 560 on the display 105. The affected area image display screen 560 includes an affected area image 561. The affected part image 561 is an image obtained by removing the guide image 551 and the transparency changing unit 552 from the images included in the imaging support screen 550 when the instruction for acquiring the affected part image is received (that is, receiving the instruction for acquiring the affected part image). Camera image 553).

  The affected area image display screen 560 includes a re-imaging button 562 and a registration button 563. The re-imaging button 562 is a button for canceling the affected area image 561 and starting re-imaging of the affected area image. When the re-imaging button 562 is selected, the display control unit 150 discards the affected part image 561 and returns to the display of the imaging support screen 550.

  The registration button 563 is a button for confirming the affected part image 561 and registering information on the new affected part and the affected part image in the affected part table 123 and the image table 124. When the registration button 563 is selected, the display control unit 150 registers information related to the affected area input on the input screen 510 in the affected area table 123 and registers the affected area image 561 in the image table 124. In this way, new registration of the affected part is completed. When the affected part image is acquired, the communication processing unit 140 transmits the information on the part name of the affected part captured this time and the information on the transmittance used in the current imaging to the server 200. Then, the UI unit 130 displays the menu screen 500 on the display 105.

FIG. 15 is a diagram illustrating an example (part 4) of the screen transition of the terminal device.
Next, a case where the existing affected area imaging button 502 is selected on the menu screen 500 will be described. When the existing affected part imaging button 502 is selected, the display control unit 150 causes the display 105 to display an imaging support screen 570 corresponding to the existing affected part for the corresponding patient. The imaging support screen 570 includes a guide image 571 and a transparency changing unit 572. The guide image 571 is a guide image generated based on an affected area image captured in the past for the affected area. The transparency changing unit 572 is an input form that accepts a change in the transparency of the guide image 571, and is represented by, for example, a slide bar, similarly to the transparency changing unit 552.

  Here, the guide image 571 is different from the guide image 551 generated based on the schema image in that the guide image 571 is generated based on the affected part image captured in the past with respect to the affected part. Since the guide image 571 is generated from past images of the same affected area of the same patient, the current affected area shown in the camera image (not shown in FIG. 15) matches the pattern such as the outline and wrinkles. The degree of this is higher than that of the guide image 551 generated from the schema image. For this reason, the patient can easily match the image of his / her affected part included in the camera image with the guide image 571. In this way, the patient can acquire the affected area image by the terminal device 100 in the same manner as in FIG. When the acquisition of the affected area image is completed, the UI unit 130 displays the menu screen 500 on the display 105.

  Note that the terminal device 100 may manage a plurality of existing affected areas whose in-treatment flag in the affected area table 123 is “TRUE” for the corresponding patient. In that case, the UI unit 130 displays an existing affected part selection screen that accepts selection of an existing affected part whose treatment flag is “TRUE” before the imaging support screen 570. And the display control part 150 displays the imaging assistance screen 570 according to the selected existing affected part.

FIG. 16 is a diagram illustrating an example (part 5) of the screen transition of the terminal device.
Next, a case where the affected area observation button 503 is selected on the menu screen 500 will be described. When the affected part observation button 503 is selected, the UI unit 130 displays the follow-up observation screen 580 on the display 105. The follow-up observation screen 580 includes affected area images 581, 582, 583, 584, a consultation button 585, a healing button 586, and a completion button 587.

  The affected part images 581, 582, 583 and 584 represent the progress of the affected part in time series. For example, the affected area image 581 is an image captured one week ago. The affected part image 582 is an image taken five days ago. The affected part image 583 is an image taken two days ago. The affected part image 584 is an image taken today. For example, the patient can easily grasp the progress of the affected part 62 by checking the affected part images 581, 582, 583, 584 imaged under the same conditions in time series.

  The consultation button 585 is a button for requesting advice for an expert regarding the affected area. When the consultation button 585 is selected, the UI unit 130 causes the display 105 to display a transmission confirmation screen 600 for confirming transmission of the affected part image. The healing button 586 is a button for inputting that the affected area has been healed by treatment. When the healing button 586 is selected, the UI unit 130 changes the in-treatment flag of the affected part from “TRUE” to “FALSE”. Then, the affected part is managed as having been treated. The completion button 587 is a button for closing the follow-up observation screen 580 and returning to the menu screen 500.

  In addition, when there are a plurality of affected parts whose treatment target flag of the affected part table 123 is “TRUE” for the corresponding patient, the UI unit 130 performs the follow-up observation before displaying the follow-up observation screen 580. An existing affected area selection screen for accepting selection is displayed. Then, the UI unit 130 displays a follow-up observation screen 580 corresponding to the selected existing affected part.

FIG. 17 is a diagram illustrating an example (part 6) of the screen transition of the terminal device.
When displaying the follow-up observation screen 580, the UI unit 130 determines whether or not the treatment period of the affected part exceeds the standard treatment period based on the disease name master 121. When the treatment period exceeds the standard treatment period, the UI unit 130 displays a consultation recommendation message 590 on the display 105. The consultation recommendation message 590 is a message that recommends consultation with a specialist for a patient. The consultation recommendation message 590 includes a confirmation button 591. When the confirmation button 591 is selected, the UI unit 130 deletes the consultation recommendation message 590. As described above, the terminal device 100 can prompt the patient to consult with the expert by displaying the consultation recommendation message 590 when the affected area is estimated to be healed slowly, and the patient's health can be improved early. Can help you recover.

FIG. 18 is a diagram illustrating an example (part 7) of the screen transition of the terminal device.
When the consultation button 585 is selected on the follow-up observation screen 580, the UI unit 130 displays the transmission confirmation screen 600 on the display 105. The transmission confirmation screen 600 includes an affected part image displayed on the follow-up observation screen 580, a stop button 601 and a transmission button 602.

  A cancel button 601 is a button for canceling the consultation with the expert and returning to the follow-up observation screen 580. The transmission button 602 is a button for instructing transmission of the affected area image displayed on the transmission confirmation screen 600 to the server 200. When the transmission button 602 is selected, the UI unit 130 transmits a corresponding affected part image to the server 200.

  Thereafter, the server 200 receives the affected part image transmitted by the terminal device 100 and provides the affected part image to the terminal device 300. When the server 200 receives expert advice from the terminal device 300, the server 200 transmits an advice reception notification to the terminal device 100.

  Upon receiving the advice reception notification, the UI unit 130 acquires the advice from the server 200 and causes the display 105 to display the advice display screen 610. For example, when receiving a notification of advice reception, the UI unit 130 displays a button for selecting reception of advice on the menu screen 500 of the relevant affected part, and when the button is selected by the patient, Get advice from 200.

  The advice display screen 610 includes the content of expert advice. For example, the expert's advice includes a disease name determined by the expert, a drug name of a drug to be used by the patient, and the like. The advice display screen 610 includes a re-registration button 611. The re-registration button 611 is a button for re-registering a disease name and a medicine name to be used in accordance with expert advice. When the re-registration button 611 is selected, the UI unit 130 displays a re-registration screen 620 on the display 105.

  The re-registration screen 620 includes a disease name input form 621, a drug name input form 622, and a completion button 623. The disease name input form 621 is an input form for inputting a disease name. The medicine name input form is an input form for inputting a medicine name. The completion button 623 is a button for confirming the contents input in the disease name input form 621 and the medicine name input form 622 and registering them in the affected area table 123. When the completion button 623 is selected, the UI unit 130 updates the setting of the disease name and the drug name of the affected part in the affected part table 123 according to the contents input in the disease name input form 621 and the drug name input form 622.

  Next, a processing procedure executed by the terminal device 100 will be described. First, the processing procedure for new registration of an affected area will be described. The new affected area registration process is executed when the new affected area registration button 501 is selected on the menu screen 500.

FIG. 19 is a flowchart illustrating a process example of new registration of an affected area.
(S10) The UI unit 130 receives input of information about the affected part (disease name, drug name, and number of days since the onset) from the input screen 510, and selects an imaging target part based on the three-dimensional model on the part selection screens 520, 530, and 540. Accept.

(S11) The communication processing unit 140 acquires the transparency corresponding to the part selected in step S10 from the server 200.
(S12) The image processing unit 152 acquires a schema image corresponding to the part selected in step S10 from the storage unit 110, and generates a guide image 551 based on the schema image. The image processing unit 152 uses a schema image corresponding to the display of the three-dimensional model when the part is selected in step S10 to generate the guide image 551. The image processing unit 152 may generate the schema image or the guide image 551 by extracting the outline of the three-dimensional model of the part when the part is selected in step S10 by image processing. The image processing unit 152 sets the transparency of the guide image 551 as the transparency acquired in step S11. Note that, even when the guide image is a line representing the contour of the corresponding part, the transparency processing using the transparency can be performed on the line.

  (S13) The camera image acquisition unit 151 starts imaging by the camera 104, and displays the camera image 553 captured by the camera 104 on the display 105. At this stage, the imaging support screen 550 is displayed on the display 105.

(S14) The image processing unit 152 superimposes and displays the guide image 551 on the camera image 553.
(S15) The image processing unit 152 determines whether the transparency changing unit 552 has received a user's instruction to change the transparency. When the instruction to change the transparency is received, the process proceeds to step S16. If the instruction to change the transparency is not accepted, the process proceeds to step S17.

(S16) The image processing unit 152 changes the transparency of the currently displayed guide image 551 in response to a user's instruction to change the transparency.
(S17) The camera image acquisition unit 151 receives a camera image acquisition instruction from the user.

  (S18) In response to the camera image acquisition instruction, the camera image acquisition unit 151 stores the currently captured camera image 553 in the storage unit 120 as the affected part image 561. It should be noted that the camera image acquisition unit 151 confirms whether or not to save to the user on the affected part image display screen 560 before saving, and saves the affected part image 561 when the registration button 563 is selected. The camera image acquisition unit 151 registers the information on the affected part image 561 saved this time in the affected part table 123 and the image table 124. Further, the image processing unit 152 stores the schema image used for generating the guide image 551 in step S12 or the guide image 551 generated in step S12 in the storage unit 120 in association with the patient ID and the affected part ID. The camera image acquisition unit 151 ends imaging by the camera 104.

(S19) The communication processing unit 140 transmits the transparency at the time of the current imaging (the transparency when the camera image acquisition instruction is received in step S17) to the server 200.
As described above, when a new affected part is newly registered (when an affected part image of the affected part is not acquired), the display control unit 150 generates a guide image based on the schema image, and uses the guide image. Then, an imaging method (an imaging angle, a composition, etc.) of the part to be imaged is guided. Thereby, the patient can smoothly pick up the affected part image of the new affected part.

Next, the processing procedure of the existing affected part imaging is demonstrated. The existing affected area imaging process is executed when the existing affected area imaging button 502 is selected on the menu screen 500.
FIG. 20 is a flowchart illustrating a processing example of imaging of an existing affected area.

(S20) The communication processing unit 140 acquires the transparency according to the part selected in step S10 from the server 200.
(S21) The image processing unit 152 refers to the patient table 122 and determines whether the patient to be imaged is a predetermined age or less. If the patient is below the predetermined age, the process proceeds to step S22. If the patient is older than the predetermined age, the process proceeds to step S23. The predetermined age is preset in the storage unit 120. For example, the predetermined age is 12 years (the upper limit of the period during which the body is considered to grow significantly).

  (S22) The image processing unit 152 refers to the image table 124 and determines whether or not a predetermined period has elapsed since the previous imaging for the existing affected area to be imaged. If the predetermined period has elapsed since the previous imaging, the process proceeds to step S24. If the predetermined period has not elapsed since the previous imaging, the process proceeds to step S23. The predetermined period is set in the storage unit 120 in advance. For example, the predetermined period is one year. The predetermined period may be changed according to the age of the patient. For example, if the age is 5 years old or less, the predetermined period is 3 months, and if it is 6-12 years old, the predetermined period is 6 years (because the body growth is particularly remarkable until about 5 years old). ).

  (S23) The image processing unit 152 generates a guide image 571 based on the previous affected part image 561 recorded in the image table 124. The image processing unit 152 sets the transparency of the guide image 571 as the transparency acquired in step S20. Then, the process proceeds to step S25.

  (S24) The image processing unit 152 generates a guide image based on the schema image saved in the storage unit 120 in step S18. Note that the image processing unit 152 may store the guide image 551 originally generated in the storage unit 120 as described above. In this case, the image processing unit 152 may acquire the guide image 551 stored in the storage unit 120. Then, the process proceeds to step S25.

(S25) The image processing unit 152 starts imaging by the camera 104, and causes the display 105 to display the camera image captured by the camera 104.
(S26) The image processing unit 152 displays the guide image superimposed on the camera image. At this time, the guide image to be superimposed is a guide image 571 based on the previous affected area image 561 when step S23 is executed, and a guide image 551 based on the schema image when step S24 is executed. At this stage, an imaging support screen 570 including a camera image captured by the camera 104, a guide image, and a transparency changing unit 572 is displayed on the display 105. However, as described above, the camera image is not shown in the imaging support screen 570 of FIG.

  (S27) The image processing unit 152 determines whether or not the user has received an instruction to change the transparency. If an instruction to change the transparency is received, the process proceeds to step S28. If the instruction to change the transparency is not accepted, the process proceeds to step S29.

(S28) The image processing unit 152 changes the transparency of the currently displayed guide image in response to the user's instruction to change the transparency.
(S29) The camera image acquisition unit 151 receives a camera image acquisition instruction from the user.

  (S30) In response to a camera image acquisition instruction, the camera image acquisition unit 151 stores the currently captured camera image in the storage unit 120 as an affected part image. The camera image acquisition unit 151 registers the information on the affected part image stored this time in the affected part table 123 and the image table 124.

(S31) The communication processing unit 140 transmits to the server 200 the transparency at the time of the current imaging (the transparency when the camera image acquisition instruction is received in step S29).
As described above, the display control unit 150 displays the guide image (auxiliary image) generated based on the affected part image based on the age of the patient (user), or displays the schema image corresponding to the part to be imaged. Whether to display another guide image generated based on the selection is selected. The other guide image is an image that guides the display position of the corresponding part in the screen. In particular, when the age of the patient is low, a guide image is generated using a schema image when a certain period has passed since the previous imaging. The reason is as follows.

  Body growth is significant in younger patients. For this reason, when a predetermined period has elapsed since the previous acquisition of the affected part image, if the guide image is generated using the affected part image before the predetermined period, the guide image is likely to be smaller than the patient's body. In this case, if a part including the affected part is imaged in accordance with the guide image, the size of the affected part cannot be properly compared, and it is not easy to appropriately grasp the progress of the affected part from the captured affected part image. For this reason, for a relatively young patient, a guide image created based on a schema image is periodically used to facilitate observation of the progress. The image processing unit 152 may change the size of the guide image to a size corresponding to the patient's age or age based on the information on the growth curve of the infant and display the information.

Next, a processing procedure for the affected area observation support will be described. The affected area observation support process is executed when the affected area observation button 503 is selected on the menu screen 500.
FIG. 21 is a flowchart illustrating a processing example of the affected area observation support.

  (S40) The UI unit 130 displays the follow-up observation screen 580 on the display 105. On the follow-up observation screen 580, the affected part images of the designated affected part are displayed in time series.

  (S41) The UI unit 130 refers to the affected part table 123 and determines whether or not the treatment period from the onset date of the affected part is longer than the standard treatment period of the relevant disease name registered in the disease name master 121. To do. If the treatment period is longer than the standard treatment period, the process proceeds to step S42. If the treatment period is equal to or shorter than the standard treatment period, the process proceeds to step S43.

(S42) The UI unit 130 causes the display 105 to display a consultation recommendation message 590 that recommends consultation with an expert.
(S43) The UI unit 130 determines whether there is an input to the consultation button 585. If there is an input to the consultation button 585, the process proceeds to step S47. If there is no input to the consultation button 585, the process proceeds to step S44.

  (S44) The UI unit 130 determines whether or not there is an input to the healing button 586. If there is an input to the healing button 586, the process proceeds to step S45. If there is no input to the healing button 586, the process proceeds to step S46.

(S45) The UI unit 130 changes the in-treatment flag of the affected part in the affected part table 123 from “TRUE” to “FALSE”. Then, the affected area observation support ends.
(S46) The UI unit 130 determines whether there is an input to the completion button 587. When there is an input to the completion button 587, the affected area observation support ends. If there is no input to the completion button 587, the process proceeds to step S43.

(S47) The UI unit 130 displays a transmission confirmation screen 600 on the display 105.
(S48) The UI unit 130 determines whether or not to transmit the affected part image displayed on the transmission confirmation screen 600 to the server 200. In the case of transmission, the process proceeds to step S49. If not transmitted, the affected area observation support ends. Here, when the cancel button 601 is selected on the transmission confirmation screen 600, the UI unit 130 determines not to transmit the affected part image to the server 200. On the other hand, when the transmission button 602 is selected on the transmission confirmation screen 600, the UI unit 130 determines to transmit the affected part image to the server 200.

(S49) The communication processing unit 140 transmits the affected part image displayed on the transmission confirmation screen 600 to the server 200.
(S50) The communication processing unit 140 receives expert advice information from the server 200 for the diseased part image transmitted in step S49. Note that a relatively long time (for example, about several hours to several days) may be required between step S49 and step S50. As described above, the communication processing unit 140 acquires the advice from the server 200 in response to an input for obtaining advice by the patient after receiving a notification from the server 200 that the advice has been received from the terminal device 300.

  (S51) The UI unit 130 causes the display 105 to display an advice display screen 610 including the advice received in step S50. In addition, when the re-registration button 611 on the advice display screen 610 is selected, the UI unit 130 displays the re-registration screen 620 on the display 105 and accepts re-registration of disease names and drug names on the re-registration screen 620. Then, the affected area observation support ends.

  As described above, the region to be imaged includes the affected part to be observed. Then, the UI unit 130 stores the affected part image in the storage unit 120 in association with the captured time. The UI unit 130 stores, in time series, a plurality of images stored in the storage unit 120 corresponding to the part, including a plurality of images of the affected part captured this time and a part of the affected part image captured in the past. It is displayed on the observation screen 580. As a result, the patient can easily follow the affected area.

Next, a processing procedure for providing transparency by the server 200 will be described.
FIG. 22 is a flowchart illustrating an example of processing for providing transparency.
(S60) The communication processing unit 230 receives information about the current imaging target part from the terminal device 100.

  (S61) Based on the transparency table 221 indicating the past results of the transparency of each part used by the user, the transparency calculation unit 240 transmits the transparency of the imaging target part received in step S60 (recommendation of transparency). Value). For example, it is conceivable that the transparency calculation unit 240 calculates a statistical value such as an average value or a percentile value of past transparency used for the current imaging target part as a recommended value of transparency.

(S62) The communication processing unit 230 transmits the transparency (recommended value of transparency) calculated by the transparency calculation unit 240 to the terminal device 100.
(S63) The communication processing unit 230 receives the information of the part imaged this time and the information of the current transparency from the terminal device 100.

  (S64) The communication processing unit 230 stores the current part information and transparency information received in step S63. Specifically, the communication processing unit 230 records the current part information and the transparency information in the transparency table 221. The recorded information is used for the next and subsequent transmission calculation. Then, the provision of transparency ends.

  Thus, the guide image is an image obtained by changing the transparency of the affected part image captured in the past. The display control unit 150 acquires a recommended value of transparency calculated based on a plurality of transmissivities used in the past for the region to be imaged, and sets the transparency of the guide image to the recommended value.

  Here, there are various parts including the affected part in the body, and if the transmittance is the same for each part, it becomes easier to see the background camera image in one part, but it is difficult to see the background camera image in another part. There is. Moreover, although the transmittance | permeability change part 552,572 can be provided and the change of the transparency by a patient can also be received, the operation of the said change is also troublesome. Therefore, the terminal apparatus 100 acquires a recommended value of transparency according to the part from the server 200. In particular, the terminal device 100 acquires a recommended value of transmittance calculated based on the results of transmittance for each part by a patient (a plurality of patients). This makes it easier to see the camera image when the guide image is displayed over the camera image for the corresponding part. As a result, the patient can easily match the camera image to the guide image, and can easily acquire the affected part image that matches the imaging conditions.

  Note that the terminal device 100 may have the function of the transmittance calculation unit 240. In this case, the terminal device 100 receives the transparency table 221 from the server 200, and calculates a recommended value of transparency according to the current imaging target region by the same method as in step S61. The terminal device 100 uses the calculated recommended value as the transparency of the guide image.

  According to the terminal device 100 of the second embodiment, when a patient (user) observes the progress of an affected part at a specific part of his / her body, it is possible to easily acquire an image with uniform imaging conditions. . For example, the user adjusts the position of the camera 104 (or the terminal device 100) so that the guide image and the image of the part included in the camera image are combined. When the guide image and the image of the part included in the camera image are combined, the patient inputs an instruction to save the camera image at that time to the terminal device 100. Then, the terminal device 100 preserve | saves a camera image as an affected part image. Thus, the patient can cause the terminal device 100 to acquire the affected area image of the current affected area under the same conditions as the affected area image captured in the past. For this reason, the patient appropriately grasps the state of the symptom such as the development / reduction of the affected area and the degree of healing at the past time point and the current time point by comparing the past affected part image with the current affected part image. It becomes possible.

  The information processing according to the first embodiment can be realized by causing the processing unit 12 to execute a program. The information processing according to the second embodiment can be realized by causing the CPU 101 to execute a program. The program can be recorded on a computer-readable recording medium 111.

  For example, the program can be distributed by distributing the recording medium 111 on which the program is recorded. Alternatively, the program may be stored in another computer and distributed via a network. For example, the computer stores (installs) a program recorded in the recording medium 111 or a program received from another computer in a storage device such as the RAM 102 or the flash memory 103, and reads and executes the program from the storage device. May be.

DESCRIPTION OF SYMBOLS 10 Information processing apparatus 11 Memory | storage part 12 Processing part 13 Imaging device 14 Display apparatus 20 Hand 21 Finger 22 Affected part 30,41 Captured image 40 Screen 42 Auxiliary image

Claims (8)

When receiving an instruction to start imaging a predetermined part of the user's body, the first image captured by the imaging device is displayed on the screen of the display device,
Based on the second image of the part of the user captured in the past, an auxiliary image indicating the display position of the part on the screen when the second image is captured in the past is the first image. To be displayed superimposed on
An imaging support program that causes a computer to execute processing. The auxiliary image is an image obtained by changing the transparency of the second image,
The computer further acquires a second transmittance calculated based on a plurality of first transmittances used in the past with respect to the part, and sets the auxiliary image to the second transmittance. ,
The imaging support program according to claim 1, wherein processing is executed. The computer further causes the auxiliary image to be displayed based on the age of the user, or another auxiliary image generated based on a schema image corresponding to the part, wherein the part in the screen is displayed. Select whether to display the other auxiliary image that guides the display position.
The imaging support program according to claim 1, wherein processing is executed. If the second image has not been acquired, the computer further includes:
A model image representing each part of the body is displayed on the screen,
Accepting selection of a part corresponding to the part included in the model image;
Displaying a third image captured by the imaging device on the screen;
Another auxiliary image that guides the display position of the part in the screen based on the schema image of the part is superimposed on the third image and displayed.
The imaging support program according to claim 1, wherein processing is executed. The site includes the affected area of follow-up,
The computer further stores the first image in a storage unit in association with the time when the first image was captured, and is a plurality of images corresponding to the part stored in the storage unit, the first image and Displaying the plurality of images including the second image on the screen in time series;
The imaging support program according to claim 1, wherein processing is executed. The auxiliary image indicates the orientation of the part with respect to the imaging device when the second image has been captured in the past.
The imaging support program according to any one of claims 1 to 5. A storage unit that stores a first image of a predetermined part of the user's body imaged in the past;
When receiving an instruction to start imaging of the part of the user, the second image captured by the imaging device is displayed on the screen of a display device, and based on the first image stored in the storage unit, A processing unit that superimposes and displays an auxiliary image indicating the display position of the part on the screen when the first image is captured in the past on the second image;
An information processing apparatus. Computer
When receiving an instruction to start imaging a predetermined part of the user's body, the first image captured by the imaging device is displayed on the screen of the display device,
Based on the second image of the part of the user captured in the past, an auxiliary image indicating the display position of the part on the screen when the second image is captured in the past is the first image. To be displayed superimposed on
Imaging support method.

Images