JP2020151461A - Imaging apparatus, information processing apparatus, and information processing system - Google Patents

Abstract

To capture an image for facilitating comparison of an affected part.SOLUTION: An imaging apparatus of this invention comprises: imaging means; and control means which performs control to acquire posture information of a subject obtained when imaging an affected part of the subject in the past, and to inform a user with the posture information of the subject when imaging the affected part of the subject by the imaging means. The affected part is a bedsore, and the posture includes information capable of identifying one of lying with one's face down, lying on one's side, and sitting.SELECTED DRAWING: Figure 5

Description

The present invention relates to an image pickup apparatus, an information processing apparatus, an information processing system, and a control method.

When a person or animal is lying down, pressure sores, so-called bedsores, may occur when the contact patch between the contact patch and the body is pressed by the body weight. Patients who develop pressure ulcers need to receive pressure ulcer care such as body pressure distribution care and skin care, and evaluate and manage pressure ulcers on a regular basis.

Non-Patent Document 1 proposes DESIGN-R (registered trademark), a pressure ulcer condition determination scale developed by the Academic Board of Education of the Japanese Society of Pressure Ulcers, as a tool for evaluating pressure ulcers. DESIGN-R is a tool for evaluating the healing process of wounds such as pressure ulcers. The name of this scale is for each observation item of depth, exudate, size, inflammation / infection, granulation tissue, and necrotic tissue. It is an acronym.

There are two types of DESIGN-R, one for severity classification for simple daily evaluation and the other for progress evaluation that shows the flow of the healing process in detail. DESIGN-R for severity classification divides six evaluation items into two categories, mild and severe, with mild being represented using lowercase letters and severe being represented using uppercase letters.
The general condition of pressure ulcers can be grasped by evaluating using a severity classification device at the time of initial treatment. Since it is possible to know which item is the problem, the treatment policy can be easily decided.

On the other hand, DESIGN-R, which can compare the severity between patients in addition to the progress evaluation, is also defined for the progress evaluation. R represents ratting (evaluation / rating). Each item is weighted differently, and the total score (0 to 66 points) of the 6 items other than the depth indicates the severity of the pressure ulcer. The course of treatment can be evaluated in detail and objectively after the start of treatment, and not only individual course evaluation but also severity comparison between patients can be performed.
Here, the size evaluation of DESIGN-R measures the major axis and the minor axis (maximum diameter orthogonal to the major axis) of the skin damage range (cm), and classifies the size, which is a numerical value obtained by multiplying each of them, into seven stages. Is. These 7 stages are s0: no skin damage, s3: 4 or less, s6: 4 or more and less than 16, s8: 16 or more and less than 36, s9: 36 or more and less than 64, s12: 64 or more and less than 100, S15: 100 or more, Is.

As described in Non-Patent Document 1, it is recommended that DESIGN-R be scored once every 1 to 2 weeks in order to evaluate the healing process of pressure ulcers and make appropriate care choices. , Pressure ulcers need to be evaluated and managed on a regular basis. In addition, accuracy is required for evaluation in order to confirm changes in the pathological condition of pressure ulcers.

Shorinsha Pressure Ulcer Guidebook 2nd Edition Pressure Ulcer Prevention and Management Guidelines (4th Edition) Compliant Editing Japan Pressure Ulcer Society ISBN13 978-4796523608 Page 23

However, even when the pressure ulcer is photographed, the appearance of the pressure ulcer may change each time the pressure ulcer is photographed because the shape, area, and pocket shape of the pressure ulcer change depending on the posture of the patient. Therefore, even if the images of pressure ulcers are compared, it may be difficult to accurately compare the progress. This is not limited to pressure ulcers, and is the same even when photographing burns and lacerations.
The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to enable an image to be taken for facilitating comparison of affected areas.

The imaging device of the present invention acquires the posture information of the subject when the affected part of the subject is photographed in the past with the imaging means, and when the imaged means captures the affected part of the subject, the posture information of the subject is obtained. It is characterized by having a control means for controlling the user to be notified.

According to the present invention, it becomes possible to take an image for facilitating comparison of affected areas.

It is a figure which shows the functional structure of an information processing system. It is a figure which shows the subject. It is a figure which shows the hardware configuration of the image pickup apparatus. It is a figure which shows the hardware configuration of an information processing apparatus. It is a flowchart which shows the processing of an information processing system. It is a figure for demonstrating the calculation method of the area of the affected part area. It is a figure for demonstrating the method of superimposing information on the image data of an affected part. It is a figure for demonstrating the method of superimposing information on the image data of an affected part. It is a figure which shows the data structure of subject information. It is a figure which shows the image data including the posture information.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings.
<First Embodiment>
FIG. 1 is a diagram showing an example of a functional configuration of the information processing system 1.
The information processing system 1 includes an image pickup device 200, which is a portable device that can be held by hand, and an information processing device 300.

FIG. 2 is a diagram showing an example of a subject 101, which is a patient whose affected portion is evaluated by the information processing system 1. In the present embodiment, an example of the pathological condition of the affected part 102 that occurs in the buttocks of the subject 101 will be described as a pressure ulcer that occurs in the buttocks.
A barcode tag 103 is attached to the subject 101. The barcode tag 103 includes a patient ID as identification information for identifying the subject. Therefore, the information processing system 1 can manage the identification information of the subject 101 in association with the image data obtained by photographing the affected portion 102. The identification information is not limited to the barcode tag 103, but may be a two-dimensional code such as a QR code (registered trademark) or a numerical value, and may be data or an ID number attached to an ID card such as a medical examination card. May be good.

In the information processing system 1, the image pickup apparatus 200 photographs the affected portion 102 of the subject 101 and the barcode tag 103, which is identification information, and transmits the images to the information processing apparatus 300. The information processing device 300 transmits the posture information of the subject 101 when the affected portion 102 of the same subject 101 is photographed in the past as the posture information associated with the received identification information to the imaging device 200. By displaying the image based on the received posture information, the image pickup device 200 can grasp the posture of the subject 101 when the affected portion 102 of the same subject 101 is photographed in the past. The posture information may include at least information that can identify the posture of the subject in any one of lying down, lying down (lower right lying down or lower left lying down), and sitting position. Further, in the present embodiment, the case where the affected part 102 is a pressure ulcer will be described as an example, but the present embodiment is not limited to the pressure ulcer, and may be a burn or a laceration.

FIG. 3 is a diagram showing an example of the hardware configuration of the image pickup apparatus 200.
As the image pickup device 200, a general single-lens camera, a compact digital camera, a smartphone or a tablet terminal provided with a camera having an autofocus function, or the like can be used.
The image pickup unit 211 has a lens group 212, a shutter 213, and an image sensor 214. By changing the positions of a plurality of lenses included in the lens group 212, the focus position and the zoom magnification can be changed. The lens group 212 also includes an aperture for adjusting the amount of exposure.

The image sensor 214 is composed of a charge storage type solid-state image sensor such as a CCD or CMOS sensor that converts an optical image into electrical data. The reflected light from the subject that has passed through the lens group 212 and the shutter 213 is imaged on the image sensor 214. The image sensor 214 generates an electric signal according to the subject image, and outputs image data based on the generated electric signal.

The shutter 213 opens and closes the blade member to expose or shield the image sensor 214 from light, and controls the exposure time of the image sensor 214. The shutter 213 may be an electronic shutter whose exposure time is controlled by driving the image sensor 214. When the electronic shutter is performed by the CMOS sensor, a reset scan is performed to make the accumulated charge amount of the pixels zero for each pixel or for each region consisting of a plurality of pixels (for example, for each line). After that, for each pixel or region for which reset scanning has been performed, scanning is performed to read out a signal according to the amount of accumulated charge after a predetermined time has elapsed.
The zoom control circuit 215 controls a motor for driving the zoom lens included in the lens group 212, and controls the optical magnification of the lens group 212.

The distance measuring system 216 calculates the distance information to the subject. The distance measuring system 216 may generate distance information based on the output of the AF control circuit 218. Further, when there are a plurality of areas to be AFed on the screen, the distance measuring system 216 generates distance information for each area by repeatedly operating the AF control circuit 218 for each area. May be good. The distance measuring system 216 may use a TOF (Time Of Flight) sensor. The TOF sensor is a sensor that measures the distance to the object based on the time difference (or phase difference) between the transmission timing of the irradiation wave and the reception timing of the reflected wave reflected by the object. Further, the ranging system 216 may use a PSD method or the like using a PSD (Position Sensitive Device) as the light receiving element.

The image processing circuit 217 performs predetermined image processing on the image data output from the image sensor 214. The image processing circuit 217 has various image data such as white balance adjustment, gamma correction, color interpolation or demosaiking, and filtering for the image data output from the image pickup unit 211 or the image data stored in the internal memory 221. Perform image processing. Further, the image processing circuit 217 performs compression processing on the image data that has undergone image processing according to a standard such as JPEG.

The AF control circuit 218 determines the position of the focus lens included in the lens group 212 based on the distance information obtained by the distance measuring system 216, and controls the motor that drives the focus lens. The AF control circuit 218 may perform TV-AF or contrast AF that extracts and integrates high-frequency components of image data and determines the position of the focus lens that maximizes the integrated value. The focus control method is not limited to contrast AF, and may be phase difference AF or other AF method. Further, the AF control circuit 218 may detect the amount of focus adjustment or the position of the focus lens, and acquire distance information to the subject based on the position of the focus lens.

The communication device 219 is a communication interface for communicating with an external device such as the information processing device 300 via a wireless network. A specific example of a network is a network based on the Wi-Fi (registered trademark) standard. Communication using Wi-Fi may be realized via a router. Further, the communication device 219 may be realized by a wired communication interface such as USB or LAN.

The system control circuit 220 has a CPU (Central Processing Unit) and controls the entire image pickup apparatus 200 by executing a program stored in the internal memory 221. Further, the system control circuit 220 controls the image pickup unit 211, the zoom control circuit 215, the distance measurement system 216, the image processing circuit 217, the AF control circuit 218, and the like. The system control circuit 220 is not limited to having a CPU, and an FPGA, an ASIC, or the like may be used.

As the internal memory 221, for example, a rewritable memory such as a flash memory or SDRAM can be used. The internal memory 221 temporarily stores various setting information such as focus position information at the time of image capture necessary for the operation of the image pickup apparatus 200, image data captured by the image pickup unit 211, image data processed by the image processing circuit 217, and the like. Memorize. Further, the internal memory 221 may temporarily store analysis data such as image data and information on the size of the subject received by the communication device 219 communicating with the information processing device 300.

The external memory 222 is a non-volatile recording medium that can be attached to the image pickup device 200 or is built in the image pickup device 200. As the external memory 222, for example, an SD card, a CF card, or the like can be used. The external memory 222 records image data image-processed by the image processing circuit 217, image data received by the communication device 219 communicating with the information processing device 300, analysis data, and the like. Further, the external memory 222 can read the recorded image data at the time of reproduction and output it to the outside of the image pickup apparatus 200.

As the display device 223, for example, a TFT (Thin Film Transistor) liquid crystal display, an organic EL display, an EVF (electronic viewfinder), or the like can be used. The display device 223 displays image data temporarily stored in the internal memory 221 and image data recorded in the external memory 222, and displays a setting screen of the image pickup device 200 and the like.

The operation unit 224 is composed of a button, a switch, a key, a mode dial provided in the image pickup device 200, a touch panel that is also used as the display device 223, and the like. Commands such as mode setting and shooting instruction by the user are notified to the system control circuit 220 via the operation unit 224.
The tilt detection device 225 detects the tilt of the image pickup device 200. In the present embodiment, the inclination of the image pickup apparatus 200 refers to an angle with reference to the horizontal. As the tilt detection device 225, for example, a gyro sensor, an acceleration sensor, or the like can be used.
The common bus 226 is a signal line for transmitting and receiving signals between each component of the image pickup apparatus 200.

FIG. 4 is a diagram showing an example of the hardware configuration of the information processing apparatus 300.
The information processing device 300 includes a CPU 310, a storage device 312, a communication device 313, an output device 314, an auxiliary arithmetic unit 317, and the like.
The CPU 310 includes an arithmetic unit 311. The CPU 310 controls the entire information processing device 300 by executing the program stored in the storage device 312, and realizes the functional configuration of the information processing device 300 shown in FIG.

The storage device 312 includes a main storage device 315 (ROM, RAM, etc.) and an auxiliary storage device 316 (magnetic disk device, SSD (Solid State Drive), etc.).
The communication device 313 is a wireless communication module for communicating with an external device such as an image pickup device 200 via a wireless network.
The output device 314 outputs the data processed by the arithmetic unit 311 and the data stored in the storage device 312 to a display, a printer, or an external network connected to the information processing device 300.

The auxiliary arithmetic unit 317 is an auxiliary arithmetic IC that operates under the control of the CPU 310. As the auxiliary arithmetic unit 317, a GPU (Graphic Processing Unit) or the like can be used. The GPU is originally a processor for image processing, but since it has a plurality of product-sum arithmetic units and is good at matrix calculation, it can also be used as a processor that performs processing for signal learning. Therefore, the GPU is generally used in the process of performing deep learning. For the auxiliary arithmetic unit 317, for example, a Jetson TX2 module manufactured by NVIDIA can be used. Further, FPGA, ASIC, or the like may be used as the auxiliary arithmetic unit 317. The auxiliary arithmetic unit 317 extracts the affected area from the image data.

The CPU 310 and the storage device 312 included in the information processing device 300 may be one or a plurality. That is, when at least one or more CPUs and at least one storage device are connected and at least one or more CPUs execute a program stored in at least one storage device, the information processing device 300 will be described later. Perform each function. The CPU is not limited to the CPU, and may be an FPGA, an ASIC, or the like.

FIG. 5 is a flowchart showing an example of processing of the information processing system 1.
In FIG. 5, S501 to S519 are processes by the image pickup apparatus 200, and S521 to S550 are processes by the information processing apparatus 300. The flowchart of FIG. 5 is started by connecting the image pickup device 200 and the information processing device 300 to a Wi-Fi standard network, which is a wireless LAN standard, respectively.

In S521, the CPU 310 of the information processing device 300 performs the search process of the connected image pickup device 200 via the communication device 313.
In S501, the system control circuit 220 of the image pickup apparatus 200 performs a response process to the search process by the information processing apparatus 300 via the communication device 219. UPnP (Universal Plug and Play) is used as a technique for searching for devices via a network. Here, in UPnP, the identification of individual devices is performed by a UUID (Universally Unique Identifier).

In S502, the system control circuit 220 uses the display device 223 to guide the user to take an overall posture capable of grasping the posture of the subject when taking a picture of the affected area and a bar code tag for identifying the subject. put out. The imaging unit 211 captures the posture of the subject and the barcode tag of the subject, respectively, in response to a shooting instruction by the user.
Here, before photographing the affected part of the subject, the subject is asked to be in a prone, lying down or sitting posture, for example, and the whole posture is photographed so that the posture of the subject when photographing the affected part can be grasped. At this time, the system control circuit 220 generates tilt information of the image pickup device 200 when the posture is photographed based on the tilt information output from the tilt detection device 225.

Next, the live view processing from S503 to S511 will be described.
In S503, the AF control circuit 218 performs AF processing that controls the drive of the lens group 212 so that the subject is in focus.
Here, since it is assumed that the user holds the image pickup apparatus 200 so that the affected portion is located in the center of the screen, the AF control circuit 218 performs AF processing in the area located in the center of the screen. Further, the AF control circuit 218 outputs distance information to the subject based on the amount of focus adjustment or the amount of movement of the focus lens.

In S504, the system control circuit 220 uses the display device 223 to guide the user to take a picture of the affected part of the subject. The image pickup unit 211 shoots a subject in response to a shooting instruction by the user.
In S505, the image processing circuit 217 acquires the captured image data, develops and compresses it, and generates, for example, JPEG standard image data. The image processing circuit 217 resizes the compressed image data to reduce the size of the image data.

The image pickup apparatus 200 transmits the resized image data in S508, which will be described later, by wireless communication. Since the larger the size of the image data to be transmitted, the longer the wireless communication takes. Therefore, in S505, the system control circuit 220 determines the size of the image data to be resized based on the allowable communication time, and the image processing circuit 217. Give instructions to.
In S532, which will be described later, the information processing apparatus 300 extracts the affected area from the image data that has been resized. Since the size of the image data affects the time and accuracy of extracting the affected area, in S505, the system control circuit 220 determines the size of the image data to be resized based on the time and accuracy of extraction.

Further, since the resizing process in S505 is a process in the live view, the frame rate of the live view image becomes slow if the processing time is long. Therefore, in S505, it is preferable that the system control circuit 220 is resized to a smaller size or the same size than the resizing process in S514 described later, which is not the process during live view.
In the present embodiment, the image size is resized to be approximately 1.1 megabytes as an 8-bit RGB color with 720 pixels × 540 pixels. However, the size of the image data to be resized is not limited to this case.

In S506, the system control circuit 220 generates distance information to the subject. Specifically, the system control circuit 220 generates distance information from the image pickup apparatus 200 to the subject based on the distance information output by the distance measuring system 216. When the AF control circuit 218 performs AF processing on each of a plurality of areas in the screen in S503, the system control circuit 220 may generate distance information for each of the plurality of areas. Further, as a method of generating the distance information, the distance information to the subject calculated by the distance measuring system 216 may be used.

In S507, the system control circuit 220 generates tilt information of the image pickup device 200 in the live view based on the tilt information output from the tilt detection device 225.
Here, since it is assumed that the user holds the image pickup device 200 so as to include the affected area in the imaging range, the system control circuit 220 provides tilt information of the image pickup device 200 when the user holds the image pickup device 200 toward the affected area. To generate.

In S508, the system control circuit 220 transmits various information to the information processing device 300 via the communication device 219. Specifically, the system control circuit 220 transmits the image data of the affected area resized in S505, the distance information to the subject generated in S506, and the tilt information of the imaging device 200 in the live view generated in S507. Further, the system control circuit 220 transmits the image data of the posture photographed in S502, the tilt information of the image pickup device 200 when the attitude is photographed, and the image data of the barcode tag to the information processing device 300. Since the patient ID included in the image data of the barcode tag is not information that changes, the image data of the barcode tag is transmitted only once for the same patient. In addition, the posture image data and the tilt information of the imaging device 200 when the posture is photographed are also transmitted only once for the same patient.

Next, the process proceeds to the processing by the information processing device 300.
In S531, the CPU 310 of the information processing device 300 receives the image data of the affected area, the distance information to the subject, and the tilt information of the image pickup device 200 in the live view transmitted by the image pickup device 200 via the communication device 313. Further, the CPU 310 receives the posture image data, the tilt information of the imaging device 200 when the posture is photographed, and the image data of the barcode tag only once for the same patient.

In S532, the CPU 310 uses the auxiliary arithmetic unit 317 to extract the affected area from the received image data of the affected area (divide the affected area from the other area). As a method of region division, semantic region division by deep learning is performed. That is, a learning computer is trained in advance using a plurality of images of the affected area of the pressure ulcer as teacher data to train a neural network model, and a trained model is generated. The auxiliary arithmetic unit 317 acquires the trained model from the computer and estimates the pressure ulcer area from the image data based on the trained model. As an example of a neural network model, a complete convolutional network (FCN (Full Convolutional Network)), which is a segmentation model using deep learning, can be applied. The inference of deep learning is processed by the GPU included in the auxiliary arithmetic unit 317, which is good at executing the product-sum operation in parallel. However, the inference of deep learning may be executed by FPGA, ASIC, or the like. It should be noted that the region division may be realized by using another deep learning model. Further, the segmentation method is not limited to deep learning, and for example, graph cut, region growth, edge detection, governing division method, and the like may be used. Further, the model of the neural network may be trained using the image of the affected area of the pressure ulcer as the teacher data inside the auxiliary arithmetic unit 317.

In S533, the arithmetic unit 311 of the CPU 310 calculates the area of the affected area as information regarding the size of the extracted affected area. The arithmetic unit 311 converts the size of the extracted affected area on the image data based on the information on the angle of view or the pixel size of the image data and the distance information generated by the system control circuit 220, thereby converting the affected area. Calculate the area of.

FIG. 6 is a diagram for explaining a method of calculating the area of the affected area.
When the image pickup apparatus 200 is a general camera, it can be treated as a pinhole model as shown in FIG. The incident light 601 passes through the principal point of the lens 212a and is received by the imaging surface of the image sensor 214. The distance from the imaging surface to the principal point of the lens is the focal length F602. Here, when the lens group 212 is approximated to a single lens 212a having no thickness, it can be considered that the two principal points, the front principal point and the posterior principal point, coincide with each other. By adjusting the focus position of the lens 212a so that the image is formed on the plane of the image sensor 214, the image pickup apparatus 200 can focus on the subject 604. By changing the focal length F602, which is the distance from the imaging surface to the principal point of the lens, the angle of view θ603 is changed and the zoom magnification is changed. At this time, the width W606 of the subject on the focal plane is geometrically determined from the relationship between the angle of view θ603 of the image pickup apparatus 200 and the subject distance D605. The width W606 of the subject is calculated using trigonometric functions. That is, the width W606 of the subject is determined by the relationship between the angle of view θ603 that changes according to the focal length F602 and the subject distance D605. By dividing the value of the width W606 of the subject by the number of pixels on the line of the corresponding image sensor 214, the length on the focal plane corresponding to one pixel on the image is acquired.

The arithmetic unit 311 determines that the product of the number of pixels in the region obtained from the result of region division in S532 and the area of one pixel obtained from the length on the focal plane corresponding to one pixel on the image is the product of the affected area. Calculate the area. In the formula for obtaining the width W606 of the subject or the length on the focal plane corresponding to one pixel on the image, data is obtained by shooting a subject having a known width W606 of the subject by changing the subject distance D605. It may be obtained recursively by acquiring it.
When the subject distance D605 is single, in order for the arithmetic unit 311 to correctly obtain the area of the affected area, it is premised that the subject 604 is a plane and this plane is perpendicular to the optical axis. .. However, when the distance information is generated for each of a plurality of areas in S506, the arithmetic unit 311 detects the inclination or change of the subject in the depth direction, and calculates the area of the affected area based on the detected inclination or change. You may.

In S534, the image processing circuit 217 generates image data in which information indicating the extraction result of the affected area and information regarding the size of the affected area are superimposed on the image data for which the affected area is to be extracted.
7 (a) and 7 (b) are diagrams for explaining a method of superimposing the information showing the extraction result of the affected area and the information on the size of the affected area on the image data.
The image 701 shown in FIG. 7A is an example of displaying the image data before the superimposition processing, and includes the subject 101 and the affected portion 102. The image 702 shown in FIG. 7B is an example of displaying the image data after the superimposition processing.

In the upper left corner of the image 702 shown in FIG. 7B, a label 711 displaying a character string 712 of the area of the affected area in white characters is superimposed on a black background. Here, the information regarding the size of the affected area is the character string 712, which is the area of the affected area calculated by the arithmetic unit 311. The background color of the label 711 and the color of the character string are not limited to black and white as long as they are easy to see. Further, by setting the transmission amount and α-blending, the user may be able to confirm the image of the portion where the label 711 overlaps.

Further, the index 713 indicating the estimated area of the affected area extracted in S532 is superimposed on the image 702. By α-blending the index 713 indicating the estimated area and the image data that is the source of the image 701, the user can confirm whether or not the estimated area that is the source for calculating the area of the affected area is appropriate. The color of the index 713 indicating the estimated area is preferably a color different from the color of the subject. Further, the range of the transmittance of the α blend is preferably a range in which the estimated area and the original affected area 102 can be distinguished. If the index 713 indicating the estimated area of the affected area is superimposed and displayed, the user can confirm whether or not the estimated area is appropriate without displaying the label 711, so S533 is omitted. You may.

In S535, the CPU 310 reads the patient ID from the image data of the barcode tag. In S536, the CPU 310 collates the read patient ID with the patient ID of the subject registered in advance in the storage device 312, and acquires information on the name of the subject.

In S537, the CPU 310 associates the image data of the affected area with the information of the patient ID and the name of the subject, and stores the information in the storage device 312. The CPU 310 processes the image data of the affected area received in S531 as information of the same patient ID and the same subject name until the next image data of the captured barcode tag is received.
Further, the CPU 310 determines whether or not the subject information corresponding to the target patient ID is stored in the storage device 312. When the subject information corresponding to the target patient ID is not stored, the CPU 310 generates the subject information corresponding to the information of the patient ID and the name of the subject. On the other hand, if the subject information corresponding to the target patient ID is already stored in the storage device 312, the process proceeds to S538.

FIG. 9A is a diagram showing an example of the data structure of the subject information 900. The subject information 900 is managed for each patient ID.
The subject information 900 includes a patient ID field 901, a subject name field 902, a posture information 903, and an affected area information 908.
The patient ID is stored in the patient ID column 901. The name of the subject is stored in the subject name field 902.
The posture information 903 includes a posture icon field 904, a posture image data field 905, a first tilt information field 906, and a second tilt information field 907. In the posture icon column 904, a posture icon schematically showing the posture of the subject when photographing the affected area, or identification information of the posture icon is stored. The posture icon corresponds to an example of a display item.

FIG. 9B is a diagram showing an example of a posture icon.
The posture icon 921 is an icon indicating a prone posture. The posture icon 922 is an icon indicating the posture of the lower right lying down with the right side facing down. The posture icon 923 is an icon indicating the posture of the lower left lying down with the left side facing down. The posture icon 924 is an icon indicating a sitting posture.

In the posture image data field 905, the posture image data obtained by capturing the posture of the subject in S502 or the address information in which the posture image data is stored is stored.
In the first tilt information column 906, tilt information of the image pickup apparatus 200 when the posture is photographed in S502 is stored. In the second tilt information column 907, tilt information of the imaging device 200 in recording imaging in which the live view is finished and the affected portion is imaged for recording is stored. In the second tilt information column 907, the tilt information of the imaging device 200 when the first or last recording image is taken in the target patient ID, or the inclination of the image pickup device 200 when the recording image is taken a plurality of times. The average value of the information is stored. The tilt information in the second tilt information column 907 is stored or updated based on the tilt information of the image pickup apparatus 200 in the recording photographing, which is stored in the tilt information column 912 described later. By referring to the tilt information stored in the second tilt information column 907 when the user takes a picture for recording, the image pickup device 200 can be used to face the surface of the affected portion.
Note that the posture information 903 stores information that can identify the posture of the subject, such as character information in which the posture of the subject is represented by characters such as "downside down", "sitting position", "lower right lying down", and "lower left lying down". You may.

The affected area information 908 includes a photographing date and time column 909, an image data column 910 of the affected area, an evaluation information column 911, and a tilt information column 912. In the shooting date / time column 909, the date and time when the recording was taken in S513, which will be described later, is stored. In the image data column 910 of the affected area, the image data of the affected area taken for recording or the address information in which the image data of the affected area is stored is stored. In the evaluation information column 911, information indicating the evaluation result of the affected area is stored. In the tilt information column 912, tilt information of the image pickup apparatus 200 in recording photography is stored.

When the subject information 900 corresponding to the target patient ID is not stored in S537, the CPU 310 includes the posture icon column 904, the posture image data column 905, and the first tilt information among the posture information 903 of the generated subject information 900. Information is added to column 906 and stored in the storage device 312. Specifically, in order for the CPU 310 to add to the posture icon field 904, first, the posture of the subject is the posture icons 921 to 924 shown in FIG. 9B based on the image data of the posture received by the auxiliary arithmetic unit 317 in S531. It is determined which of the above corresponds to. Next, the CPU 310 stores the posture icon or the identification information of the posture icon in the posture icon column 904. Further, the CPU 310 stores the posture image data received in S531 in the posture image data field 905. Further, the CPU 310 stores the tilt information of the image pickup apparatus 200 when the posture received in S531 is photographed in the first tilt information column 906.
On the other hand, when the subject information 900 corresponding to the target patient ID is stored in S537, the affected area has been photographed in the past, and each information is already stored in the posture information 903 and the affected area information 908 of the subject information 900. Since there is, proceed to S538.

In S538, the CPU 310 of the information processing device 300 transmits information indicating the extraction result of the affected area and information regarding the size of the affected area to the imaging device 200 via the communication device 313. In the present embodiment, the CPU 310 transmits to the image pickup apparatus 200 the image data in which the image data of the affected area generated in S534 is superposed with the information indicating the extraction result of the affected area and the information on the size of the affected area.
Further, the CPU 310 transmits the posture information 903 of the subject information 900 to the image pickup device 200 via the communication device 313 in order to notify the user of the posture of the subject when the affected portion is photographed in the past. Specifically, the CPU 310 transmits a posture icon, posture image data, tilt information of the imaging device 200 when the posture is photographed, and tilt information of the imaging device 200 in recording imaging. When the CPU 310 transmits image data in which information indicating the extraction result of the affected area and information on the size of the affected area are superimposed on the image data of the affected area a plurality of times during the live view, the posture information is transmitted. Send 903 only the first time. The CPU 310 may transmit the tilt information of the image pickup apparatus 200 in the live view received in S531. Further, if the subject information 900 corresponding to the target patient ID is not stored in S537 because the recording image has not been taken in the past, the information is not stored in the second tilt information column 907. Therefore, the tilt information of the image pickup apparatus 200 in the recording photographing is not transmitted.

Next, the process of the image pickup apparatus 200 is started.
In S509, the system control circuit 220 of the imaging device 200 superimposes information indicating the extraction result of the affected area and information on the size of the affected area on the image data of the affected area transmitted from the information processing device 300. Is received via the communication device 219. Further, the system control circuit 220 communicates the posture icon transmitted from the information processing device 300, the posture image data, the tilt information of the imaging device 200 when the posture is photographed, and the tilt information of the imaging device 200 in the recording imaging. Receive via device 219.

In S510, the system control circuit 220 displays on the display device 223 image data in which information indicating the extraction result of the affected area and information on the size of the affected area are superimposed on the image data of the affected area. In this way, by superimposing information or the like indicating the extraction result of the affected area on the image data of the live view, the user confirms whether or not the estimated area and area of the affected area are appropriate. Then, you can proceed to shooting for recording.
Further, the system control circuit 220 displays the posture information of at least one of the received posture icon, posture image data, and tilt information of the image pickup device 200 when the posture is photographed on the display device 223. In this way, the user is notified of the posture information of the subject when the affected part is photographed in the past. The system control circuit 220 may display the tilt information of the image pickup device 200 in the recording imaging and the tilt information of the image pickup device 200 in the live view.

10 (a) and 10 (b) are diagrams showing an example of image data including posture information. The same reference numerals are given to the same images as those in FIGS. 7 (a) and 7 (b), and the description thereof will be omitted as appropriate.
The image 1001 shown in FIG. 10A is an example of displaying image data in which the posture icon 1002 is superimposed on the image 702 shown in FIG. 7B.
The system control circuit 220 displays on the display device 223 an image 1001 in which the posture icon or the posture icon 1002 based on the posture icon identification information received in S509 is superimposed on the image 702 shown in FIG. 7B.
Here, the posture icon 1002 functions as a button that can be touch-operated by the user via a touch panel that is also used as the display device 223. The system control circuit 220 changes the screen and displays the image 1003 shown in FIG. 10B in response to a touch operation on the posture icon 1002 by the user.

Image 1003 shown in FIG. 10B is an example of displaying image data of posture. In the upper left corner of the image 1003, a label 1006 containing tilt information 1004 and a character string 1005 is displayed in white characters on a black background.
The system control circuit 220 displays the image 1003 on which the label 1006 is superimposed on the image data of the posture received in S509 on the display device 223. The system control circuit 220 displays the tilt information 1004 based on the tilt information of the image pickup device 200 when the posture is photographed, which is received in S509. Further, when the posture information received in S509 includes character information indicating the posture, the system control circuit 220 displays the character string 1005 of the label 1006 based on the character information of the posture.

In this way, by notifying the user of the posture information of the subject when the affected part of the same subject was photographed in the past before photographing the affected part for recording, the user can photograph the affected part of the subject in the past. You can grasp your posture. Therefore, the user can appropriately photograph the affected part of the subject by having the subject take the same posture as when the subject was photographed in the past.

Specifically, by displaying the posture icon 1002 schematically showing the posture of the subject, the user can immediately grasp the posture of the subject when the affected part of the subject is photographed in the past. Further, by displaying the image 1003 in which the posture of the subject is photographed, the posture of the subject when the affected portion of the subject is photographed in the past can be accurately grasped. Further, by displaying the tilt information 1004 of the image pickup device 200, it is possible to grasp the tilt of the image pickup device 200 when the posture is photographed. However, the image displaying the posture information is not limited to the cases shown in FIGS. 10A and 10B, and any image may be used as long as the user can grasp the posture of the subject.

The system control circuit 220 may display the tilt information of the image pickup apparatus 200 in the recording imaging received in S509. By referring to the displayed tilt information, the user can take a picture of the affected part with the same inclination as when the affected part was taken in the past, and the imaging device 200 can face the surface of the affected part.
At this time, the system control circuit 220 may display the tilt information of the image pickup apparatus 200 in the live view generated in S507 or received in S509. In this case, since the user can refer to the inclination of the imaging device 200 at the present time, it is possible to match the inclination when the affected part is photographed in the past. Further, the system control circuit 220 may display information on the difference between the tilt information of the image pickup device 200 in the recording imaging and the tilt information of the image pickup device 200 in the live view. The difference information may be generated by the system control circuit 220 of the image pickup apparatus 200, or may be generated by the information processing apparatus 300 and received by the image pickup apparatus 200.

In S511, the system control circuit 220 determines whether or not the user has received a shooting instruction by pressing the release button included in the operation unit 224.
When the imaging instruction is received, in S512 or later, the process proceeds to image the affected area for recording. On the other hand, if the shooting instruction is not received, the process returns to S503 and the above-mentioned processing after S503 is performed. Therefore, by repeating the processes S503 to S511 until the shooting instruction is received, the image pickup apparatus 200 continuously transmits the image data of the live view to the information processing apparatus 300. Further, each time the image pickup device 200 transmits, the information processing device 300 receives image data obtained by superimposing information indicating the extraction result of the affected area and information on the size of the affected area on the image data of the affected area.

In S512, the AF control circuit 218 performs AF processing that controls the drive of the lens group 212 so that the subject is in focus. This process is the same as that of S503.
In S513, the image pickup unit 211 shoots a subject in response to a shooting instruction by the user. Specifically, the imaging unit 211 captures the affected area as a still image for recording.
When the system control circuit 220 determines in S537 that the subject information 900 corresponding to the target patient ID is not stored, the system control circuit 220 first captures the affected portion for recording and then captures the posture of the subject. You may give guidance to the user. Specifically, the system control circuit 220 adjusts the magnification of the image pickup unit 211 so that the whole body of the subject is photographed after photographing the affected portion, and photographs the image. When the posture of the subject is automatically photographed in this way, the process of photographing the posture of the subject in S502 can be omitted. Information to the effect that the subject information 900 corresponding to the target patient ID is not stored can be received from the information processing device 300 in S509.

In S514, the image processing circuit 217 acquires the captured image data, develops and compresses it, and generates, for example, JPEG standard image data. This process is the same as that of S505. However, in order to give priority to the accuracy when measuring the affected area, it is preferable to perform the resizing process with a size larger than or the same size as the image data in S505. The size of the resized image data is, for example, approximately 4.45 megabytes in the case of 1440 pixels × 1080 pixels and 4-bit RGB color. However, the size of the resized image data is not limited to this case.

In S515, the system control circuit 220 generates distance information to the subject. This process is the same as that of S506.
In S516, the system control circuit 220 generates tilt information of the image pickup device 200 in recording imaging based on the tilt information output from the tilt detection device 225. This process is the same as the process of S507.
In S517, the system control circuit 220 uses the image data of the affected area resized in S514, the distance information to the subject generated in S515, and the tilt information of the imaging device 200 in the recording imaging generated in S516 as a communication device. It is transmitted to the information processing apparatus 300 via 219.

Next, the process of the information processing device 300 is started.
In S541, the CPU 310 of the information processing device 300 receives the image data of the affected area, the distance information to the subject, and the tilt information of the image pickup device 200 in the recording imaging, which are transmitted by the image pickup device 200, via the communication device 313.
In S542, the CPU 310 extracts the affected area from the received image data of the affected area by using the auxiliary arithmetic unit 317 (dividing the affected area from the other area). This process is the same as that of S532.

In S543, the arithmetic unit 311 of the CPU 310 calculates the area of the affected area as information regarding the size of the extracted affected area. This process is the same as the process of S533.
In S544, the arithmetic unit 311 calculates the evaluation information of the affected area. Specifically, the arithmetic unit 311 determines the lengths of the major and minor axes of the extracted affected area and the affected area based on the length on the focal plane corresponding to one pixel on the image obtained in S543. Calculate the area of the circumscribed rectangle. In the pressure ulcer evaluation index DESIGN-R, it is stipulated that the size of a pressure ulcer measures the value of the product of the major axis and the minor axis. By analyzing the major axis and the minor axis in the information processing system 1 of the present embodiment, compatibility with the data measured by DESIGN-R can be ensured. Since DESIGN-R does not have a strict definition, mathematically, a plurality of major axis and minor axis calculation methods can be considered.

As a first example of the method of calculating the major axis and the minor axis, the arithmetic unit 311 calculates a rectangle (Minimum bounding rectangle) having the smallest area among the rectangles circumscribing the affected area. Next, the lengths of the long side and the short side of this rectangle are calculated, the length of the long side is set as the long diameter, and the length of the short side is calculated as the short diameter. Next, the area of the rectangle is calculated based on the length on the focal plane corresponding to one pixel on the image obtained in S543.
As a second example of the method of calculating the major axis and the minor axis, the arithmetic unit 311 selects the maximum ferret diameter which is the maximum caliper length as the major axis, and selects the minimum ferret diameter as the minor axis. The maximum ferret diameter, which is the maximum caliper length, may be selected as the major diameter, and the length measured in the direction orthogonal to the axis of the maximum ferret diameter may be selected as the minor diameter.
Any method can be selected for the calculation method of the major axis and the minor axis based on the compatibility with the conventional measurement results.

The process of calculating the length of the major axis and the minor axis of the affected area and the rectangular area is not executed for the image data received in S531. Since the purpose is to allow the user to confirm the extraction result of the affected area during the live view, the processing time is obtained by omitting the image analysis process corresponding to S544 for the image data received in S531. Is being reduced.

In S545, the image processing circuit 217 generates image data in which information indicating the extraction result of the affected area and information regarding the size of the affected area are superimposed on the image data for which the affected area is to be extracted. The information regarding the size of the affected area here includes evaluation information of the affected area such as the major axis and the minor axis of the affected area.
8 (a), 8 (b) and 8 (c) show information showing the extraction result of the affected area and information on the size of the affected area including the major axis and the minor axis of the affected area as image data. It is a figure for demonstrating the superimposing method. Since a plurality of pieces of information regarding the size of the affected area can be assumed, they will be described with reference to FIGS. 8A to 8C.

The image 801 shown in FIG. 8A uses a Minimum bounding rectangle as a method for calculating the major axis and the minor axis. In the upper left corner of the image 801 as information on the size of the affected area, a label 711 displaying the character string 712 of the area of the affected area in white characters on a black background is superimposed as in FIG. 7B. ..
Further, in the upper right corner of the image 801 as information on the size of the affected area, a label 812 indicating the major axis and the minor axis calculated based on the Minimum bounding rectangle is superimposed. The label 812 includes the character string 813 and the character string 814. The character string 813 represents the length of the major axis (unit: cm), and the character string 814 represents the length of the minor axis (unit: cm). Further, in the image 801, a rectangular frame 815 representing a Minimum bounding rectangle is superimposed on the affected area. By superimposing the rectangular frame 815 together with the length of the major axis and the minor axis, the user can confirm which part of the image the length is measured.

Further, a scale bar 816 is superimposed on the lower right corner of the image 801. The scale bar 816 is for measuring the size of the affected area 102, and the size of the scale bar with respect to the image data is changed according to the distance information. Specifically, the scale bar 816 is a bar in which a scale of up to 5 cm is carved in 1 cm units based on the length on the focal plane corresponding to 1 pixel on the image obtained in S543, and is an imaging device. It corresponds to the size on the focal plane of 200, that is, on the subject. The user can grasp the size of the subject or the affected part 102 by referring to the scale bar 816.

Further, the above-mentioned DESIGN-R Size evaluation index 817 is superimposed on the lower left corner of the image 801. In the DESIGN-R Size evaluation index 817, the major axis and minor axis (maximum diameter orthogonal to the major axis) of the skin damage range are measured (unit is cm), and the values obtained by multiplying each are classified into the above-mentioned 7 stages. Has been done. In the present embodiment, the index 817 obtained by replacing the major axis and the minor axis with the values output by the respective calculation methods is superimposed.

In the image 802 shown in FIG. 8B, the maximum ferret diameter is used as the major axis and the minimum ferret diameter is used as the minor axis. In the upper right corner of the image 802, a character string 823 indicating the major axis length and a label 822 displaying the character string 824 indicating the minor axis length are superimposed. Further, in the affected area of the image 802, an auxiliary line 825 corresponding to the measurement position of the maximum ferret diameter and an auxiliary line 826 corresponding to the minimum ferret diameter are displayed. By superimposing the auxiliary lines 825 and 826 together with the character strings 823 and 824 indicating the lengths of the major axis and the minor axis, the user can confirm which part of the image the length is measured. ..

The image 803 shown in FIG. 8C has the same major axis as the image 802, but the minor axis is measured as a length measured in a direction orthogonal to the axis of the maximum ferret diameter instead of the minimum ferret diameter. In the upper right corner of the image 803, a label 832 displaying a character string 823 indicating the major axis length and a character string 834 indicating the minor axis length is superimposed. Further, in the affected area of the image 803, an auxiliary line 825 corresponding to the measurement position of the maximum ferret diameter and an auxiliary line 836 corresponding to the length measured in the direction orthogonal to the axis of the maximum ferret diameter are displayed.

The various information superimposed on the image data shown in FIGS. 8A to 8C may be any one or a combination of two or more, and the information to be displayed can be selected by the user. Good. Further, the images shown in FIGS. 7 (a), 7 (b), 8 (a), 8 (b) and 8 (c) are examples, and information on the size of the affected area 102 and the affected area is provided. The display form, display position, size, font, font size, font color, positional relationship, etc. can be changed according to various conditions.

In S546, the CPU 310 of the information processing device 300 transmits information indicating the extraction result of the affected area and information regarding the size of the affected area to the imaging device 200 via the communication device 313. In the present embodiment, the CPU 310 transmits to the image pickup apparatus 200 the image data in which the image data of the affected area generated in S545 is superposed with the information indicating the extraction result of the affected area and the information on the size of the affected area.

In S547, the CPU 310 reads the patient ID from the image data of the barcode tag. If the patient ID has already been read in S535, the process can be omitted.
In S548, the CPU 310 collates the read patient ID with the patient ID of the subject registered in advance, and acquires the information of the name of the subject. If the information on the name of the subject has already been acquired in S536, the process can be omitted.

In S549, the CPU 310 adds information to the imaging date / time column 909, the image data column 910 of the affected area, the evaluation information column 911, and the tilt information column 912 of the affected area information 908 of the subject information 900 corresponding to the target patient ID, and stores the information. Store in device 312.
Specifically, the CPU 310 stores information on the date and time of shooting in S513 in the shooting date and time column 909. Further, the CPU 310 stores the image data of the affected area received in S541 in the image data column 910 of the affected area. Further, the CPU 310 stores the evaluation information calculated in S544 in the evaluation information column 911. Further, the CPU 310 stores the tilt information of the image pickup apparatus 200 in the recording photographing received in S541 in the tilt information column 912. As described in the subject information 900 of FIG. 9A, the CPU 310 stores the tilt information of the second tilt information column 907 of the posture information 903 based on the tilt information stored in the tilt information column 912. Can be updated or updated.

When the subject information corresponding to the target patient ID is not stored in the storage device 312, the CPU 310 generates the subject information corresponding to the information of the patient ID and the name of the subject, and the posture information of the subject information 900. Information is stored in 903 and affected area information 908.
Further, when the subject information corresponding to the target patient ID is stored in the storage device 312, the CPU 310 uses the image data already stored in the posture image data field 905 and the posture obtained in S502 of this shooting. It may be determined whether or not the image data of the above matches. Here, when the image data match, it means that the postures of the subjects included in both image data are the same. Therefore, for example, when the subject included in one image data is prone and the subject included in the other image data is lying down, the CPU 310 determines that the image data do not match. When the image data do not match, the CPU 310 updates the image data already stored in the posture image data field 905 with the posture image data obtained in S502 of the current shooting and stores it. The CPU 310 may update and store at least one of the posture icon field 904 and the first tilt information field 906 of the posture information 903, not limited to the posture image data.

Next, the process of the image pickup apparatus 200 is started.
In S518, the system control circuit 220 of the imaging device 200 superimposes information indicating the extraction result of the affected area and information on the size of the affected area on the image data of the affected area transmitted from the information processing device 300. Is received via the communication device 219.
In S519, the system control circuit 220 displays on the display device 223 the image data obtained by superimposing the information indicating the extraction result of the affected area and the information on the size of the affected area on the received image data of the affected area for a predetermined time. Here, the system control circuit 220 displays any of the images 801 to 803 shown in FIGS. 8A to 8C, and returns to the process of S503 when a predetermined time elapses.

As described above, according to the present embodiment, when the user photographs the affected portion with the imaging device 200, the posture of the subject is determined by notifying the user of the posture information of the subject when the affected portion of the same subject was photographed in the past. You can shoot in the same posture as when you shot in the past. Therefore, it is possible to take an image in which the user can perform the progress comparison more accurately.
Further, in the present embodiment, DESIGN-R (registered trademark) is used as an evaluation index of pressure ulcer, but the present invention is not limited to this. Other evaluation indicators such as Bates-Jensen Wound Assessment Tool (BWAT), Pressure Ulcer Scale for Healing (PUSH), and Pressure Sore Status Tool (PSST) may be used.

(First modification)
In S502 of the flowchart of FIG. 5 described above, the case where the posture of the subject is photographed has been described, but the case is not limited to this case. For example, in S502, the image pickup apparatus 200 may be configured so that the posture of the subject can be selected by the user. Specifically, in S502, the system control circuit 220 selectively displays the posture icons 921 to 924 shown in FIG. 9B or the character information indicating the posture on the display device 223. Therefore, the user can select a posture icon corresponding to the posture of the subject or character information. In S508, the system control circuit 220 transmits the posture icon (including the posture icon identification information) selected by the user or the character information to the information processing device 300.
By allowing the user to select the posture of the subject in this way, the posture of the subject can be easily specified. Further, since the process of transmitting and receiving the posture image data can be omitted, the processing load of the information processing system 1 can be reduced.

(Second modification)
In S538 of the flowchart of FIG. 5 described above, the case where the posture information 903 of the subject information 900 is transmitted to the imaging device 200 in order to notify the user of the posture of the subject when the affected part is photographed in the past has been described. , Not limited to this case. For example, in S537, when it is determined that the subject information 900 corresponding to the target patient ID is not stored in the storage device 312, the CPU 310 does not have to transmit the posture information 903 to the image pickup device 200. That is, when the subject information 900 corresponding to the target patient ID is not stored in the storage device 312 in S537, the user uses the posture of the subject when the affected part was photographed in the past in order to photograph the subject for the first time this time. This is because there is little need to notify.

(Third variant)
In S510 of the flowchart of FIG. 5 described above, the case where the system control circuit 220 displays the posture of the subject when the affected part is photographed in the past on the display device 223 has been described, but the case is not limited to this case. For example, the system control circuit 220 may notify the posture of the subject when the affected portion is photographed in the past by sound using an acoustic device (not shown).

Although the present invention has been described above with various embodiments and modifications, the present invention is not limited to the above-described embodiments and modifications, and changes and the like can be made within the scope of the present invention. The embodiments and modifications described above may be combined in a timely manner. For example, the object to be analyzed by the information processing apparatus 300 is not limited to the affected area, and may be an object included in the image data.

<Other Embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or recording medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

1: Information processing system 200: Imaging device 211: Imaging unit 216: Distance measuring system 217: Image processing circuit 219: Communication device 220: System control circuit 223: Display device 225: Tilt detection device 300: Information processing device 311: Arithmetic logic unit 312: Storage device 313: Communication device 317: Auxiliary arithmetic unit

Claims (20)

Imaging means and
A control means for acquiring the posture information of the subject when the affected part of the subject is photographed in the past and controlling the posture information of the subject to be notified to the user when the affected part of the subject is photographed by the imaging means. ,
An imaging device characterized by having. The control means
The imaging device according to claim 1, wherein the posture information of the subject is displayed on the display device. The control means
As the posture information, a display item schematically showing the posture of the subject, image data obtained by photographing the posture of the subject, tilt information of the imaging device when the posture of the subject is photographed, and the posture of the subject. The imaging device according to claim 2, wherein at least one of the character information represented by characters is displayed on the display device. The affected area is a pressure ulcer,
The imaging according to any one of claims 1 to 3, wherein the posture information includes information capable of identifying at least one posture of the subject in a prone position, lying down, or sitting position. apparatus. The control means
The imaging device according to any one of claims 2 to 4, wherein the posture information of the subject is superimposed on the image data of the live view captured by the imaging means and displayed on the display device. The control means
The claim is characterized in that the posture information of the subject is displayed on the display device by transitioning from the screen on which the image data of the live view captured by the imaging means is displayed to a different screen according to an operation by the user. Item 2. The imaging device according to any one of Items 2 to 5. 5. The image data of the live view taken by the image pickup means is the image data received from the external device after being transmitted from the image pickup device to the external device and image-processed by the external device. Or the image pickup apparatus according to 6. It has a communication means for transmitting the identification information of the subject to an external device.
The control means
The imaging according to any one of claims 1 to 7, wherein the identification information is transmitted to the external device by the communication means, and the posture information associated with the identification information is received from the external device. apparatus. It has a communication means for transmitting the posture information of the subject to an external device when the affected part of the subject is photographed by the imaging means.
The control means
The imaging according to any one of claims 1 to 7, wherein the posture information transmitted by the communication means and stored in the external device is received from the external device via the communication means. apparatus. The communication means
The imaging device according to claim 9, wherein image data obtained by photographing the posture of the subject by the imaging means is transmitted to the external device. The communication means
The imaging device according to claim 9, wherein the posture information selected by the user from a plurality of posture information is transmitted to the outside. It is characterized by having a communication means that receives the identification information of the subject from the imaging device and transmits the posture information of the subject when the affected part of the subject is photographed in the past, which is associated with the identification information, to the imaging device. Information processing device. The subject identification information and the posture information of the subject are associated and stored in the storage device.
The communication means receives the identification information of the subject and the posture information of the subject from the image pickup apparatus.
When the posture information stored in the storage device associated with the same identification information as the identification information received by the communication means does not match the posture information received by the communication means, it is stored in the storage device. The information processing device according to claim 12, further comprising a control means for updating the posture information to the posture information received by the communication means. The communication means receives the identification information of the subject and the posture information of the subject from the image pickup apparatus.
Control to associate the identification information received by the communication means with the posture information and store the subject information corresponding to the identification information received by the communication means in the storage device when the subject information corresponding to the identification information is not stored in the storage device. The information processing apparatus according to claim 12, wherein the information processing apparatus has means. An information processing system having an image pickup device and an information processing device.
The information processing device
It has a communication means that receives the identification information of the subject from the imaging device and transmits the posture information of the subject when the affected part of the subject is photographed in the past, which is associated with the identification information, to the imaging device.
The image pickup device
Imaging means and
When the affected portion of the subject is photographed by the imaging means, the control means for notifying the user of the posture information of the subject transmitted by the communication means.
An information processing system characterized by having. The step of acquiring the posture information of the subject when the affected part of the subject was photographed in the past, and
A step of controlling the posture information of the subject to be notified to the user, and
Steps to shoot the affected part of the subject and
A method for controlling an imaging device, which comprises. The step of receiving the identification information of the subject from the imaging device,
A step of transmitting the posture information of the subject when the affected part of the subject is photographed in the past, which is associated with the identification information, to the image pickup apparatus.
A method for controlling an information processing device, which comprises. It is a control method of an information processing system having an image pickup device and an information processing device.
A step of transmitting subject identification information from the imaging device to the information processing device, and
A step of transmitting the posture information of the subject when the affected part of the subject is photographed in the past, which is associated with the identification information, from the information processing device to the imaging device.
A step of controlling the posture information acquired by the imaging device to notify the user, and
The step of photographing the affected part of the subject and
A control method of an information processing system, characterized in that it has. A program for causing a computer to function as each means of the device according to any one of claims 1 to 14. A computer-readable recording medium on which a program for making a computer function as each means of the device according to any one of claims 1 to 14 is recorded.

Images