WO2023195091A1

WO2023195091A1 - Processing device, processing program, and processing method

Info

Publication number: WO2023195091A1
Application number: PCT/JP2022/017151
Authority: WO
Inventors: 渉 ▲高▼橋; 敦史福田; 賢 ▲高▼橋
Original assignee: アイリス株式会社
Priority date: 2022-04-06
Filing date: 2022-04-06
Publication date: 2023-10-12

Abstract

[Problem] To connect to an imaging device configured to capture an image of a subject, and improve the usability of the imaging device. [Solution] Provided is a processing device that is communicably connected, via a network, to an imaging device configured to capture an image, the subject of which is a natural orifice of a person subject to imaging, and that comprises at least one processor, wherein the at least one processor is configured to perform processing for: obtaining operation information, which relates to an operation of the imaging device performed by an operator of the imaging device, in order to capture an image in the imaging device; and outputting related information, which relates to the imaging device, on the basis of the obtained operation information.

Description

Processing device, processing program and processing method

The present disclosure relates to a processing device, a processing program, and a processing method connected to a photographing device configured to photograph an image of a subject.

2. Description of the Related Art Conventionally, photographing devices have been known for photographing an image of a living body such as a human being or a part thereof as a subject. For example, Patent Document 1 describes a camera head including a first housing that houses an image sensor, a cable section that is connected to the camera head and transmits at least an image signal from the image sensor, and a tube through which the cable section is inserted. a signal transmission unit comprising a second housing having a shape, an annular member sandwiched between the first housing and the second housing and having an outer circumferential surface for identifying the type of camera head, and the first housing. and an annular sealing member fixed by the second casing and the annular member.

JP2016-214661A

Therefore, based on the above-mentioned technology, the present disclosure provides various embodiments that allow a camera to be connected to a photographing device configured to photograph an image of a subject, and to improve the usability of the photographing device. The purpose is to provide a processing device, a processing program, and a processing method.

According to one aspect of the present disclosure, "a processing device communicably connected via a network to a photographing device configured to photograph an image of a subject's natural aperture as a subject, and including at least one processor. wherein the at least one processor acquires operation information related to an operation of the photographing device performed by an operator of the photographing device in order to photograph the image with the photographing device, and the acquired operation A processing device configured to perform processing for outputting related information related to the photographing device based on the information is provided.

According to one aspect of the present disclosure, "a processing device configured to capture an image of a subject's natural aperture as a subject and a processing device communicatively connected via a network to the imaging device," Obtaining operation information related to an operation of the photographing device performed by an operator of the photographing device in order to photograph the image, and outputting related information related to the photographing device based on the acquired operation information. A processing program that functions as a processor for , is provided.

According to one aspect of the present disclosure, "a processing device communicably connected via a network to a photographing device configured to photograph an image of a subject's natural aperture as a subject, and including at least one processor. , the processing method executed by the at least one processor, wherein the at least one processor performs an operation of the photographing device performed by an operator of the photographing device in order to photograph the image with the photographing device. and a step of outputting related information related to the photographing device based on the obtained operation information.

According to the present disclosure, it is possible to provide a processing device, a processing program, and a processing method that are connected to a photographing device configured to photograph an image of a subject and can improve the usability of the photographing device. .

Note that the above effects are merely illustrative for convenience of explanation, and are not limiting. In addition to or in place of the above effects, any effects described in the present disclosure or effects obvious to those skilled in the art may be achieved.

FIG. 1 is a diagram showing a usage state of an imaging device 200 according to an embodiment of the present disclosure. FIG. 2 is a diagram showing a usage state of the imaging device 200 according to an embodiment of the present disclosure. FIG. 3 is a block diagram showing the configuration of the processing system 1 according to an embodiment of the present disclosure. FIG. 4 is a block diagram showing the configuration of a display device 100 and a photographing device 200 according to an embodiment of the present disclosure. FIG. 5 is a block diagram showing the configuration of a server device 300 according to an embodiment of the present disclosure. FIG. 6A is a diagram conceptually showing an imaging device management table stored in server device 300 according to an embodiment of the present disclosure. FIG. 6B is a diagram conceptually showing an operator management table stored in the server device 300 according to an embodiment of the present disclosure. FIG. 6C is a diagram conceptually showing a target person management table stored in the server device 300 according to an embodiment of the present disclosure. FIG. 6D is a diagram conceptually showing a related information table stored in the server device 300 according to an embodiment of the present disclosure. FIG. 7 is a diagram showing a processing sequence executed between the display device 100, the imaging device 200, and the server device 300 according to an embodiment of the present disclosure. FIG. 8 is a diagram showing a processing flow executed in the imaging device 200 according to an embodiment of the present disclosure. FIG. 9A is a diagram showing a processing flow executed in the server device 300 according to an embodiment of the present disclosure. FIG. 9B is a diagram illustrating a processing flow related to generation of a trained model according to an embodiment of the present disclosure. FIG. 9C is a diagram showing a processing flow executed in the server device 300 according to an embodiment of the present disclosure. FIG. 9D is a diagram showing a processing flow executed in the server device 300 according to an embodiment of the present disclosure. FIG. 10A is a diagram illustrating an example of a screen displayed on the display device 100 according to an embodiment of the present disclosure. FIG. 10B is a diagram illustrating an example of a screen displayed on the display device 100 according to an embodiment of the present disclosure. FIG. 10C is a diagram illustrating an example of a screen displayed on the display device 100 according to an embodiment of the present disclosure. FIG. 10D is a diagram illustrating an example of a screen displayed on the display device 100 according to an embodiment of the present disclosure. FIG. 10E is a diagram illustrating an example of a screen displayed on the display device 100 according to an embodiment of the present disclosure.

Various embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that common components in the drawings are given the same reference numerals.

<First embodiment>
1. Overview of Processing System 1 The processing system 1 according to the present disclosure is mainly used to photograph the inside of a subject's oral cavity and obtain a subject image. In particular, the processing system 1 is used to image the back of the throat of the oral cavity, specifically, the pharynx. Therefore, in the following, a case will be mainly described in which the processing system 1 according to the present disclosure is used to photograph the pharynx. However, the pharynx is an example of a region to be imaged, and naturally, other regions in the oral cavity such as the tonsils and larynx, and other natural openings such as the external auditory canal, vagina, rectum, and nasal cavity are also included in the present disclosure. Such a processing system 1 can be suitably used.

As an example, the processing system 1 according to the present disclosure determines the possibility of contracting a predetermined disease from a subject image obtained by photographing a subject including at least the pharyngeal region of the oral cavity of the subject, and determines the possibility of contracting a predetermined disease. It is used to diagnose or assist in the diagnosis of diseases. An example of a disease determined by the processing system 1 is influenza. The possibility of influenza infection is usually diagnosed by examining the subject's pharynx and tonsils and determining the presence or absence of findings such as follicles in the pharynx region. However, by using the processing system 1 to determine the possibility of contracting influenza and outputting the result, it becomes possible to diagnose or assist. Note that the determination of the possibility of contracting influenza is one example. The processing system 1 can be suitably used even if it is only for photographing a subject. Further, the processing system 1 can be suitably used for any disease that causes a difference in oral cavity findings depending on the disease. Note that differences in findings are not limited to those discovered by a doctor or the like and whose existence is medically known. For example, any difference that can be recognized by a person other than a doctor or that can be detected by artificial intelligence or image recognition technology can be suitably applied to the processing system 1. Examples of such diseases include, in addition to influenza, infectious diseases such as streptococcal infection, adenovirus infection, EB virus infection, mycoplasma infection, hand-foot-and-mouth disease, herpangina, candidiasis, arteriosclerosis, diabetes, Diseases that exhibit vascular disorders or mucosal disorders such as hypertension, and tumors such as tongue cancer, pharyngeal cancer, and oral cavity cancer are included.

Note that in this disclosure, terms such as "determination" and "diagnosis" for diseases are used, but these do not necessarily mean definitive determination or diagnosis by a doctor. For example, it may naturally include that the processing system 1 of the present disclosure is used by the subject himself or by an operator other than a doctor, and the display device 100 included in the processing system 1 determines or diagnoses the subject. .

Furthermore, in the present disclosure, subjects to be photographed by the photographing device 200 may include any human being, such as a patient, a subject, a diagnosis subject, and a healthy person. Further, in the present disclosure, an operator who holds the imaging device 200 and performs an imaging operation is not limited to a medical worker such as a doctor, nurse, or laboratory technician, but may include any person such as the subject himself/herself. It is assumed that the processing system 1 according to the present disclosure is typically used in a medical institution. However, the application is not limited to this case, and may be used at any location such as the target's home, school, or workplace.

Furthermore, in the present disclosure, an institution associated with an operator is, for example, an institution associated with an operator, an imaging device, a display device, or a server device, and means a company, group, organization, or the like. Typically, examples include medical institutions, departments, departments, etc. to which medical personnel such as doctors, nurses, and laboratory technicians, who are operators, belong.

Furthermore, in the present disclosure, as described above, the subject only needs to include at least a part of the subject's natural orifice. Further, the disease to be determined may be any disease as long as it shows a difference in the findings at the natural orifice that is the subject. However, in the following, a case will be described in which the subject includes at least a portion of the oral cavity, particularly the pharynx or the vicinity of the pharynx, and the possibility of contracting influenza is determined as the disease.

Furthermore, in the present disclosure, the subject image may be one or more moving images or one or more still images. As an example of the operation, when the power button is pressed, the camera captures a live view image, and the captured live view image is displayed on the display 203. Thereafter, when the operator presses the photographing button, one or more still images are photographed by the camera, and the photographed images are displayed on the display 203. Alternatively, when the subject presses the shooting button, shooting of a moving image is started, and the image being shot with the camera during that time is displayed on the display 203. Then, when the shooting button is pressed again, shooting of the moving image is finished. In this way, in a series of operations, various images such as through images, still images, and videos are taken by the camera and displayed on the display, but the subject image does not only refer to a specific image among these images. , may include all images taken by the camera.

FIG. 1 is a diagram showing a usage state of an imaging device 200 according to an embodiment of the present disclosure. According to FIG. 1, the operator attaches the auxiliary tool 400 to the distal end of the imaging device 200 so as to cover it, and inserts the imaging device 200 together with the auxiliary tool 400 into the oral cavity 712 of the subject 700. Specifically, first, an operator (which may be the subject 700 himself or a person different from the subject 700) attaches the auxiliary tool 400 to the tip of the imaging device 200 so as to cover it. Then, the operator inserts the imaging device 200 with the auxiliary tool 400 attached into the oral cavity 712. At this time, the tip of the auxiliary tool 400 passes through the incisors 711 and is inserted to the vicinity of the soft palate 713. That is, the imaging device 200 is similarly inserted up to the vicinity of the soft palate 713. At this time, the tongue 714 is pushed downward by the auxiliary tool 400 (functioning as a tongue depressor) and the movement of the tongue 714 is restricted. This allows the operator to secure a good field of view for the imaging device 200 and to take good pictures of the pharynx 715 located in front of the imaging device 200.

The photographed subject image (typically, an image including the pharynx 715) is transmitted from the photographing device 200 to the server device 300 that is communicably connected via a wired or wireless network. The processor of the server device 300 that receives the subject image processes the processing program stored in the memory, thereby determining the possibility of contracting a predetermined disease. The results are then sent to the display device 100 and output to a display or the like via the output interface of the display device 100.

FIG. 2 is a diagram showing a usage state of the imaging device 200 according to an embodiment of the present disclosure. Specifically, FIG. 2 is a diagram showing a state in which the operator 600 grips the imaging device 200. According to FIG. 2, the imaging device 200 is composed of a main body 201, a grip 202, and a display 203 from the side inserted into the oral cavity. The main body 201 and the grip 202 are formed into a substantially columnar shape with a predetermined length along the insertion direction H into the oral cavity. Further, the display 203 is arranged on the side of the grip 202 opposite to the main body 201 side. Therefore, the photographing device 200 is formed into a generally columnar shape as a whole, and is held by the operator 600 in a manner similar to holding a pencil. That is, since the display panel of the display 203 faces toward the operator 600 when in use, it is possible to easily handle the photographing device 200 while checking the subject image photographed by the photographing device 200 in real time.

Furthermore, when the operator 600 holds the grip 202 in a direction in which a subject image is displayed in the normal direction on the display 203, the photographing button 220 is arranged on the top side of the grip. Therefore, when the operator 600 holds it, the operator 600 can easily press the shooting button 220 with his/her index finger or the like.

2. Configuration of processing system 1 FIG. 3 is a schematic diagram of processing system 1 according to an embodiment of the present disclosure. According to FIG. 3, the processing system 1 includes a display device 100, a photographing device 200, and a server device 300, and each device is communicably connected via a wired or wireless network. The display device 100 inputs subject information, interview information, diagnostic information, etc. required for processing in the server device 300, and determines related information related to the imaging device 200 and the possibility of contracting a predetermined disease. The results are received from the server device 300 and output.

The tip of the imaging device 200 is inserted into the oral cavity of the subject, photographs the inside of the oral cavity, particularly the pharynx, and transmits the photographed subject image to the server device 300 via a wired or wireless network. Further, the photographing device 200 transmits, to the server device 300, operation information regarding an operation performed by the operator or information used to generate the operation information in connection with the photographing.

The server device 300 receives the operation information transmitted by the imaging device 200 or the information used to generate the operation information, and outputs related information based on the operation information. Further, the server device 300 receives and processes a subject image photographed by the photographing device 200. Furthermore, the server device 300 determines the possibility of contracting a predetermined disease based on the received subject image, medical interview information, and finding information, and transmits the result to the display device 100 .

Note that in this disclosure, the processing device means the display device 100, the server device 300, or a combination thereof. That is, although the case where the server device 300 functions as a processing device will be described below, the display device 100 can also function as a processing device in the same way. Furthermore, in the present disclosure, storage and processing performed by the processing device may be distributed to other terminal devices, other server devices, and the like. In other words, the processing device is not limited to only one configured with a single housing, but may include the display device 100, the photographing device 200, the server device 300, another terminal device, another server device, or a combination thereof. including.

Further, in FIG. 3, only one display device 100 and only one photographing device 200 are shown. However, for example, in an institution such as a large hospital, it is possible that a plurality of display devices 100 or a plurality of imaging devices 200 may be managed and operated. Furthermore, the operator of the server device 300 may, for example, collectively manage and operate the display device 100 and the photographing device 200 that are respectively used by a plurality of institutions. Therefore, in the present disclosure, the processing system 1 can include one or more display devices 100, photographing devices 200, and server devices 300.

FIG. 4 is a block diagram showing the configuration of a display device 100 and a photographing device 200 according to an embodiment of the present disclosure. Specifically, FIG. 4 is a diagram specifically showing the configurations of the display device 100 and the photographing device 200 in the processing system 1. According to FIG. 4, the display device 100 includes a processor 111, a memory 112, an input interface 113, an output interface 114, and a communication interface 115. The photographing device 200 also includes a camera 211, a light source 212, a processor 213, a memory 214, an output interface 215, an input interface 210, a communication interface 216, a sensor 217, and a battery 218. Each of these components is electrically connected to each other via control lines and data lines. Note that the display device 100 and the photographing device 200 do not need to include all of the components shown in FIG. 4, and can be configured by omitting some of them, or can include other components. be. For example, the display device 100 or the photographing device 200 can include a battery or the like for driving each component. Note that the display device 100 and the photographing device 200 only need to be communicably connected through a wired or wireless network, as described with reference to FIG. 3, and there is no need for the two to be configured to be able to directly communicate.

2-1. Configuration of Display Device 100 First, in the display device 100, the processor 111 functions as a control unit that controls other components of the processing system 1 based on a processing program stored in the memory 112. Based on the processing program stored in the memory 112, the processor 111 inputs subject information, interview information, finding information, etc., and outputs related information received from the server device 300. Specifically, the processor 111 performs "a process of accepting input of target person information related to a target person by an operator or the target person himself/herself via the input interface 113" and a "process of accepting input of target person information related to the target person via the communication interface 115". ``Process of transmitting subject information to the server device 300'', ``Process of receiving input of interview information and finding information of the subject by the operator or the subject via the input interface 113'', ``Process of receiving input of interview information and finding information of the subject via the communication interface 115''; ``Process of transmitting the received interview information and finding information to the server device 300 together with the subject information'', ``selecting a subject via the input interface 113, and determining the possibility of the selected subject being affected by a predetermined disease. ``A process of transmitting a request to the server device 300 via the communication interface 115'', ``A process of transmitting a request to the server device 300 via the communication interface 115'', ``a process of transmitting a request to the server device 300 via the communication interface 115, and a process of transmitting a request to the server device 300 via the communication interface 115 to send a request to the server device 300 via the communication interface 115 to send a request to the server device 300 via the communication interface 115, A process of receiving a determination result and a subject image shown in FIG. and the like are executed based on the processing program stored in the memory 112. The processor 111 is mainly composed of one or more CPUs, but may be appropriately combined with GPUs, FPGAs, etc.

The memory 112 is comprised of RAM, ROM, nonvolatile memory, HDD, etc., and functions as a storage unit. The memory 112 stores instructions for various controls of the processing system 1 according to the present embodiment as a processing program. Specifically, the memory 112 stores information such as "processing for receiving input of subject information related to a subject by an operator or the subject himself/herself via the input interface 113" and "processing for receiving input of subject information related to the subject via the communication interface 115". ``Process of transmitting subject information to the server device 300'', ``Process of receiving input of interview information and finding information of the subject by the operator or the subject via the input interface 113'', ``Process of receiving input of interview information and finding information of the subject via the communication interface 115''; ``Process of transmitting the received interview information and finding information to the server device 300 together with the subject information'', ``selecting a subject via the input interface 113, and determining the possibility of the selected subject being affected by a predetermined disease. ``A process of transmitting a request to the server device 300 via the communication interface 115'', ``A process of transmitting a request to the server device 300 via the communication interface 115'', ``a process of transmitting a request to the server device 300 via the communication interface 115, and a process of transmitting a request to the server device 300 via the communication interface 115 to send a request to the server device 300 via the communication interface 115 to send a request to the server device 300 via the communication interface 115, A process of receiving a determination result and a subject image shown in FIG. A processing program for the processor 111 to execute, such as a process for outputting the information received by the processor 111, is stored therein. In addition to the processing program, the memory 112 also stores subject information, subject images, interview information, finding information, related information, and the like.

The input interface 113 functions as an input unit that receives an instruction input from an operator to the display device 100. Examples of the input interface 113 include a "confirm button" for making various selections, a "back/cancel button" for returning to the previous screen or canceling the entered confirmation operation, and a pointer output to the output interface 114. Physical key buttons include a cross key button for movement, an on/off key for turning on and off the power of the display device 100, and a character input key button for inputting various characters. Note that it is also possible to use a touch panel as the input interface 113, which is provided to be superimposed on the display that functions as the output interface 114 and has an input coordinate system corresponding to the display coordinate system of the display. In this case, icons corresponding to the physical keys are displayed on the display, and the operator inputs instructions via the touch panel to select each icon. The detection method of the subject's instruction input using the touch panel may be any method such as a capacitance method or a resistive film method. In addition to the above, a mouse, a keyboard, etc. can also be used as the input interface 113. The input interface 113 does not always need to be physically provided in the display device 100, and may be connected via a wired or wireless network as necessary.

The output interface 114 functions as an output unit for outputting information such as determination results and related information received from the server device 300. An example of the output interface 114 is a display including a liquid crystal panel, an organic EL display, a plasma display, or the like. However, the display device 100 itself does not necessarily need to be equipped with a display. For example, an interface for connecting to a display or the like that can be connected to the display device 100 via a wired or wireless network can function as the output interface 114 that outputs display data to the display or the like.

The communication interface 115 functions as a communication unit for transmitting and receiving subject information, interview information, subject images, finding information, related information, etc. to and from the server device 300 connected via a wired or wireless network. . Examples of the communication interface 115 include wired communication connectors such as USB and SCSI, wireless communication transmitting and receiving devices such as wireless LAN, Bluetooth (registered trademark), and infrared rays, and various connection terminals for printed mounting boards and flexible mounting boards. There are various things that can be mentioned.

2-2. Configuration of Photographing Apparatus 200 In the photographing apparatus 200, the camera 211 functions as a photographing unit that detects reflected light reflected on the oral cavity of the subject and generates a subject image. The camera 211 includes, for example, a CMOS image sensor to detect the light, and a lens system and a drive system to realize a desired function. The image sensor is not limited to a CMOS image sensor, but other sensors such as a CCD image sensor can also be used. Although not particularly illustrated, the camera 211 can have an autofocus function, and is preferably set to focus on a specific portion, for example, in front of a lens. Further, the camera 211 may have a zoom function, and is preferably set to take images at an appropriate magnification depending on the size of the pharynx or influenza follicle.

The light source 212 is driven by instructions from the processor 213 of the imaging device 200, and functions as a light source section for irradiating light into the oral cavity. Light source 212 includes one or more light sources. In this embodiment, the light source 212 includes one or more LEDs, and each LED emits light having a predetermined frequency band toward the oral cavity. The light source 212 uses light having a desired band among an ultraviolet light band, a visible light band, and an infrared light band, or a combination thereof. Note that when determining the possibility of contracting influenza in the display device 100, it is preferable to use light in the visible light band.

The processor 213 functions as a control unit that controls other components of the imaging device 200 based on the processing program stored in the memory 214. Based on the processing program stored in the memory 214, the processor 213 executes the process of "receiving a target person who has not yet been photographed through a communication interface 216 from a server device 300 that is communicatively connected to the photographing device 200 via a network." ``Process of receiving information'', ``Process of selecting subject information of a subject to be photographed from a list of subjects who have not yet been photographed'', ``Process of selecting subject information of a subject to be photographed from a list of subjects to be photographed who have not yet been photographed'', ``Process of selecting subject information of a subject to be photographed from a list of subjects to be photographed who have not yet been photographed''; ``Processing of photographing a subject image and transmitting it to server device 300'', ``Processing of detecting, by sensor 217, a state that changes due to the operation of the photographing device operated by the operator''; Operation information resulting from the operation or information used to generate the operation information (information detected by the sensor 217, operations including the operation history of the operator's operation of the photographing device 200, the change history of the settings of the photographing device 200, etc.) "processing for transmitting logs (including logs, etc.) to the server device 300 via the communication interface 216" is executed based on the processing program stored in the memory 214. The processor 213 is mainly composed of one or more CPUs, but may be appropriately combined with a GPU, FPGA, etc.

The memory 214 is composed of RAM, ROM, nonvolatile memory, HDD, etc., and functions as a storage unit. The memory 214 stores instructions for various controls of the processing system 1 according to the present embodiment as a processing program. Specifically, the memory 214 performs "a process of receiving subject information of a subject to be photographed who has not yet been photographed via the communication interface 216 from the server device 300 that is communicably connected to the photographing device 200 via a network"; ``Process of selecting subject information of a subject to be photographed from a list of subjects who have not yet been photographed''; "a process of transmitting information to the server device 300", "a process of detecting, by the sensor 217, a state that changes due to the operation of the imaging device operated by the operator", "a process of detecting operation information or information resulting from an operation of the imaging device 200 by the operator" A process of transmitting information used to generate operation information (including information detected by the sensor 217, a history of operations performed by the operator, operation logs, etc.) to the server device 300 via the communication interface 216, etc. , stores a processing program for execution by the processor 213. In addition to the processing program, the memory 214 also stores subject information, subject images, operation information, etc. of the subject.

The output interface 215 functions as an output unit for outputting the subject image photographed by the photographing device 200, subject information, and the like. An example of the output interface 215 is the display 203, but the output interface 215 is not limited to this, and may be configured from other liquid crystal panels, organic EL displays, plasma displays, etc. Further, the display 203 does not need to be provided, and for example, an interface for connecting to a display, etc. that can be connected to the display device 100 via a wired or wireless network is an output interface that outputs display data to the display, etc. It is also possible to function as 114.

The input interface 210 functions as an input unit that accepts the subject's instruction input to the display device 100 and the photographing device 200. Examples of the input interface 210 include a "shooting button" for instructing the shooting device 200 to start and end recording, a "power button" for turning on and off the power of the shooting device 200, and a "power button" for making various selections. "Confirm button", "Back/Cancel button" for returning to the previous screen or canceling the entered confirmation operation, and physical key buttons such as a cross key button for moving icons etc. displayed on the output interface 215. It will be done. Note that these various buttons and keys may be physically prepared, or may be displayed as icons on the output interface 215 and may be displayed on the output interface 215 using a touch panel or the like superimposed on the output interface 215 and arranged as the input interface 210. It may also be possible to make the selection possible. The detection method of the subject's instruction input using the touch panel may be any method such as a capacitance method or a resistive film method.

The communication interface 216 functions as a communication unit for transmitting and receiving information to and from the server device 300 and/or other devices. Examples of the communication interface 216 include wired communication connectors such as USB and SCSI, wireless communication transmitting and receiving devices such as wireless LAN, Bluetooth (registered trademark), and infrared rays, and various connection terminals for printed mounting boards and flexible mounting boards. There are various things that can be mentioned.

The sensor 217 functions as a detection unit for detecting the state of each component making up the imaging device 200. Examples of the sensor 217 include a voltage sensor, a current sensor, a temperature sensor, a GPS, an acceleration sensor, or a combination thereof. A detected value (for example, voltage, current, temperature, position information, or a combination thereof) detected by the sensor 217 is transmitted to the server device 300 as operation information or one of the pieces of information used to generate the operation information.

The battery 218 functions as a power supply unit that supplies power to drive each component of the imaging device 200. The battery 218 is charged by receiving power from the power source when connected to the power source, and starts discharging to drive each component when the photographing device 200 is turned on by pressing a power button or the like. do.

2-3. Configuration of server device 300 FIG. 5 is a block diagram showing the configuration of server device 300 according to an embodiment of the present disclosure. According to FIG. 5, server device 300 includes a memory 311, a processor 312, and a communication interface 313. Each of these components is electrically connected to each other via control lines and data lines. Note that the server device 300 does not need to include all of the components shown in FIG. 5; it is possible to omit some of them, or to add other components. For example, it is also possible to configure the server device 300 by connecting it with another server device. Moreover, it is also possible to connect with other database devices and configure the server device 300 as an integrated unit.

The memory 311 is composed of RAM, ROM, nonvolatile memory, HDD, etc., and functions as a storage unit. The memory 311 stores instructions for various controls of the processing system 1 according to the present embodiment as a processing program. Specifically, the memory 311 stores "processing for acquiring operation information related to the operation of the photographing device 200 by the operator in order to photograph an image of a subject with the photographing device 200" and "processing for acquiring operation information related to the operation of the photographing device 200 by the operator to photograph an image of a subject with the photographing device 200" A process of outputting related information related to the device 200 to at least one of the display device 100 and the photographing device 200,” and “receiving a subject image photographed by the photographing device 200 from the photographing device 200 via the communication interface 313.” ``Process of receiving medical interview information and finding information inputted on display device 100 from display device 100 via communication interface 313, and storing it in a subject management table in association with subject ID information''; When the display device 100 receives a request to determine the possibility of a predetermined disease for a selected subject from the display device 100 via the communication interface 313, a subject image associated with the subject from the image management table is displayed. , a process of reading out interview information and finding information associated with the subject from the subject management table and determining the possibility, and a process of displaying the determined results on the display device 100 via the communication interface 313. It stores processing programs for the processor 312 to execute, such as "transmission processing". In addition to the processing program, the memory 311 also stores an imaging device management table (FIG. 6A), an operator management table (FIG. 6B), a subject management table (FIG. 6C), and a related information table (FIG. 6D). Stores various information etc.

The processor 312 functions as a control unit that controls other components of the server device 300 based on the processing program stored in the memory 311. Based on the processing program stored in the memory 311, the processor 312 performs processing for determining the possibility of contracting a predetermined disease and processing for outputting related information. Specifically, the processor 312 performs "a process of acquiring operation information related to an operation of the photographing device 200 by an operator to photograph an image of a subject with the photographing device 200" and a "process of acquiring operation information related to the operation of the photographing device 200 by an operator to photograph an image of a subject with the photographing device 200"; A process of outputting related information related to the device 200 to at least one of the display device 100 and the photographing device 200,” and “receiving a subject image photographed by the photographing device 200 from the photographing device 200 via the communication interface 313.” ``Process of receiving medical interview information and finding information inputted on display device 100 from display device 100 via communication interface 313, and storing it in a subject management table in association with subject ID information''; When the display device 100 receives a request to determine the possibility of a predetermined disease for a selected subject from the display device 100 via the communication interface 313, a subject image associated with the subject from the image management table is displayed. , a process of reading out interview information and finding information associated with the subject from the subject management table and determining the possibility, and a process of displaying the determined results on the display device 100 via the communication interface 313. "transmission processing" and the like are executed based on the processing program stored in the memory 311. The processor 312 is mainly composed of one or more CPUs, but may be appropriately combined with a GPU, FPGA, or the like.

The communication interface 313 functions as a communication unit for transmitting and receiving information to and from the display device 100, the photographing device 200, and/or other devices. Examples of the communication interface 313 include wired communication connectors such as USB and SCSI, wireless communication transmitting and receiving devices such as wireless LAN, Bluetooth (registered trademark), and infrared rays, and various connection terminals for printed mounting boards and flexible mounting boards. There are various things that can be mentioned.

3. Information stored in the memory 311 of the server device 300 FIG. 6A is a diagram conceptually showing an imaging device management table stored in the server device 300 according to an embodiment of the present disclosure. The information stored in the imaging device management table is updated and stored as needed according to the progress of processing by the processor 312 of the server device 300.

According to FIG. 6A, the photographing device management table stores operation information, related information, software (SW) version information, parts information, etc. in association with photographing device ID information. “Photographing device ID information” is information unique to each photographing device 200 and used to identify each photographing device 200. As the imaging device ID information, an arbitrary character string or code may be assigned by the server device 300, or information such as a manufacturing number or product number may be used. “Operation information” is information related to an operation by an operator for photographing an image with each photographing device 200. An example of such information is an output value from the sensor 217 such as a voltage sensor, current sensor, or temperature sensor (it may be the output value itself from the sensor 217, or data obtained by processing the output value). ), the number of times the auxiliary device 400 is worn, the number of times the image is taken, the number of times the battery 218 is charged, the voltage of the battery 218 when fully charged, the accumulated total charging time of the battery 218, and the number of times the battery 218 is overcharged. the number of times the battery 218 is over-discharged, the number of times the light source 212 is turned on, the time the light source 212 is turned on, the current value flowing through the light source 212, the display time of the through image, the usage time of the photographing device 200 (for example, when the power is turned on) to off, the time from the end of sleep to the start of sleep), the number of presses of each physical key, the location where the imaging device 200 is used, the number of times the possibility of contracting a predetermined disease is determined, communication speed, communication time, wireless Examples include the radio field strength of communication, the operation log for the photographing device 200, or a combination thereof. Such operation information is obtained by being received from the photographing device 200 or generated by the server device 300 based on information used to generate the operation information received from the photographing device 200.

"Related information" is information generated based on operation information, and is information related to the imaging device 200. Examples include information on how to use the imaging device 200, information that promotes at least one of the replacement, inspection, and calibration of various components (parts, etc.) that make up the imaging device 200, and information that promotes the purchase of the auxiliary tool 400. Examples include information on promotion, information on fees for determining morbidity for a predetermined disease, and the like. Specifically, the information regarding how to use the photographing device 200 includes information such as tips for using the photographing device 200, a manual, methods for improving operation, promotion of charging the battery 218, and estimated time for charging completion. Information that facilitates the replacement, inspection, and/or calibration of various components includes the battery 218, the LED used in the light source 212, the COMS sensor used in the camera 211, and various lenses installed in the photographing device 200. , Notification of the quality of the light guide tube for transmitting the light emitted from the light source 212 to the outside, physical keys, touch panel, software built into the photographing device 200, etc., and notification of the timing of replacement, inspection, calibration, etc.; Examples include notification of replacement, inspection, and calibration methods. Information regarding the judgment fee includes notifications of judgment fees that have occurred so far, notifications of judgment fees that are predicted to occur in the future, notifications of payment methods for the judgment fees, and the like. Examples of information that promotes the purchase of the auxiliary tool 400 include information indicating the number of remaining auxiliary tools 400 in stock, information indicating the purchasing method, and the like.

"SW version information" is information for specifying the version of the processing program currently stored in each imaging device 200. The information is updated every time a new version of the processing program is installed. “Parts information” is information for specifying each of the various components (parts, etc.) installed in each imaging device 200. Such information may be any information as long as it can identify each of the main components, but typically includes a serial number. The part information may be output together with the related information, or if there is a need to repair or replace only the part that corresponds to the manufacturing lot, the part information can be used to output the part information based on the information. is used to identify the photographing device 200 installed.

FIG. 6B is a diagram conceptually showing an operator management table stored in the server device 300 according to an embodiment of the present disclosure. The information stored in the operator management table is updated and stored as needed according to the progress of processing by the processor 312 of the server device 300.

According to FIG. 6B, in the operator management table, institution ID information, operator ID information, display device ID information, photographing device ID information, etc. are stored in association with each other. "Institution ID information" is information specific to the institution to which each operator belongs, and is information for identifying each institution. For example, when the operator is a medical worker, a code or name is used that identifies the medical institution, department, department, etc. to which the medical worker belongs. The operator ID information is information unique to each operator and used to identify each operator. “Display device ID information” is information specific to each display device 100 and is information for identifying each display device 100. Based on the display device ID information, it is possible to specify the display device 100 held by each institution or each operator. “Photographing device ID information” is information specific to the photographing device 200 and is information for identifying each photographing device 200. Based on the imaging device ID information, it is possible to specify the imaging device 200 managed by each institution or each operator.

FIG. 6C is a diagram conceptually showing a target person management table stored in the server device 300 according to an embodiment of the present disclosure. The information stored in the target person management table is updated and stored as needed according to the progress of processing by the processor 312 of the server device 300.

According to FIG. 6C, the subject management table stores interview information, finding information, subject images, determination result information, etc. in association with subject ID information. “Target ID information” is information unique to each target person and used to identify each target person. The subject ID information is generated every time a new subject is registered by the operator or the subject himself/herself. "Interview information" is information inputted by, for example, an operator or a subject, and is information used as a reference for diagnosis by a doctor or the like, such as the subject's medical history and symptoms. "Finding information" is information input by an operator such as a doctor, and indicates a situation that differs from normal as obtained by various examinations such as visual inspection, interview, palpation, and auscultation, as well as tests to assist in diagnosis. It is information. The “subject image” is information indicating the image data itself or the storage location of the subject image used to determine the possibility of contracting a predetermined disease. "Determination result information" is information indicating the determination result of the possibility of contracting a predetermined disease such as influenza based on the determination image. An example of such determination result information is the positivity rate for influenza. However, it is not limited to the positive rate, but any information that indicates the possibility, such as information specifying whether it is positive or negative, may be used. Further, the determination result does not need to be a specific numerical value, and may be in any format, such as classification according to the high positive rate or classification indicating whether it is positive or negative.

FIG. 6D is a diagram conceptually showing a related information table stored in the server device 300 according to an embodiment of the present disclosure. The information stored in the related information table is updated and stored whenever it is added to the processor 312 of the server device 300 through processing.

According to FIG. 6D, the related information table stores related information that is output in association with operation information for each type of related information that is output. Note that the types of related information shown in FIG. 6D are just examples, and of course it is possible to add or subtract as appropriate depending on the related information that is desired to be output. First, when referring to "information promoting the purchase of auxiliary device 400" as the type of related information, nothing is output when the "number of times worn" is from 0 to less than 100 times, and when it is less than 100 times and 150 times. When the 181st to 200th operation information is obtained, it outputs ``The stock of auxiliary tools is low.'' and 201. From the first time onward, the message "Please purchase auxiliary tools from here." is output. Also, when referring to "information regarding how to use the photographing device 200" as the type of related information, if the output value of the "voltage sensor" which is the operation information is greater than or equal to X1 and less than X2, it is considered to be a normal value and nothing is output. , if the value is greater than or equal to X2 and less than X3, the message "The battery is low" is output. If the value is greater than or equal to X3 and less than X4, the message "Please charge the battery" is output. As mentioned above, there are various combinations of operation information to be acquired and related information to be output in addition to these, but the details will be described later.

4. Processing sequence executed by processing system 1 FIG. 7 is a diagram showing a processing sequence executed between display device 100, photographing device 200, and server device 300 according to an embodiment of the present disclosure. Of these, S11 to S21 are processes for outputting related information mainly based on information received from the photographing device 200, and S41 to S46 are processes for outputting related information mainly based on information received from the display device 100. show. In addition, although the case where a single display device 100 and a single photographing device 200 are connected will be described below, the case where a plurality of display devices 100 and a plurality of photographing devices 200 are connected is naturally applicable. can also be processed in the same way.

According to FIG. 7, the power of the photographing device 200 is turned on by pressing the power button or the like, and the photographing device 200 is activated (S11). Then, the photographing device 200 extracts the target person information of the target to be photographed based on the operator's instruction input received by the input interface 210 from the target person information of the target who has not been photographed received from the server device 300. Select (S12). Next, the photographing device 200 determines whether or not the auxiliary tool 400 is worn, and if it is not yet worn, outputs a wearing display to encourage the user to wear it via the output interface 215 (S13). Note that this display is just an example, and the wearing may be prompted by other sounds, flashing lights, vibrations, or the like. When it is detected that the auxiliary tool 400 is attached to the photographing device 200 (S14), the photographing device 200 photographs a subject image based on the operator's instruction input received at the input interface 210 (S15). ). Note that although here, the attachment of the auxiliary tool 400 is detected, this process itself may be skipped. Further, instead of detecting whether the auxiliary device 400 is attached, for example, a confirmation display for the operator to confirm that the auxiliary device 400 is attached is output via the output interface 215, and the operator inputs a predetermined operation in response to the confirmation display (for example, , tap operation, etc.), the operator himself/herself may be able to input that he/she is wearing the auxiliary device 400. Furthermore, the attachment of the auxiliary tool 400 is detected by, for example, using the camera 211 to detect an image specific to the auxiliary tool 400, or by pressing a pre-installed switch when the auxiliary tool 400 is attached. Good too. Furthermore, these methods may be used to detect not only attachment but also removal of the auxiliary tool 400.

When the subject image is photographed, the photographing device 200 transmits the photographed subject image (T11) to the server device 300 along with the subject ID information of the first subject via the communication interface 216. When the server device 300 receives the subject image via the communication interface 313, it stores the received subject image in the subject management table in association with the subject ID information received together.

Additionally, in the series of processes described above, the photographing device 200 acquires various operation information or information used to generate the operation information. After the photographing device 200 is activated in S11, the voltage of the battery 218 is detected at predetermined intervals by a voltage sensor connected to the battery 218. Further, the number of times each physical key is pressed and the number of times the touch panel is operated are counted when the power button is pressed in S11, an instruction is input for selecting a subject in S12, and an instruction is input for photographing in S15. Further, in S15, the light source 212 is turned on in conjunction with photographing, so that the number of times the light source 212 is turned on is counted, and the lighting time is measured. Furthermore, when the light source 212 is turned on in S15, the value of the current flowing through the light source 212 is detected using a current sensor connected to the light source 212. Furthermore, when the attachment of the auxiliary tool 400 is detected in S14, the number of times the auxiliary tool 400 is attached is counted. Although an example is given here, other operation information or information used to generate operation information may of course be acquired. Details of the acquired operational information or the information used to generate the operational information method will be described later.

After reading the operation information acquired as described above or the information used to generate the operation information, the photographing device 200 transmits the read information (T12) to the server device 300 together with the photographing device ID information (S17). The server device 300 refers to the operation information in the photographing device management table based on the received photographing device ID information, and updates and stores the operation information or information used to generate the operation information (S18). Then, the server device 300 generates related information by referring to the related information table based on the updated and stored operation information and identifying related information that satisfies the conditions set in the operation information (S19). . When the related information is generated, the server device 300 stores the generated related information in the imaging device management table and outputs it to the display device 100 and the imaging device 200 (T13). The display device 100 and the photographing device 200 that have received the related information output the received related information to a display or the like via the output interface 114 and the output interface 215 (S20 and S21).

Next, according to FIG. 7, the display device 100 receives selection of the subject information of the subject to be diagnosed on the screen on which the list of subject information of the subject is displayed (S41). At this time, although not particularly illustrated, the display device 100 generates interview information and observation information of the subject based on input by the subject or the operator, and stores the generated information in the subject management table of the server device 300 in advance. Then, the display device 100 refers to the subject management table, reads out the subject ID information of the subject, and transmits it to the server device 300 together with a request for determining the possibility of contracting a predetermined disease (T41). When the server device 300 receives the determination request, it refers to the subject image in the subject management system based on the subject ID information received together with the judgment request, and displays the subject image and interview information associated with the subject ID information. and read out finding information. Then, the server device 300 determines the possibility of contracting a predetermined disease based on the read subject image, and stores the determination result in the subject management table (S42). The server device 300 transmits the stored determination result (T42) to the display device 100. The display device 100 that has received the determination result displays the received determination result on the display via the output interface 114 (S43).

Additionally, the server device 300 acquires various operation information in a series of processes. For example, when the possibility of contracting a predetermined disease is determined in S42, the number of times of determination among the operation information in the operator management table is updated and stored (S44). From then on, similarly to S19 to S21, the server device 300 refers to the related information table based on the updated and stored operation information and identifies related information that satisfies the conditions set in the operation information. Related information is generated (S45). When the related information is generated, the server device 300 stores the generated related information in the imaging device management table and outputs it to the display device 100 and the imaging device 200 (T23). The display device 100 and the photographing device 200 that have received the related information output the received related information to a display or the like via the output interface 114 and the output interface 215 (S46 and S47).

Note that in the example of FIG. 7, the process of reading out the operation information in the photographing device 200 and transmitting it to the server device 300 is executed after the subject image is transmitted. However, the timing of transmission is each time the operation information or information used to generate the operation information is acquired by the imaging device 200, the timing at every predetermined period, and the timing when communicating with the server device 300 for transmitting other information. It may be performed at any timing or a combination thereof. Further, although the generated related information is output to both the display device 100 and the photographing device 200, it may be output to only one of them. The display device 100 is used frequently for inputting medical interview information and finding information, and therefore is frequently used by the operator. Therefore, it is desirable to at least output it to the display device 100. Further, the method of outputting the related information in the display device 100 and the photographing device 200 is not limited to displaying on a display or the like, but may be any method such as sound, issuing of an LED, etc., vibration, or a combination thereof. This completes the processing sequence.

5. Process flow executed by photographing device 200 FIG. 8 is a diagram showing a process flow executed by photographing device 200 according to an embodiment of the present disclosure. Specifically, FIG. 8 is a diagram showing a processing flow executed in the photographing process of S11 to S17 in FIG. 7. The processing flow is mainly performed by the processor 213 of the photographing device 200 reading and executing a processing program stored in the memory 214. In addition, in the following processing flow, an example of information acquired as operation information or information used to generate operation information is illustrated, but of course the information is not limited to the exemplified information only, and other information may also be used. Information may be obtained.

According to FIG. 8, the photographing device 200 starts the photographing device 200 by detecting a press of the power button or the like (S111). At this time, the processor 213 stores, for example, the following information in the memory 214 as the operation information or information used to generate the operation information.
・The number of presses of the power button in response to the press of the power button ・Detected value from the voltage sensor connected to the battery 218 ・Current time obtained by referring to the timer (start time of use of the imaging device 200)
- Location of use of the imaging device 200 using GPS - Communication speed of the communication interface 216 - Radio field strength when performing wireless communication via the communication interface 216 - Operation log - Detection value from the acceleration sensor - Heat generation of the imaging device 200 Detected value from a temperature sensor installed at a location (for example, around the light source 212, a control board on which the processor 213, etc. is mounted)

Note that, even if not specifically exemplified in the following processing, for example, a detection value from a voltage sensor connected to the battery 218, a communication speed of the communication interface 216, a radio wave intensity when performing wireless communication via the communication interface 216, The operation log, the detected value from the acceleration sensor, etc. are acquired at any time and stored in the memory 214.

Next, when the processor 213 receives the subject information of the subjects whose subject images have not been captured from the server device 300 via the communication interface 216, the processor 213 outputs the subject information of the subjects whose subject images have not been captured as a list via the interface 215. output to the display. Then, the processor 213 receives selection of subject information of the subject to be photographed from the list via the input interface 210 (S112). At this time, the processor 213 stores, for example, the following information in the memory 214 as the operation information or information used to generate the operation information.
- Receiving selection operations via the touch panel, number of touch panel operations and operation log

Next, the processor 213 activates the camera 211 and determines whether or not the auxiliary device 400 is normally attached to the photographing device 200. If it is determined that the auxiliary device 400 is not attached, the auxiliary device 400 is outputted (S113). Then, the processor 213 determines whether or not the auxiliary tool 400 is worn at a predetermined period while the auxiliary tool 400 wearing display screen is output, and if it is determined that the auxiliary tool 400 is worn normally, the processor 213 proceeds to the next process. Note that such a determination may be made using any processing method, but one example is a processing method that detects that a predetermined switch is turned on when the auxiliary device 400 is attached; Examples include a processing method of detecting the presence or absence of a pattern specific to an image photographed in a certain state, or a combination thereof. At this time, the processor 213 stores, for example, the following information in the memory 214 as the operation information or information used to generate the operation information.
・Wearing of the auxiliary device 400 used to count the number of times the auxiliary device 400 is worn ・Current time obtained by referring to the timer (through image display start time)
・Operation log

Next, when it is determined that the auxiliary device 400 has been worn normally, the processor 213 starts photographing the inside of the oral cavity including the pharynx using the camera 211 (S114). Specifically, photography is performed by turning on the light source 212 to irradiate the subject with light, and detecting reflected light from the subject, such as the pharynx, with the camera 211. At this time, the processor 213 stores, for example, the following information in the memory 214 as the operation information or information used to generate the operation information.
-Current time obtained by referring to the timer (lighting start time of the light source 212)
・Current time obtained by referring to the timer (through image display end time)
- Information that the light source 212 is turned on, which is used to count the number of times the light source 212 is turned on - Detected value from a current sensor connected to the light source 212 - Detected value from a temperature sensor installed around the light source 212 - Battery 218 The detection value from the current sensor installed around the light guide tube.The number of presses of the photograph button in response to the press of the photograph button.The number of times the subject image is taken in response to the press of the photograph button. Detection values and operation logs from temperature sensors

Next, when a subject image is photographed by photographing as described above, the processor 213 outputs the photographed subject image to the display via the output interface 215. When the processor 213 receives the operator's confirmation operation via the input interface 210, the processor 213 transmits the photographed subject image together with the subject ID information to the server device 300 via the communication interface 216 (S115). If an instruction for re-imaging is accepted instead of a confirmation operation, the process returns to S114 again. At this time, the processor 213 stores, for example, the following information in the memory 214 as the operation information or information used to generate the operation information.
-Current time obtained by referring to the timer (lighting end time of the light source 212)
・Receive a confirmation operation via the touch panel, and record the number of touch panel operations and operation log

Here, the battery 218 of the photographing device 200 is a rechargeable secondary battery. Therefore, for example, by connecting the photographing device 200 to a charging device (not shown) for charging, the above-described photographing process is temporarily charged, and charging of the battery 218 of the photographing device 200 can be started. It is possible. Therefore, when the processor 213 detects the connection to the charging device, it controls the battery 218 to start charging. At this time or while being charged, the processor 213 stores, for example, the following information in the memory 214 as the operation information or information used to generate the operation information.
・Detection of connection to the charging device ・Current time obtained by referring to the timer (time when use of the imaging device 200 ends)
- Current time obtained by referring to the timer (charging start time and charging end time for battery 218)
・Operation log

Next, the processor 213 reads each operation information stored in the memory 214 or the information used to generate the operation information from the memory 214 as described above (S116), and uses the read information to It is transmitted along with the device ID information to the server device 300 via the communication interface 216 (S117). Although not particularly illustrated, the transmitted operation information or the information used to generate the operation information is processed by the processor 312 of the server device 300 and is associated with the imaging device ID information as operation information. It is stored in the imaging device management table. Furthermore, although the processes related to reading and transmitting these information are performed at this timing in the example of FIG. 8, they may of course be performed at other timings. For example, it may be performed at any timing, such as each time the information is acquired, every predetermined time, the timing when communicating with the server device 300 for transmitting other information, or a combination thereof. With the above, the processing flow ends.

6. Process flow executed by server device 300
6-1. Determination Process FIG. 9A is a diagram showing a processing flow executed in the server device 300 according to an embodiment of the present disclosure. Specifically, FIG. 9A is a diagram showing a processing flow executed in the determination processing of S41 to S44 in FIG. 7. The processing flow is mainly performed by the processor 312 of the server device 300 reading and executing a processing program stored in the memory 311. In addition, in the following processing flow, although an example of operation information and related information is illustrated, it is not limited to only the illustrated information of course, and information other than these may be used.

According to FIG. 9A, the processor 312 receives, from the display device 100 via the communication interface 313, a request to determine the possibility of contracting a predetermined disease, along with the subject ID information of the subject selected as a diagnosis subject ( S211). Furthermore, the processor 312 stores the interview information and observation information input in advance by the subject or the operator on the display device 100 in association with the subject ID information in the subject management table. Therefore, based on the received subject ID information, the processor 312 refers to the interview information and observation information in the subject management table, and stores the interview information and observation information associated with the subject ID information of the subject. Read out (S212 and S213). Further, based on the similarly received subject ID information, the processor 312 refers to the subject images in the subject management table and reads out the subject image associated with the subject ID information of the subject (S214).

Next, the processor 312 uses the read interview information, finding information, and subject image to execute a process of determining the possibility of contracting a predetermined disease (S215).

Here, as an example of the above-described determination process, it is conceivable to input this information into a learned determination model as shown below and make a determination. However, it is of course possible to employ not only this processing method but also any processing method, such as a processing method in which a judgment is made based on the degree of matching with an image showing a diseased state through image analysis processing.

FIG. 9B is a diagram showing a processing flow related to generation of a trained model according to an embodiment of the present disclosure. Specifically, FIG. 9B is a diagram showing a processing flow related to generation of a learned determination model used in the determination process of S215 in FIG. 9A. The processing flow may be executed by the processor 312 of the server device 300, or may be executed by a processor of another device.

According to FIG. 9B, the processor 312 executes the step of acquiring a subject image that includes at least a portion of the pharynx (S311). Further, the processor 312 executes a step of acquiring interview information and finding information that are stored in advance in association with the subject ID information of the subject who is the subject of the subject image (S311). Next, the processor 312 performs a processing step in which a correct label is assigned in advance to the subject of the subject image based on the results of a rapid influenza test using immunochromatography, a PCR test, a virus isolation culture test, etc. (S312). Then, the processor 312 executes a step of storing the assigned correct label information as determination result information in association with the subject image, interview information, and finding information (S313). Note that although the subject image itself is used here, feature amounts obtained from the determination image may also be used.

When the subject image, the medical interview information, the finding information, and the correct label information associated therewith are obtained, the processor executes a step of performing machine learning of a disease determination pattern for the disease using them (S314). For example, this machine learning method provides a set of information to a neural network that combines neurons, and learns while adjusting the parameters of each neuron so that the output from the neural network is the same as the correct label information. This is done by repeating. Then, a step of acquiring a learned judgment model is executed (S315). The acquired learned determination model may be stored in the memory 311 of the server device 300 or in another device connected to the server device 300 via a wired or wireless network.

Next, returning to FIG. 9A again, the processor 312 stores the result determined by the determination process in the target person management table in association with the target person ID information, and also stores the stored determination result and the subject used in the determination. The image is transmitted to the display device 100 (S216). At this time, the processor 312 updates and stores the following information as the operation information of the photographing device management table in the memory 311, as an example (S217).
・Number of times the possibility of contracting a given disease is judged/Operation log

6-2. Related information output processing FIG. 9C is a diagram showing a processing flow executed in the server device 300 according to an embodiment of the present disclosure. Specifically, FIG. 9C is a diagram showing a processing flow executed in the related information output processing of SS19 to S21 and S45 to S47 in FIG. 7. The processing flow is mainly performed by the processor 312 of the server device 300 reading and executing a processing program stored in the memory 311. In addition, in the following processing flow, although an example of operation information and related information is illustrated, it is not limited to only the illustrated information of course, and information other than these may be used.

According to FIG. 9C, the processor 312 determines whether operation information or information used to generate the operation information has been received from the photographing device 200 via the communication interface 313 (S231). When the information used to generate the operation information is received, the operation information is generated based on the information (for example, when the lighting start time and the lighting end time of the light source 212 are received, the operation information is (calculating the lighting time of the light source 212 as information) and acquiring operation information.

Next, the processor 312 stores the operation information received in S231, the operation information generated in S231, and the operation information acquired in S217 of FIG. The information is updated and stored (S232). Then, the processor 312 refers to the related information table, reads related information that satisfies the conditions associated with the related information from each piece of operation information, and generates related information (S233). The processor 312 outputs the read related information to the display device 100 specified by the display device ID information associated with the image capturing device ID information and the image capturing device 200 specified by the image capturing device ID information (S234). .

An example of the correspondence between operation information or information used to generate operation information and related information is as follows. Note that the conditions such as numerical values described below and the information to be output are merely examples, and other conditions and information may of course be set.

[Information promoting the purchase of auxiliary tool 400]
- Operation information or information used to generate operation information: Information indicating that the auxiliary tool 400 is worn is acquired by the imaging device 200, and processed by the processor 312 in the server device 300 to determine the number of times the auxiliary tool 400 is worn. is stored as operation information. The information indicating that the auxiliary device 400 has been worn is obtained by the operator performing an operation input indicating that it was determined that the auxiliary device 400 was normally worn and that the attachment of the auxiliary device 400 was confirmed in S113 of FIG. 8, that the shooting button was pressed in S114 of FIG. 8, that the confirmation operation was accepted in S115 of FIG. 8, that the number of times the possibility of contracting a predetermined disease was determined in S215 of FIG. 9A, or a combination thereof. Generated based on. Note that when the auxiliary device 400 is purchased and the user inputs a reset operation, the count is reset and the count starts again from 1.
・Related information: Based on the number of wearing times stored as operation information, nothing will be output if the "number of wearing times" is from 0 to less than 100, and when it reaches the 100th and 150th times, a message will be displayed saying "Auxiliary devices are out of stock." From the 181st to the 200th time, the message "The stock of auxiliary tools is running low." is output each time the operation information is acquired, and from the 201st time onwards, the message "The stock of auxiliary tools is low" is output. Please do so.” is output. The related information output from the 201st time onwards includes a link to a web page where the auxiliary tool 400 is purchased, contact information for the purchase location, and the like.

[Information promoting physical key exchange, etc.]
- Operation information or information used to generate operation information: Information indicating that the power button has been pressed and information indicating that the shooting button has been pressed are acquired by the photographing device 200, and the server device 300 processes the processor 312. After the processing, the number of presses of each physical key is stored as operation information. Note that the count of the number of presses is reset when the physical key is replaced, for example, when the user inputs a reset operation, and the count is restarted from 1.
・Related information: Based on the number of presses of each physical key stored as operation information, each time a predetermined number of times, such as the t1th (e.g., 10,000th) or t2th (e.g., 20,000th) time, is reached. "The physical key may have deteriorated." is output, and after the t3th time (for example, 30,000 times), "Please replace the physical key. Please contact ○○ for replacement." is output, and information for notifying deterioration of the physical key and promoting at least one of replacement, inspection, and calibration is output.

[Information promoting touch panel replacement, etc.]
- Information used to generate operation information or operation information: information indicating that a tap, drag, or swipe operation has been performed on the touch panel for a selection operation or a confirmation operation is acquired via the input interface 210 of the photographing device 200. In the server device 300, the number of times the touch panel is operated is stored as operation information after processing by the processor 312. Note that when the touch panel is replaced or the like and the user inputs a reset operation, the count is reset and the count starts again from 1.
・Related information: Based on the number of touch panel operations stored as operation information, each time a predetermined number of times such as t4th (for example, 50,000th) or t5th (for example, 100,000th) is reached, " The touch panel sensor may have deteriorated.'' is output, and after the 6th time (for example, 300,000 times), the message ``Please replace the touch panel.Please contact ○○ for replacement.'' is output. and outputs information for notifying deterioration of the touch panel and promoting at least one of replacement, inspection, and calibration.

[Information promoting replacement of light source 212, etc.]
- Operation information or information used to generate operation information: The lighting start time and lighting end time of the light source 212 are acquired in the photographing device 200 from the current time information obtained by referring to the timer, and the server device 300 acquires the lighting start time and the lighting end time. After the process of 312, the lighting time of the light source 212 is stored as operation information. Further, an output of an on signal to the light source 212 is acquired in the imaging device 200, and the number of times the light source 212 is turned on is stored as operation information through processing by the processor 312 in the server device 300. Further, a detected value from a current sensor connected to the light source 212 of the photographing device 200 is acquired in the photographing device 200, and the detected value is stored as operation information in the server device 300. Further, a detection value from a temperature sensor installed around the light source 212 is acquired by the photographing device 200 and stored as operation information in the server device 300. Incidentally, when the light source 212 is replaced or the like and the user inputs a reset operation, the count is reset and the count starts again from 1 for the lighting time and the number of times the light source 212 is lit.

・Related information: Based on the lighting time stored as operation information, each time the cumulative lighting time reaches a predetermined time such as s1 hours (for example, 1,000 hours), s2 hours (for example, 2,000 hours), " "The LED of light source 212 may have deteriorated." is output, and after s3 hours (for example, 10,000 hours) "Please replace the LED of light source 212. Please contact ○○ for replacement. ”, and information for notifying the deterioration of the light source 212 and promoting at least one of replacement, inspection, and calibration is output. Also, based on the number of lighting times stored as operation information, each time a predetermined number of times such as t7th (for example, 10,000th time), t8th (for example, 20,000th time), etc. are reached, "LED of light source 212 "There is a possibility of deterioration." is output, and after the t9th time (for example, 100,000 times), "Please replace the LED of light source 212. Please contact ○○ for replacement." is output. and outputs information for notifying deterioration of the light source and promoting at least one of replacement, inspection, and calibration. In addition, the detected value from the current sensor stored as operation information is compared with a predetermined threshold (for example, 200 mA; there may be multiple thresholds such as an upper limit and a lower limit), and when the value exceeds the threshold. If the current is above or below the current level, it is determined that an abnormal current is flowing, and the message ``Please check the LED of light source 212'' or ``Replace the LED of light source 212.'' For replacement, please contact ○○. ” is output, and information for notifying the deterioration of the LED of the light source 212 and promoting at least one of replacement, inspection, and calibration is output. In addition, the detected value of the temperature sensor stored as operation information is compared with a predetermined threshold value (for example, 60 degrees Celsius), and if the threshold value is exceeded, the LED of the light source 212 is notified of deterioration, replaced, or inspected. Information for facilitating at least one of the above and the calibration is output. In addition, the information such as the lighting time, the number of lighting times, the detected value from the current sensor, and the temperature stored as the operation information above is stored in chronological order as historical data, and the light source 212 is adjusted based on the comparison with past historical data. The deterioration of the LED may be notified, or the predicted time of deterioration may be notified. Such a notification can be generated by determining deterioration or predicting the timing of deterioration using a learned prediction model obtained by machine learning using historical data as input data.

[Information promoting replacement of battery 218, etc.]
- Operation information or information used to generate operation information: Information indicating that connection of the battery 218 to the charging device has been detected is acquired, and is processed by the processor 312 in the server device 300 to generate the previous information. The cumulative number of charging times is stored as operation information. In addition, the current amount of charge is obtained from the integrated value of the amount of current charged and the amount of current discharged by the current sensor connected to the battery 218, and the detected value from the voltage sensor connected to the battery 218 is acquired by the imaging device. 200, and the detected value from the voltage sensor at the time of full charge is stored as operation information in the server device 300. In addition, a detected value of the cell voltage of the battery 218 is acquired in the photographing device 200 by a voltage sensor connected to the battery 218, and the detected value is processed by the processor 312 in the server device 300, resulting in an overcharge state in which an excessive voltage is applied. The number of times this is displayed is stored as operation information. Conversely, the number of times an over-discharge state, which is a state of excessive discharge, has been displayed is stored as operation information. Further, from the current time information obtained by referring to the timer, the charging start time and charging end time for the battery 218 are acquired in the photographing device 200, and the cumulative charging time of the battery 218 is obtained by processing by the processor 312 in the server device 300. is stored as operation information. Further, a detected value from a temperature sensor installed around the battery 218 is acquired by the photographing device 200 and stored as operation information in the server device 300. Note that the number of charging times, the number of overcharging and overdischarging, and the cumulative charging time are reset when the battery 218 is replaced, for example, when the user inputs a reset operation, and the count is restarted from 1. Ru.

・Related information: Based on the number of charging times, overcharging times, or overdischarging times stored as operation information, at a predetermined number of times, such as the t10th (for example, 500th time), t11th (for example, 1,000th time), etc. Each time it reaches the limit, the message "Battery 218 may have deteriorated." is output, and after the t12th (for example, 2,000th) time, "Please replace the battery 218. For replacement, please contact ○○." ” is output, and information for notifying the deterioration of the battery 218 and promoting at least one of replacement and inspection is output. Note that the conditions for outputting the related information are set to different times for each of the number of times of charging, the number of overcharging, and the number of overdischarging. In addition, the detected value from the voltage sensor at full charge stored as operation information is compared with a predetermined threshold (for example, 3.7V), and if the value is lower than the threshold, the battery has deteriorated and cannot be fully charged. Please check the battery 218 or replace the battery 218. For exchange, please contact ○○. ” is output, and information for notifying the deterioration of the battery 218 and promoting at least one of replacement and inspection is output. In addition, based on the cumulative charging time stored as operation information, each time the cumulative charging time reaches a predetermined time such as s4 hours (for example, 500 hours) or s5 hours (for example, 1,000 hours), the message "The battery 218 "There is a possibility of deterioration." is output, and after s6 hours (for example, 2,000 hours), "Please replace the battery 218. Please contact ○○ for replacement." is output, and the battery 218 Deterioration of the device is notified, and information for promoting at least one of replacement and inspection is output. In addition, the detected value of the temperature sensor stored as operation information is compared with a predetermined threshold value (for example, 50 degrees Celsius), and if the threshold value is exceeded, the battery 218 is notified of deterioration or replaced or inspected. Information for promoting at least one of the above is output. In addition, information such as the number of charging times, the number of overcharging times, the number of overdischarging, the detected value from the voltage sensor, the charging time, the detected value from the temperature sensor, etc. stored as the operation information above is stored in chronological order as historical data, Deterioration of the battery 218 may be notified based on comparison with past historical data, or a predicted timing of deterioration may be notified. Such a notification can be generated by determining deterioration or predicting the timing of deterioration using a learned prediction model obtained by machine learning using historical data as input data.

[Information to assist in charging the battery 218]
- Operation information or information used to generate operation information: from the integrated value of the amount of current charged and the amount of current discharged by the current sensor connected to the battery 218, or the output of the voltage sensor connected to the battery 218 The current remaining charge amount is acquired from the value, and the remaining charge amount is stored in the server device 300 as operation information. Further, an operation log of the photographing device 200, information indicating the location of use, and current time information are acquired by the photographing device 200 and stored as operation information in the server device 300.
- Related information: From the operation log stored as operation information, information indicating the location of use, and current time information, predicted values of usage frequency and power consumption for each day of the week are calculated for each institution. Then, a decrease in the amount of charge of the battery 218 is predicted based on the remaining charge amount obtained as the operation information and the predicted value. Based on the results, if it is predicted that the charge amount of the battery 218 will be less than the specified amount (for example, 500 mAh) during medical treatment hours at a medical institution such as a hospital, related information will be displayed indicating that the battery 218 is connected to a charging device. Please start charging.'' and information to facilitate charging is output. In medical institutions, etc., it would be a serious problem if the imaging device 200 becomes unusable due to a decrease in the remaining capacity of the battery 218 during medical treatment hours, so such a problem can be solved by notifying the patient in advance.

The predicted values of usage frequency and power consumption for each day of the week for each institution are learned predictions obtained by machine learning using the past history of usage frequency and power consumption for the same day of the week by the same institution as input data. It is possible to calculate using a model. However, you are not limited to this method; any method can be used, such as a method that uses the maximum usage frequency, maximum power usage, average usage frequency, average power usage, etc. for each day of the week as predicted values from past history. It is.

[Information to assist in charging the battery 218]
- Operation information or information used to generate operation information: A detected value from a voltage sensor connected to an AC adapter for supplying power from the charging device to the battery 218 is acquired in the imaging device 200, and is acquired in the server device 300. The detected value is acquired as operation information.
・Related information: The time required to fully charge is calculated from the voltage sensor detection value stored as operation information and the predetermined battery capacity, and a notification stating "It will take XX hours to fully charge." is related. Output as information.

[Information promoting replacement of charging device, etc.]
- Information used to generate operation information or operation information: A detected value from a voltage sensor or a current sensor connected to an electric wire for supplying power from the charging device to the battery 218 is acquired in the imaging device 200, and is transmitted to the server device. At 300, the detected value is acquired as operation information.
・Related information: The detected value of the voltage sensor or current sensor stored as operation information and a predetermined threshold (for example, 0.5A as the threshold of the current sensor. Also, there may be multiple thresholds such as an upper limit and a lower limit) ), and if the threshold value is exceeded or lower than the threshold, information notifying that the charging device has deteriorated or that a charging device that does not meet the recommended standards is connected is output as related information. Note that, at this time, for example, to connect the charging device and the photographing device 200, either a cable capable of both data communication and power supply, or a cable capable of only power supply is generally used. Therefore, which cable was used for connection may be detected and output as related information. For example, if a cable that can only supply power is used, data communication cannot be performed via the cable, so the usability is further improved by outputting it as related information.

[Information promoting replacement of CMOS image sensor, etc.]
- Operation information or information used to generate operation information: The display start time and display end time of the through image are acquired in the imaging device 200 from the current time information obtained by referring to the timer, and the server device 300 acquires the display start time and display end time. After the process of step 312, the cumulative usage time of the CMOS image sensor is stored as operation information. Further, information indicating that the photographing button has been pressed is acquired by the photographing device 200, and processed by the processor 312 in the server device 300, and the number of photographing times is stored as operation information. Incidentally, when the CMOS image sensor is replaced or the like, and the user inputs a reset operation, the cumulative usage time and the number of times of photographing are reset, and counting starts again from 1.
・Related information: Based on the cumulative usage time of the CMOS sensor stored as operation information, each time the cumulative usage time reaches a predetermined time such as s7 hours, s8 hours, etc., a message saying "The CMOS image sensor may deteriorate." ” is output, and after s9 hours, “Please replace the CMOS image sensor. Please contact ○○ for replacement.” is output, and the CMOS image sensor is notified, or at least one of replacement and inspection is output. Information to promote this will be output. Also, based on the number of shooting times stored as operation information, each time a predetermined number of shots is reached, such as t13th, t14th, etc., the message "The CMOS image sensor may deteriorate." is output, and after the t15th time, "CMOS image sensor" is output. Please replace the image sensor. For replacement, please contact be done.

Note that when outputting information promoting replacement of the CMOS image sensor or the light source 212 as related information, a subject image photographed by the photographing device 200 may be used as the operation information. By analyzing the subject image, changes in color, saturation, brightness, and pixel loss are determined, and related information is sent to ○○ for replacement. Please contact us.'' and information for facilitating at least one of replacement and inspection of the CMOS image sensor and the light source 212 is output. In addition, for this judgment, the subject image is applied to a trained judgment model obtained by machine learning using the subject image for learning and appropriate or inappropriate label information for each subject image as input data. This can be done by inputting the information. However, the method is not limited to this method, and other image analysis methods may be used, such as analyzing the degree of similarity with an appropriate subject image.

[Information promoting replacement of various lenses and light guide tubes, etc.]
- Operation information or information used to generate operation information: From the current time information obtained by referring to the timer, the start time and end time of use of the imaging device 200 are acquired in the imaging device 200, and the server device 300 In response to processing by the processor 312, the cumulative usage time of the photographing device 200 is stored as operation information. Further, the detection value from a temperature sensor installed around the light guide tube is acquired by the photographing device 200, and is stored as operation information in the server device 300. Further, from the current time information obtained by referring to the timer, the lighting start time and lighting end time of the light source 212 are acquired in the photographing device 200, and the lighting time of the light source 212 is processed by the processor 312 in the server device 300. It is stored as operation information. Note that the cumulative usage time is reset when the various lenses and light guide tubes are replaced, for example, and the user inputs a reset operation, and the count is restarted from 1.

- Related information: Based on the cumulative usage time of the photographing device 200 stored as operation information, each time the cumulative usage time reaches a predetermined time such as s10 hours, s11 hours, etc. "Please replace the various lenses and light guide tubes. Please contact ○○ for replacement." is output after s12 hours, indicating that the various lenses and light guide tubes have deteriorated. Information for notifying or promoting at least one of replacement and inspection is output. Note that it is also possible to set a different number of times for each type of lens or for each light guide tube. In addition, related information includes information that compares the detected value of the temperature sensor stored as operation information with a predetermined threshold value (for example, 120 degrees Celsius), and notifies that abnormal heat generation is observed when the threshold value is exceeded. is output as Further, for example, the light guide tube is easily affected by light and heat from the light source 212. Therefore, it is also possible to output related information by further combining the lighting time of the light source 212 acquired as described above with the information on the cumulative usage time described above.

[Information that promotes inspection of the photographing device 200, etc.]
- Operation information or information used to generate operation information: A detected value from the acceleration sensor is always acquired by the imaging device 200 and stored as operation information in the server device 300.
・Related information: The detection value of the acceleration sensor stored as operation information is compared with a predetermined threshold value, and if the value exceeds the threshold value, the camera is judged to have been used abnormally such as falling or colliding. Information prompting inspection etc. of 200 is output as related information. In addition, such notifications can be generated by determining abnormal detected values using a trained prediction model obtained by machine learning using historical data of detected values from acceleration sensors as input data. be. Further, the operation information itself is output in order to estimate the cause of a failure of the photographing device 200 at the time of inspection.

[Information that promotes inspection of the photographing device 200, etc.]
- Operation information or information used to generate operation information: The detected value from a temperature sensor installed at a heat generating part of the imaging device 200 (for example, around the light source 212, a control board on which the processor 213, etc. is mounted) The information is constantly acquired by the photographing device 200 and stored as operation information in the server device 300.
- Related information: Compare the detected value of the temperature sensor stored as operation information with a predetermined threshold, and if it exceeds the threshold, check the imaging device 200 as there is a risk of abnormal heat generation, etc. The prompting information is output as related information. In addition, such notifications can be generated by determining abnormal detected values using a trained prediction model obtained by machine learning using historical data of detected values from temperature sensors as input data. be. Further, the operation information itself is output in order to estimate the cause of a failure of the photographing device 200 at the time of inspection.

[Information that assists in improving the communication status of the photographing device 200]
- Operation information or information used to generate operation information: The communication speed of communication performed with the display device 100 or the server device 300 via the communication interface 216 of the photographing device 200 and the detected value of the radio field intensity during the communication The information is acquired by the imaging device 200, processed by the processor 312 of the server device 300, and stored as operation information.
・Related information: Compare the communication speed or radio field strength stored as operation information with a predetermined threshold (for example, 1 Mbps for communication speed), and if it exceeds the threshold, it is determined that the communication state is not good and the communication state is determined. Information on how to improve is output as related information.

[Information that assists in operating the imaging device 200]
- Operation information or information used to generate operation information: The communication speed of communication performed with the display device 100 or the server device 300 is acquired in the camera device 200 via the communication interface 216 of the camera device 200, and the communication speed is acquired by the camera device 200. The information is processed by the processor 312 and stored as operation information.
- Related information: Based on the communication speed stored as operation information, the expected waiting time when transmitting the subject image etc. from the display device 100 to the server device 300 is calculated, and the calculated expected waiting time is output as related information. be done.

[Information to assist in the use of the aid 400]
- Operation information or information used to generate operation information: Information indicating that the auxiliary tool 400 has been removed or not is acquired by the imaging device 200 and processed by the processor 312 in the server device 300. The information is stored as operation information.
- Related information: Based on the information stored as operation information indicating that the auxiliary device 400 has been removed or has not been removed, it is determined that the auxiliary device 400 is still attached, and the auxiliary device 400 is not removed. If it is determined that it remains the same, related information indicating this is generated. By outputting information that promotes the removal of the used auxiliary tool 400 in this way, it is possible to prevent infection between subjects.

[Information that assists in proper operation of the imaging device 200]
- Operation information or information used to generate operation information: The operation log of the photographing device 200, information indicating the location of use, and current time information are acquired by the photographing device 200 and stored as operation information in the server device 300.
- Related information: When an abnormal operation is detected from the operation log stored as operation information, information regarding an operation method to improve it is output as related information. As an example, the subject image is transmitted to the server device 300 by a confirming operation after the subject image is captured in photographing processing, but the number of times a canceling operation is performed instead of the confirming operation is predetermined. If the number of times is greater than a threshold value (for example, three times), information to assist in photographing, such as "Be sure to firmly fix the photographing device 200 so that it does not move during photographing." is output as related information. In addition, if the lighting time of the light source 212 or the display time of the through image is longer than a predetermined threshold (for example, 3 minutes as the lighting time of the light source), the message "Turn off the power to the imaging device 200 after shooting?" , "Let's put the imaging device 200 upright and put it into a sleep state." Information that assists in the use of the imaging device 200 is output as related information. Note that these times and times are reset by the operator inputting a predetermined operation or by outputting related information, and are counted again from 1.

Note that the above threshold value can be calculated using a learned prediction model obtained by machine learning using the history of past operation logs of the same institution on the same day of the week as input data. However, it is not limited to this method, and any method can be used, such as a method using the maximum value, minimum value, or average value for each day of the week from past history as a threshold value.

[Information that assists in proper operation of the imaging device 200]
- Operation information or information used to generate operation information: A subject image photographed by the photographing device 200 is acquired by the photographing device 200, and is also stored as operation information in the server device 300.
・Related information: By analyzing the subject images stored as operation information, for example, if it is determined that there are many images with inappropriate angles of view or images with a lot of blur, the related information will be superimposed on the through image and the subject image will be displayed. A guide display is output to guide the position of the camera, and assistance is provided in order to capture a good image of the subject, such as "Be sure to fix the camera device 200 firmly so that it does not move during shooting." Information is output as related information. In addition, by analyzing the subject image, we can determine the subject's condition and shooting environment, such as whether the subject's mouth is not open enough, and depending on the results, we can tell you to "open your mouth a little wider." Notifications to the target person, such as "Please maintain this state.", are output as related information.

In addition, for these judgments, operation information is added to a trained judgment model obtained by machine learning using subject images for learning and appropriate or inappropriate label information for each subject image as input data. This can be done by inputting the subject image obtained as follows. However, the method is not limited to this method, and other image analysis methods may be used, such as analyzing the degree of similarity with an appropriate subject image. In addition, especially when notifying a target person, instead of outputting the related information to the display device 100 or in addition to outputting it to the display device 100, the related information may be output to the display of the photographing device 200. Alternatively, it may be output as audio from the photographing device 200 or the display device 100.

[Information that assists in proper operation of the imaging device 200]
- Operation information or information used to generate operation information: A detected value from a temperature sensor installed around the light source 212 (for example, a board on which an LED of the light source 212 is installed) is acquired in the imaging device 200, and is sent to the server. The information is stored in the device 300 as operation information.
・Related information: Compare the detected value of the temperature sensor stored as operation information with a predetermined threshold value, and if it exceeds the threshold value, the LED of the light source 212 may be at a high temperature. Please refrain from operating the device.'' A notification is output as related information.

[Information regarding fees for determining the possibility of contracting a given disease]
- Operation information or information used to generate operation information: When the result of determining the possibility of contracting a predetermined disease in S216 of FIG. 9A is transmitted, the operation information is stored in the server device 300 as the number of determinations. Ru. Note that the count of the number of determinations is reset when processing related to determination fee calculation and settlement is performed, and counting starts again from 1.
- Related information: When the processor 312 of the server device 300 reads out the number of determinations as operation information, it calculates the determination fee for each display device ID information. Specifically, the processor 312 calculates the determination fee by multiplying the preset fee per determination by the number of determinations. Then, when the determination fee for each display device ID information is calculated, the processor 312 refers to the operator management table and specifies the operator ID information with which the display device ID information is associated. Next, the processor 312 adds up all the determined fees for the display device ID information associated with the operator ID information, and calculates the determined fee for each operator ID information. For example, if two pieces of display device ID information are associated with the operator ID information, the respective determination fees calculated for the two pieces of display device ID information are added up.

When the judgment fee is calculated for each operator ID information, information notifying the calculated judgment fee and information notifying the payment method of the judgment fee are generated as related information, and are associated with the operator ID information. The information is transmitted to the display device 100. Note that, at this time, the summation is performed for each operator ID information, but the processor 312 can also refer to the operator management table and further summation for each institution ID information. By doing so, it becomes possible to bill a judgment fee for each medical institution or department, for example, and it is possible to further improve convenience.

That is, a processing device such as the server device 300 is communicably connected via a network to a photographing device configured to photograph an image of a subject's natural orifice, and the processing device includes at least one processor. A device,
The at least one processor,
Determining the possibility of contracting a predetermined disease based on the image taken by the imaging device,
Outputting the result of the determination or information generated based on the result of the determination to assist in diagnosing the disease;
calculating a usage fee to be charged to the operator of the photographing device or the institution to which the operator belongs according to at least one of the number of determinations or the number of outputs;
Outputting information indicating the calculated usage fee,
is configured to perform processing for

Note that in the above, for example, if it is possible to determine the possibility of contracting a plurality of diseases, when receiving the determination request from the display device 100, information specifying the disease to be determined is also received. The processor 312 of the server device 300 can store the determined fee for each determined disease in advance, and calculate the usage fee for each determined disease in the calculation of the usage fee described above.

Further, as operation information, an operation log of the photographing device 200 or the number of determinations, information indicating the location of use, and current time information are acquired in the photographing device 200 and stored as operation information in the server device 300. In such a case, it is possible to generate a learned judgment frequency prediction model by performing machine learning using the history of usage frequency for each predetermined period by each engine or each operator as input data. The processor 312 can obtain a predicted value of the number of determinations by inputting the institution ID information or operator ID information and the period into the learned determination number prediction model. Therefore, the processor 312 calculates the predicted value of the judgment fee for each institution or each operator in the input period from the obtained predicted value of the number of judgments. The processor 312 outputs the obtained predicted value as related information. By doing so, it becomes possible to calculate a predicted value for each institution or each operator and secure the fee in advance as a budget, thereby further improving convenience.

Furthermore, various methods can be used to output usage fees. For example, it is also possible to add up usage fees for each predetermined period such as daily, weekly, monthly, and yearly and output the sum. Further, for example, if the usage fee has been paid in advance, it is also possible to output the current remaining prepaid fee. Further, for example, in a case where a fixed amount is to be paid every predetermined period regardless of the number of times of determination, it is also possible to output a comparison between the determined fee and the case where the fee is paid each time.

[Other related information]
In addition to the related information exemplified above, the operation information exemplified above is used to notify the deterioration of the display, notify at least one of replacement, inspection, and calibration, and notify the timing of maintenance of the imaging device 200. Various related information such as notifications may be output.

6-3. Processing program update processing FIG. 9D is a diagram showing a processing flow executed in the server device 300 according to an embodiment of the present disclosure. Specifically, FIG. 9D is a diagram showing a processing flow executed in a process for updating the processing program installed as the SW of the photographing device 200. The processing flow is mainly performed by the processor 312 of the server device 300 reading and executing a processing program stored in the memory 311.

Here, each device such as the processing system 1 or the imaging device 200 and the processing program are devices that are subject to regulations or approvals specified by predetermined laws, regulations, government orders, guidelines, etc. (for example, drugs, medical devices, etc.). Act on Assuring Quality, Efficacy, and Safety (so-called Pharmaceutical Devices Act), Federal Food, Drug, and Cosmetic Act, Medical Device Amendment Act, European Medical Device Regulations (Medical Device Regulation), In Vitro Diagnostic Regulation, Medical Device Regulations of the State Council of China No. 739), contraindications and prohibitions, Related documents such as published precautions information that describes the principle of the device, how to use it, precautions for use, etc., package inserts, history of revision instructions, review reports, reexamination reports, emergency safety information, etc. Created. Then, when these related documents are revised or issued, it is necessary to promptly notify the operator etc. of the revision or issuance, so the fact that the revision or issuance has been made is notified as related information.

Specifically, the processor 312 determines whether revision or issuance information of a related document such as an attached document has been received via the communication interface 313. If the information has been received, the processor 312 generates, as related information, information indicating that a related document such as an attached document has been revised or issued. Further, the display device 100 stores links that are stored in advance to obtain related documents that have not yet been revised or issued, and a recording medium such as a two-dimensional code in which the links are recorded. Therefore, the processor 312 displays the content stored in the display device 100 such as a link where the related document after revision or publication is obtained as related information, and the link is recorded as a recording medium such as a two-dimensional code. It is possible to include a processing program for updating. The processor 312 then outputs the relevant information to the display device 100, the photographing device 200, or a combination thereof via the communication interface 313.

In addition to this, the processing program installed as the SW of the imaging device 200 may need to be updated due to revisions of related documents such as attached documents. Therefore, a case will be described below in which the processor 312 executes processing related to updating a processing program specified in the relevant document due to the revision of the relevant document.

Specifically, according to FIG. 9D, the processor 312 determines whether revision information of a related document such as an attached document has been received via the communication interface 313 (S241). When the processor 312 has received revision information of a related document such as an attached document, the processor 312 refers to the photographing device management table to refer to the SW version information of the currently stored processing program (S242). Next, the processor 312 compares the SW version information of the processing program specified by the received revision information with the stored SW version information, and determines whether the processing program needs to be updated. As a result, if the two are different, the processor 312 determines that an update is necessary, and provides related information that prompts the processing program to be updated (for example, "The attached document has been revised. Download the latest SW from below. ) (S233). Once the information is generated, the processor 312 outputs the information to the display device 100, the photographing device 200, or a combination thereof via the communication interface 313 (S234). With the above, the processing flow ends.

That is, a processing device such as the server device 300 is communicably connected via a network to a photographing device configured to photograph an image of a subject's natural orifice, and the processing device includes at least one processor. A device,
The at least one processor,
Store version information of the processing program installed in the imaging device in memory,
At least one of the processing device, the imaging device, and the processing program obtains version information specified in a notification resulting from a related document created for predetermined regulation or approval;
If the acquired version information differs from the version information stored in the memory, outputting information prompting an update to the processing program specified by the acquired version information;
is configured to perform processing for

Here, we have explained the case where the revision or issuance of related documents such as attached documents is notified as related information. However, the notification is not limited to the revision or issuance of related documents, and other received information may be notified as related information. For example, it is also possible to output notifications of recalls or defects in various components (parts, etc.) constituting the photographing device 200 as related information. Specifically, the processor 312 of the server device 300 receives defect information regarding various components from other server devices etc. via the communication interface 313. The processor 312 then refers to the parts information in the imaging device management table in the memory 311 based on the manufacturing lot included in the received defect information. If the serial number stored as part information is included in the manufacturing lot, the processor 312 performs at least one of replacement, inspection, and calibration of the corresponding component (part) of the imaging device 200. Generate prompt notifications with relevant information. The processor 312 then outputs the generated related information to the display device 100, the photographing device 200, or a combination thereof via the communication interface 313.

7. Output of related information FIGS. 10A to 10E are diagrams showing examples of screens displayed on the display device 100 according to an embodiment of the present disclosure. Specifically, FIG. 10A is a screen showing a list of subjects displayed on the display device 100, in which subjects for determining the possibility of contracting a predetermined disease, subjects for inputting interview information, and subjects for inputting finding information are displayed. This is an example of a screen displayed when selecting a target person. Further, FIGS. 10B to 10E are examples of screens on which related information is displayed on the display device 100.

According to FIG. 10A, a subject list screen is displayed on the display via the output interface 114. This screen includes a list screen 11, and on the list screen 11, information on people who have left the facility, such as the current status, name of the person, ID information of the person, and attribute information, is displayed as a list in a line-by-line format for each person. There is. In addition, at the bottom of the list screen 11, there is an interview input icon 12 for inputting interview information, a diagnosis icon 13 for inputting finding information and sending a judgment request, and a new icon for newly registering a new subject. A registration icon 14 is displayed. By performing operation input on each of these icons, it is possible to execute processing corresponding to each icon.

In this way, the subject list screen shown in FIG. 10A is a screen that is frequently referenced by the user when making a determination using the processing system 1.

Next, according to FIG. 10B, a case is described in which information promoting the purchase of the auxiliary tool 400 is received and output as related information. As shown in FIG. 10B, the related information display 15 is displayed superimposed on the subject list screen.

Similarly, according to FIG. 10C, a case is described in which information promoting updating of a processing program is received and output as related information. As shown in FIG. 10C, the related information display 17 is displayed superimposed on the subject list screen.

Similarly, according to FIG. 10D, a case is described in which information that assists the operation of the photographing device 200 is received and output as related information. As shown in FIG. 10D, the related information display 18 is displayed superimposed on the subject list screen.

In this way, in the examples shown in FIGS. 10B to 10C, by outputting related information on the target person list screen, which is a frequently referenced screen, it is possible to draw more attention to the user. . Note that the output method mentioned here is just an example, and of course output may be performed using other methods. For example, when related information is received, a notification may be displayed at the top of the screen, and by clicking on the notification, the screen may be switched and detailed information thereof may be output. Furthermore, when the relevant information is received, the reception may be notified by sound, LED, vibration, or a combination thereof, and the detailed information may be output by opening a predetermined application program.

According to FIG. 10E, a case is described in which information indicating a determination fee is received and output as related information. According to FIG. 10E, the usage fee for each display device 100 that has transmitted the determination request and the total amount of usage fees for the institutions to which these display devices 100 are associated are displayed on the usage fee display screen 19. Furthermore, a payment icon 20 and a cancel icon 21 are displayed at the bottom of the usage fee display screen. By accepting an operation input to the payment icon, the screen moves to a screen where information indicating the payment method output as related information is displayed, and it becomes possible to proceed with the payment process.

In this way, by displaying the judgment fee on the frequently used display device 100, it is possible to eliminate failure to grasp the judgment fee and further improve convenience.

Note that the output destinations of the related information shown in FIGS. 10B to 10C do not need to be displayed only on the display device 100, and may also be output to, for example, the photographing device 200 or other terminal device owned by a pre-registered operator. good.

As described above, the present embodiment provides a processing device, a processing program, and a processing method that are connected to a photographing device configured to photograph an image of a subject and can improve the usability of the photographing device. I can do it.

8. Variant

In the above embodiments, the case has been described in which the interview information and finding information are input in advance by the operator or the subject, or are received from an electronic medical record device or the like connected to a wired or wireless network. However, instead of or in addition to these, this information may be obtained from a photographed subject image. Further, the medical interview information and observation information may be written in advance on a paper medium or the like by hand or in a mark sheet format, and may be acquired by reading this using an optical reading device such as a camera or a scanner. In addition, even if the interview information and finding information entered on a separately provided terminal device, etc., or their storage location are recorded on a recording medium such as a two-dimensional code, and this information is acquired via the recording medium. good. Finding information and interview information associated with the learning subject image are given as correct labels to the learning subject image, and a learned information estimation model is obtained by machine learning these sets using a neural network. Then, the processor 111 inputs the subject image to the learned information estimation model, thereby obtaining desired interview information and attribute information. Examples of such interview information and attribute information include gender, age, degree of redness of the throat, degree of swelling of tonsils, presence or absence of white moss, and the like. This saves the operator the trouble of inputting medical interview information and finding information.

Furthermore, in the above embodiment, a case has been described in which the related information is generated based on the operation information, but the related information can also be generated by further considering information other than the operation information. For example, it is possible to further take into consideration information about the approximate replacement timing of various components that is registered in advance at the time of manufacturing or maintenance of the photographing device 200. Although the timing for outputting the related information has not yet been reached based on the operation information, the related information may be output when the estimated time is reached.

Each trained model described in the above embodiments was generated using a neural network or a convolutional neural network. However, the information is not limited to these, and it is also possible to generate it using machine learning such as the nearest neighbor method, decision tree, regression tree, and random forest.

In the above embodiment, a case has been described in which the server device 300 performs determination processing and output processing of related information. However, these various processes can be appropriately distributed and processed by the display device 100, the photographing device 200, other devices (including a cloud server device, etc.), and the like.

In the above embodiment, the location of use was obtained through GPS. However, this is just one example of acquiring the place of use; for example, it may be acquired by acquiring institution ID information and specifying the place associated with the institution ID information.

Note that these modified examples are the same as the configuration, processing, and procedure of the embodiment described in FIGS. 1 to 10E, except for the points specifically described above. Therefore, detailed explanation of those matters will be omitted. It is also possible to configure a system by appropriately combining or replacing the elements described in each modification and each embodiment.

The processes and procedures described herein can be implemented not only by what is explicitly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described in this specification can be realized by implementing logic corresponding to the processes in a medium such as an integrated circuit, volatile memory, nonvolatile memory, magnetic disk, or optical storage. be done. Further, the processes and procedures described in this specification can be implemented as computer processing programs and executed by various computers including display devices and server devices.

Even if processes and procedures described herein are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, software, components, or modules. It may be implemented by multiple components and/or multiple modules. Further, even if it is explained that the various information described in this specification is stored in a single memory or storage unit, such information may be stored in multiple memories or storage units provided in a single device. The information may be stored in a distributed manner in a plurality of memories distributed and arranged in a plurality of devices. Additionally, the software and hardware elements described herein may be implemented by integrating them into fewer components or decomposing them into more components.

1 processing system 100 display device 200 photographing device 300 server device 400 auxiliary tool

Claims

A processing device communicably connected via a network to a photographing device configured to photograph an image of a natural aperture of a subject, the processing device including at least one processor,
the at least one processor,
obtaining operation information related to an operation of the photographing device performed by an operator of the photographing device in order to photograph the image with the photographing device;
outputting relevant information related to the photographing device based on the acquired operation information;
A processing device configured to perform processing for.
The operation information is information indicating the number of operations on the photographing device,
The related information is information for promoting at least one of replacement, inspection, and calibration of the components of the photographing device, which is generated based on the operation information.
The processing device according to claim 1.
The operation information is information detected by a sensor arranged in the photographing device,
The related information is information for promoting at least one of replacement, inspection, and calibration of the components of the photographing device, which is generated based on the operation information.
The processing device according to claim 1.
The image is captured by the imaging device using an auxiliary tool that is inserted into the natural orifice together with the imaging device, and which is recommended to be replaced each time it is inserted. Photographed,
The related information is information for promoting the purchase of the auxiliary tool,
The processing device according to claim 1.
The operation information is information indicating the state of a battery required for operation of the photographing device,
The related information is information for facilitating at least one of replacing and inspecting the battery,
The processing device according to claim 1.
The operation information is information indicating the state of a battery required for operation of the photographing device,
The related information is information indicating the charging status of the battery,
The processing device according to claim 1.
The processing device according to claim 4 or 5, wherein the information indicating the state of the battery is information detected by a voltage sensor or a current sensor connected to the battery.
The related information is information indicating a lighting state of a light source for irradiating light into the natural opening when photographing the image,
The related information is information for facilitating at least one of replacement, inspection, and calibration of the light source.
The processing device according to claim 1.
The processing device according to claim 1, wherein the related information is auxiliary information for assisting the operation by the operator based on the operation information.
The processing device is communicably connected via the network to a display device for inputting subject information related to the subject,
the related information is output to the display device;
The processing device according to claim 1.
A processing device communicatively connected via a network to a photographing device configured to photograph an image of a subject's natural aperture as a subject;
obtaining operation information related to an operation of the photographing device performed by an operator of the photographing device in order to photograph the image with the photographing device;
outputting relevant information related to the photographing device based on the acquired operation information;
A processing program that functions as a processor for
in a processing device that is communicably connected via a network to an imaging device configured to capture an image of a natural aperture of a subject and includes at least one processor; A processing method comprising:
the at least one processor,
acquiring operation information related to an operation of the photographing device performed by an operator of the photographing device in order to photograph the image with the photographing device;
outputting relevant information related to the photographing device based on the acquired operation information;
processing methods including;