JP2018109801A - Terminal device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a terminal device that can more accurately acquire a measurement result displayed on a display part of a health appliance.SOLUTION: A terminal device executes processing to recognize, from an image of a health appliance included in a picked-up image, character strings indicating measured values displayed on a display of the health appliance, and determines, for each of a plurality of recognized characters included in the recognized character strings that are a result of recognition of the displayed character strings indicating the measured values, whether or not the probability indicating the accuracy of recognition of the recognized characters is less than a first threshold. When the probability is less than the first threshold, the terminal device further executes processing to recognize, from each of N (N is an integer of 1 or more) picked-up images further created by imaging means, the displayed character strings indicating measured values. The terminal device determines, as measured values from the health appliance, the recognized character strings consisting of recognized characters that are specified on the basis of results of N+1 times of recognition of the displayed character strings indicating the measured values, and outputs a determined result.SELECTED DRAWING: Figure 11

Description

  The present disclosure relates to a terminal device, and more particularly, to a terminal device capable of capturing a measurement result of the health device by imaging the health device.

  Various measuring devices for measuring various biological information relating to the health condition of the human body such as blood pressure and pulse have been developed. The user's health condition is confirmed by taking it into a terminal device for collecting measurement data of such a measuring instrument.

  For example, Japanese Patent Laid-Open No. 2001-224557 (Patent Document 1) discloses a data input device. The data input device includes image acquisition means for obtaining an image of data displayed on the data display unit of the measuring instrument, number reading means for reading the numbers in the obtained image, and display means for displaying the read numbers. .

JP 2001-224557 A

  The display data analysis apparatus according to Patent Document 1 uses various markers attached to the measurement apparatus, the color of the display unit on which the measurement data is displayed, reflectance information, and the like in order to facilitate detection of measurement data from the captured image. I'm considering doing that. However, in Patent Document 1, since the accuracy of the actually detected measurement data is not studied at all, the display data analysis device displays measurement data different from the measurement data displayed on the display unit of the measurement device. There is a possibility of output.

  The present disclosure has been made in view of the above, and an object in one aspect is to provide a terminal device capable of more accurately acquiring a measurement result displayed on a display unit of a health device. .

  A terminal device according to an embodiment includes an imaging unit that images a health device as a subject, and a character that indicates a measurement value displayed on the display of the health device from an image of the health device included in a captured image generated by the imaging unit. Recognizing means for executing processing for recognizing a column, and each of a plurality of recognized characters included in a recognized character string that is a recognition result of the character string indicating the displayed measurement value represents the recognition accuracy of the recognized character Determining means for determining whether or not the accuracy is less than the first threshold value. When the accuracy of at least one recognized character among the plurality of recognized characters is less than the first threshold, the recognizing unit further includes each of N (N is an integer of 1 or more) captured images further generated by the imaging unit. Then, a process of recognizing the character string indicating the displayed measurement value is further executed. Based on the recognition result for N + 1 times of the character string indicating the displayed measurement value, the terminal device recognizes at least one recognized character obtained by recognizing the character for each character included in the character string. Of these, a recognition unit that identifies the recognition character with the highest number of times of recognition, and a determination unit that determines a recognition character string composed of the identified recognition characters as a measurement value of the health device, and an output that outputs a determination result of the determination unit And a control means.

  Preferably, when the accuracy for each of the plurality of recognized characters is equal to or greater than the first threshold, the determining unit determines a recognized character string including the plurality of recognized characters as a measurement value of the health device.

  Preferably, the output control means displays each of the plurality of recognized characters included in the recognized character string, which is a recognition result of the character string indicating the displayed measurement value, in a predetermined color according to the accuracy of the recognized character. On the display.

  Preferably, the output control means displays information indicating the accuracy of the recognized character for each of the plurality of recognized characters included in the recognized character string, which is a recognition result of the character string indicating the displayed measurement value. To display.

  Preferably, the determination means has a second accuracy smaller than the first threshold value for each of the plurality of recognized characters included in the recognized character string that is a recognition result of the character string indicating the displayed measurement value. It is further determined whether or not it is less than the threshold value. When the accuracy of at least one recognized character among the plurality of recognized characters is less than the second threshold, the terminal device determines, from the captured image generated by the imaging unit, each remaining recognized character other than at least one recognized character The image forming apparatus further includes a generating unit that generates a partial image obtained by cutting out a region including each character corresponding to, and a combining unit that combines a plurality of partial images generated by the generating unit. The recognition unit executes a process of recognizing a character string indicating the displayed measurement value from the image combined by the combining unit.

  Preferably, the output control means outputs abnormality information when the numerical value represented by the recognized character string determined as the measurement value of the health device is out of the predetermined range.

  According to the present disclosure, the measurement result displayed on the display unit of the health device can be acquired more accurately.

It is a figure which shows schematic structure of the system containing a terminal device and health equipment. It is a block diagram showing an example of the hardware constitutions of a terminal device. It is a figure which shows an example of the external appearance of the main-body part of a blood pressure meter. It is a figure for demonstrating the display method (the 1) of a guide. It is a figure for demonstrating the display method (the 2) of a guide. It is a figure for demonstrating the display system (the 3) of a guide. It is a figure which shows the example of a screen which displayed the recognition result in real time. It is a figure for demonstrating the synthetic | combination method of a some image. It is a figure for demonstrating the alerting | reporting function of an imaging direction. It is a figure for demonstrating the production | generation method of the captured image using a HDR function. It is a block diagram which shows the function structure of a terminal device. It is a flowchart which shows an example of the process sequence of a terminal device. It is a flowchart which shows the modification of the process sequence of a terminal device.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Overall configuration>
FIG. 1 is a diagram illustrating a schematic configuration of a system 1 including a terminal device and a health device.

  Referring to FIG. 1, system 1 includes a terminal device 10 that is a user terminal, blood pressure monitors 20 and 21 that are examples of health equipment for measuring the user's biological information, and a weight / body composition meter 23. . The health equipment is not limited to the sphygmomanometers 21 and 21 and the body weight / body composition meter 23, and may be a sleep meter, a pedometer, a thermometer or the like capable of displaying the measurement results.

  The sphygmomanometer 20 is a stationary upper arm sphygmomanometer in which a main body and a cuff (arm band) are separate bodies. The sphygmomanometer 21 is a wrist watch type sphygmomanometer in which a main body and a cuff are integrated. The sphygmomanometer 21 has a function of wearing a wrist for a long time like a wristwatch and measuring a pulsation every beat for 24 hours continuously, a function capable of being measured by simply wearing a wristwatch and pressing a measurement start button. . Thereby, the sphygmomanometer 21 can always measure the blood pressure of the user. The body weight / body composition meter 23 can measure body weight, body fat percentage, visceral fat level, subcutaneous fat percentage, and the like, and displays the measurement results on a display. In the following, for convenience of explanation, the blood pressure monitor 20 will be described as a representative example of “health equipment”.

  The terminal device 10 is, for example, a smartphone having a camera and a touch panel. The terminal device 10 has a function of detecting (recognizing) a measurement result displayed on the display of each health device from captured images generated by capturing each health device using a camera. In the following description, a smartphone that is a portable device will be described as a representative example of a “terminal device”. However, the terminal device may be another terminal device having an imaging function such as a foldable mobile phone, a tablet terminal device, and a PDA (Personal Data Assistance).

  Although details will be described later, the terminal device 10 images (photographs) a health device in a state where the measurement result is displayed using a camera. The terminal device 10 recognizes the measurement results from the generated captured images using various recognition methods, and outputs the recognized measurement results. For example, the terminal device 10 displays the recognized measurement value (maximum blood pressure value or the like) on the display of the own device.

<Hardware configuration>
(Terminal device)
FIG. 2 is a block diagram illustrating an example of a hardware configuration of the terminal device 10. Referring to FIG. 2, the terminal device 10 includes, as main components, a processor 152, a memory 154, an input device 156, a display 158, a wireless communication unit 160, a communication antenna 162, a camera 163, A memory interface (I / F) 164, a communication interface (I / F) 166, a speaker 168, a microphone 170, and a direction sensor 172 are included. Moreover, the terminal device 10 further includes a timer unit for measuring time.

  The processor 152 is typically an arithmetic processing unit such as a CPU (Central Processing Unit) or an MPU (Multi Processing Unit). The processor 152 functions as a control unit that controls the operation of each unit of the terminal device 10 by reading and executing the program stored in the memory 154. The processor 152 implements each process (step) of the terminal device 10 to be described later by executing the program.

  The memory 154 is realized by a RAM (Random Access Memory), a ROM (Read-Only Memory), a flash memory, or the like. The memory 154 stores a program executed by the processor 152, data used by the processor 152, and the like.

  The input device 156 receives an operation input to the terminal device 10. Typically, the input device 156 is realized by a touch panel. The touch panel is provided on a display 158 having a function as a display unit, and is, for example, a capacitance type. The touch panel detects a touch operation on the touch panel by an external object every predetermined time, and inputs touch coordinates to the processor 152. However, the input device 156 may include a button or the like.

  The wireless communication unit 160 connects to the mobile communication network via the communication antenna 162 and transmits and receives signals for wireless communication. Thereby, the terminal device 10 can communicate with other communication devices via a mobile communication network such as LTE (Long Term Evolution), for example.

  The camera 163 is realized by, for example, a charge coupled device (CCD) method, a complementary mental oxide semiconductor (CMOS) method, or other methods.

  The memory interface 164 reads data from the external storage medium 165. The processor 152 reads data stored in the storage medium 165 via the memory interface 164 and stores the data in the memory 154. The processor 152 reads data from the memory 154 and stores the data in the external storage medium 165 via the memory interface 164.

  The storage medium 165 is non-volatile such as a CD (Compact Disc), a DVD (Digital Versatile Disk), a BD (Blu-ray (registered trademark) Disc), a USB (Universal Serial Bus) memory, and an SD (Secure Digital) memory card. Includes media for storing programs.

  The communication interface (I / F) 166 is a communication interface for exchanging various data with other devices, and is realized by an adapter, a connector, or the like. In this embodiment, BLE (Bluetooth (registered trademark) low energy) is adopted as a communication method. However, the communication method may be a wireless communication method using a wireless LAN or the like, or a wired communication method using USB (Universal Serial Bus) or the like.

  The speaker 168 converts the sound signal given from the processor 152 into sound and outputs it to the outside of the terminal device 10. The microphone 170 receives an audio input to the terminal device 10 and gives an audio signal corresponding to the audio input to the processor 152.

  The direction sensor 172 detects the direction in which the terminal device 10 is facing. Specifically, the direction sensor 172 detects in which direction the display surface of the display 158 is directed. For example, the direction sensor 172 includes at least one of a three-axis acceleration sensor, a three-axis gyro sensor, and a three-axis geomagnetic sensor. Thereby, the terminal device 10 can acquire the posture information of the terminal device 10 when the user is holding the terminal device 10 by detecting the inclination of the own device with three axes using the direction sensor 172. it can. Note that the terminal device 10 may acquire azimuth information by detecting a geomagnetic vector with three axes using a three-axis geomagnetic sensor.

(Health equipment)
The health device only needs to provide information processing as described below as a whole, and a well-known hardware configuration can be adopted. For example, the health device includes a processor for executing various processes, a memory for storing programs and data, a display for displaying measurement results, and an input interface for receiving instructions from the user. In addition, a time measuring unit for measuring time and various hardware used for measuring biological information are included. For example, the sphygmomanometer 20 includes a cuff that can be wound around the user's upper arm (or wrist), a cuff pressure adjusting mechanism such as a piezoelectric pump and an exhaust valve, and a pressure for detecting the cuff pressure as the various hardware. A sensor and a temperature sensor for detecting the outside air temperature are included.

  FIG. 3 is a diagram illustrating an example of the appearance of the main body of the sphygmomanometer 20. Referring to FIG. 3, main body 22 of sphygmomanometer 20 includes a display 24, a forward / back button 26, a lamp 27, a user number switch 28, and a blood pressure measurement start / stop button 29. Note that brand information 41 (for example, brand “AAA”) and model information 43 (for example, model “XYZ-ABC”) of the sphygmomanometer 20 are provided on the outer surface of the main body 22. The user number switch 28 is used when storing measurement results for two persons separately. In the example of FIG. 3, the user with the number “1” is selected. The lamp 27 is a lamp indicating whether the cuff is wound properly, and is shown to be appropriate in the example of FIG.

  A measurement result is displayed on the display 24. Specifically, the measurement result includes a measurement value including a maximum blood pressure value (for example, 118 mmHg), a minimum blood pressure value (for example, 78 mmHg), and a pulse rate (for example, 70 nights / minute) displayed in the region 50, The measurement date and time (for example, 7:30 on July 1) displayed in 52, the user number (for example, “1”) displayed in the area 54, and the current temperature (for example, displayed in the area 56) 24 degrees) and various messages (for example, a message indicating the completion of measurement) displayed in the area 58. The measurement result may include information indicating the pulse wave state, information indicating the suitability of the cuff winding method, and the like.

<Recognition of measurement results>
The terminal device 10 according to the present embodiment has various functions used for recognizing measurement results displayed on a health device. Hereinafter, various functions will be described.

(Guide display)
A guide display method used for assisting the user's operation when the terminal device 10 recognizes the character string indicating the measurement value of the sphygmomanometer 20 will be described. The “character” in the present embodiment means a symbol that combines lines, dots, etc. to transmit and record a word (language), and not only alphanumeric characters, kanji, kana characters, etc., but also pictograms In addition, an arbitrary figure such as a stamp is attached with an identification code.

  FIG. 4 is a diagram for explaining a guide display method (part 1). When the terminal device 10 receives an instruction to shift to the character recognition mode from the user via the input device 156, the terminal device 10 shifts the operation mode of the terminal device 10 to the recognition mode and recognizes the character string included in the captured image. To start. The terminal device 10 automatically repeats generation of a captured image, display of the generated captured image, and character recognition processing at a predetermined period during the character recognition mode. Referring to FIG. 4, in the display method (part 1), a screen 500 including a frame 510 for guiding a range in which measured values (maximum blood pressure, minimum blood pressure, pulse) of blood pressure monitor 20 are accommodated during the character recognition mode. Is displayed on the display 158. Here, the screen 500 in FIG. 4 is a screen in which the captured image of the sphygmomanometer 20 and the frame 510 are superimposed.

  The frame 510 is displayed as a frame indicating the outer peripheral four corners of a recognition range (analysis range) in which a character string can be recognized. Therefore, the inside of the frame 510 is a recognition range. The terminal device 10 executes processing for recognizing a measurement result that is captured so as to be within the recognition range. Specifically, the terminal device 10 recognizes various character strings included in the measurement result.

  Here, the terminal device 10 stores screen information indicating what information is displayed in which area of the display 24 of the sphygmomanometer 20 in the memory 154. Therefore, for example, when the image is captured so that the display 24 fits within the frame 510 that is the recognition range, the terminal device 10 uses the screen information to determine what information each recognized character string is. Can be identified. For example, in the terminal device 10, a plurality of character strings “118”, “78”, and “70” recognized in the area 50 of the display 24 within the frame 510 are respectively a maximum blood pressure value, a minimum blood pressure value, and a pulse rate. Can be identified.

  In addition, the terminal device 10 receives an instruction from the user and specifies that the health device that is the subject is the blood pressure monitor 20. Alternatively, the terminal device 10 extracts feature information (shape, model, brand) and the like of the health device from the captured image, and compares the extracted feature information with the feature information of each health device stored in advance. Thus, it may be specified that the health device as the subject is the sphygmomanometer 20.

  According to the display method (part 1), since the user can grasp that the inside of the frame 510 is within the recognition range, the terminal device 10 can easily recognize the measurement value.

  FIG. 5 is a diagram for explaining a guide display method (part 2). Referring to FIG. 5, in the display method (part 2), a screen 530 including an object 520 representing the contour of the shape of sphygmomanometer 20 is displayed on display 158 during the character recognition mode. Note that the terminal device 10 may further display an object indicating a character (brand information 41 or the like) provided on the outer surface of the sphygmomanometer 20. In the example of FIG. 5, the captured image of the sphygmomanometer 20 is not shown for easy viewing of the object 520.

  The user captures an image so that the sphygmomanometer 20 overlaps the object 520 by moving the terminal device 10. When the sphygmomanometer 20 is imaged so as to overlap the object 520, the terminal device 10 recognizes various character strings included in the measurement result.

  According to the display method (part 2), the user can be guided to image the sphygmomanometer 20 at a position where the terminal device 10 can easily recognize the measurement result.

  FIG. 6 is a diagram for explaining a guide display method (part 3). Referring to FIG. 6, in the display method (part 3), a screen 550 including a rectangular translucent object 540 and translucent objects 542, 544, and 546 indicating each segment in 7 segments during the character recognition mode. Is displayed on the display 158.

  As shown in FIG. 3, the character string indicating the measurement value (maximum blood pressure value, minimum blood pressure value, pulse rate) of the sphygmomanometer 20 is composed of characters of a 7-segment display method. Therefore, as in the example of FIG. 6, a translucent object 542 indicating each segment for the systolic blood pressure value, a translucent object 544 indicating each segment for the systolic blood pressure value, and a translucent object 546 indicating each segment for the pulse rate, Is displayed. A semi-transparent object 540 indicates a recognition range and corresponds to an inner portion of the frame 510 in FIG. In the example of FIG. 6, the captured image of the sphygmomanometer 20 is not shown for easy viewing of each translucent object.

  By moving the terminal device 10, the user images each object 542, 544, 546 so that the measurement value displayed on the display 24 of the sphygmomanometer 20 overlaps. When each of the objects 542, 544, and 546 is imaged so that each measurement value overlaps, the terminal device 10 recognizes various character strings included in the measurement result.

  Note that the terminal device 10 can grasp in which area of the display 24 each measurement value is displayed by using the screen information of the sphygmomanometer 20. Therefore, the terminal device 10 includes the entire display 24 in which the recognition range indicated by the semi-transparent object 540 is included in the captured image when each semi-transparent object 542, 544, 546 is imaged so that each measurement value overlaps. As described above, the position, size, and the like of each translucent object can be set.

  According to the display method (No. 3), the user can be guided to image the sphygmomanometer 20 at a position where the terminal device 10 can easily recognize the measurement result. In the case of the 7-segment display method, since each segment is easy to read, there is an advantage that the user can easily adjust each measured value to each translucent object 542, 544, 546. However, when the character string indicating the measurement value of the sphygmomanometer 20 is displayed in a display method other than the seven-segment display method (for example, dot matrix display method), each of the translucent objects 542, 544, and 546 is also a dot. Displayed in a matrix display mode.

  The terminal device 10 may use a display method in which at least two of the above display methods (part 1) to (part 3) are combined.

(Use of recognition accuracy)
The terminal device 10 recognizes a character string indicating a measurement value (hereinafter, also referred to as “display measurement value”) displayed on the display 24 of the sphygmomanometer 20 from the captured image, and uses the recognized character string as the blood pressure. It is regarded as a total of 20 measured values. At this time, in order to increase the accuracy of the measured value displayed on the display 158 (that is, the degree of coincidence between the numerical value indicated by the displayed measured value and the numerical value indicated by the recognized character string) as much as possible, the terminal device 10 recognizes the recognized character. The accuracy indicating the recognition accuracy of (recognized character) is used.

  First, the terminal device 10 executes processing for recognizing a display measurement value from a captured image. Thereby, the terminal device 10 acquires the recognized character string recognized from the captured image and the accuracy of each recognized character included in the recognized character string. The accuracy may be calculated using a known technique. For example, the accuracy is calculated by comparing the feature amount of the character image with a dictionary pattern registered in advance in the memory 154.

  Next, for each of the plurality of recognized characters included in the recognized character string that is the recognition result of the character string indicating the display measurement value, the terminal device 10 has an accuracy of the recognized character that is less than a threshold Th1 (for example, 90%). It is determined whether or not. For example, as illustrated in FIG. 4, it is assumed that a character string “118” indicating a systolic blood pressure value is included in the captured image, and a recognized character string “118” is obtained by character recognition. In this case, the accuracies for the recognized characters “1”, “1”, and “8” are acquired as, for example, “85”, “90”, and “95”, respectively. The terminal device 10 determines whether or not each accuracy “85”, “90”, “95” is less than the threshold value Th1. Note that the terminal device 10 makes the same determination for a character string indicating a minimum blood pressure value and a character string indicating a pulse rate.

  When the terminal device 10 determines that the accuracy of all the recognized characters corresponding to the display measurement value is equal to or greater than the threshold Th1, the display device 158 uses the recognized character string including these recognized characters as the measured value of the sphygmomanometer 20. To display. For example, the accuracy “85”, “90”, and “95” for each of the plurality of recognition characters corresponding to the maximum blood pressure value are all equal to or higher than the threshold Th1, and thus the terminal device 10 uses the sphygmomanometer 20 to measure the maximum blood pressure. The hypertension value is determined to be “118” (deemed), and the determination result (for example, the maximum blood pressure value “118”) is displayed on the display 158.

  On the other hand, when the accuracy of at least one recognized character among the plurality of recognized characters is less than the threshold Th1, the terminal device 10 obtains a display measurement value from each of the captured images for a predetermined number of times (for example, twice). A process for recognizing the indicated character string is executed. For example, a character string “118” indicating a systolic blood pressure value is included in the captured image, and a recognition character string “110” is obtained by the recognition process, and each recognition character “1”, “1”, “0” Assume that the probabilities are “85”, “90”, and “60”, respectively. In this case, since the accuracy “60” for the recognized character “0” is less than the threshold Th1, the terminal device 10 executes a process of recognizing a character string indicating a display measurement value from each of the captured images for a predetermined number of times. To do.

  Next, the terminal device 10 is based on the recognition result of the character string indicating the display measurement value included in the first captured image and the recognition result of the character string included in each of the two additional captured images. Thus, the recognition character string to be adopted as the measurement value of the sphygmomanometer 20 is determined.

  For example, a plurality of characters in the character string included in the first captured image are recognized as “1”, “1”, and “0”, respectively, and the plurality of characters in the second captured image are respectively “1”, Assume that “1” and “8” are recognized, and the plurality of characters in the third captured image are recognized as “1”, “1”, and “8”, respectively. In this case, the recognized character obtained by recognizing the hundredth character is only “1”, and the number of times recognized as the recognized character “1” is three. Similarly, the recognized character obtained by recognizing the tens place character is only “1”, and the number of times recognized as the recognized character “1” is three. On the other hand, there are “0” and “8” as recognized characters obtained by recognizing the first digit, and the number of times recognized as the recognized character “0” is one. "Is recognized twice.

  Here, the terminal device 10 specifies that the recognized character with the highest number of recognition is “8” among the plurality of recognized characters “0” and “8”, which are the recognition results of the first place character. To do. For the hundredth character and the tenth character, the recognized character “1” is specified. The terminal device 10 determines the recognized character string “118” composed of the recognized characters “1”, “1”, and “8” thus identified as the maximum blood pressure value, and the determination result is displayed on the display 158. indicate.

  As described above, when the terminal device 10 determines that the accuracy of the recognized character is low, the terminal device 10 executes the recognition processing a predetermined number of times again, and determines each recognized character adopted by the majority vote as a measurement value of the sphygmomanometer 20. . Therefore, a recognized character string composed of recognized characters with low accuracy is not adopted as a measurement value of the sphygmomanometer 20. Therefore, the measurement value displayed on the display 24 of the sphygmomanometer 20 can be acquired more accurately.

(Real-time display of recognition results)
Here, it is assumed that the terminal device 10 erroneously recognizes a character string indicating a display measurement value included in the captured image. In this case, the terminal device 10 determines an erroneous recognition character string as a measurement value of the sphygmomanometer 20, and the determination result is displayed. The user compares the measured value displayed on the display 24 of the sphygmomanometer 20 with the measured value displayed on the display 158 of the terminal device 10 so that the terminal device 10 erroneously recognizes the measured value. To figure out. Therefore, the user needs to perform an operation for causing the terminal device 10 to recognize the measurement value displayed on the display 24 of the sphygmomanometer 20.

  In order to avoid the above work as much as possible, the function of displaying the recognition result of the display measurement value included in the captured image on the display 158 in real time will be described.

  FIG. 7 is a diagram showing an example of a screen displaying the recognition result in real time. Here, the terminal device 10 repeats generation of a captured image, display of the captured image, and recognition of a character string included in the captured image at a predetermined period during the character recognition mode. It is assumed that the display 24 of the sphygmomanometer 20 displays the maximum blood pressure value “133”, the minimum blood pressure value “106”, and the pulse rate “82”. A region 570 in FIG. 7 indicates a region where the screen brightness is high due to reflection of external light or the like.

  Referring to FIG. 7, terminal apparatus 10 recognizes character strings “133”, “106”, and “82” of the maximum blood pressure value, the minimum blood pressure value, and the pulse rate from the captured image. The terminal device 10 displays a captured image, a frame 510, and a screen 600 including objects 562, 564, and 566 that are recognition results. For ease of explanation, the screen 600 shows only a captured image of the display 24 portion of the sphygmomanometer 20. Further, a character string other than the measurement value displayed on the display 24 is also omitted.

  The object 562 indicates the recognition result of the character string indicating the systolic blood pressure value (that is, the recognition character string “133”), and the object 564 indicates the recognition result of the character string indicating the diastolic blood pressure value (that is, the recognition character string “ 106 "), and the object 566 indicates the recognition result of the character string indicating the pulse rate (that is, the recognized character string" 82 "). From this, the user can grasp that the measured value displayed on the display 24 of the sphygmomanometer 20 is correctly recognized by the terminal device 10. Thereafter, the user can determine the recognized character string as the measurement value of the sphygmomanometer 20, for example, by pressing a determination button or the like.

  Further, as illustrated in the example of FIG. 7, the terminal device 10 may display each of the plurality of recognized characters in a predetermined color according to the accuracy of the recognized character. Specifically, in the object 562 that is the recognition result of the character string indicating the systolic blood pressure value, the accuracy for the hundreds digit “1” is 70% or more and less than 90%. The accuracy for the digit “3” is 90% or more. Therefore, the terminal apparatus 10 displays the hundreds digit “1” of the object 562 and the tens and first digits “3” in different colors. Thereby, the user can perform operations such as changing the direction of the camera 163 of the terminal device 10 while confirming the recognized character string and the color of the recognized character string.

(Image composition)
Here, a function of combining a plurality of captured images will be described. FIG. 8 is a diagram for explaining a method of combining a plurality of images. Here, a method of synthesizing a captured image generated in a certain cycle p and a captured image generated in a cycle p + 1 next to the cycle p will be described. For ease of explanation, the captured images 722 and 726 indicate captured images of the display 24 portion of the sphygmomanometer 20. Further, a character string other than the measurement value displayed on the display 24 is also omitted.

  Referring to FIG. 8, in the captured image 722 generated in the period p, each region indicating the first place of the highest blood pressure value, the first place of the lowest blood pressure value, and the pulse rate is reflected by external light or the like. It overlaps with a region 752 where the screen brightness is high. Therefore, the accuracy for each recognized character, which is the recognition result of each character indicating the highest blood pressure value, the lowest blood pressure value, and the pulse rate, is low, for example, the threshold Th2 (for example, 40%) Less than.

  In this case, the terminal device 10 determines, from the captured image 722, each character corresponding to each remaining recognized character other than the recognized character having the accuracy less than the threshold Th2 (in this case, hundreds and tens of the maximum blood pressure value). A partial image 724 is generated by cutting out a region including characters and the tenth digit of the minimum blood pressure value. The terminal device 10 stores the partial image 724 in the memory 154.

  Next, in the captured image 726 generated at the period p + 1, the region indicating the first place of the maximum blood pressure value overlaps with the region 754 with high screen luminance. In this case, the terminal device 10 determines, from the captured image 726, each character corresponding to each remaining recognized character other than the recognized character having the accuracy less than the threshold Th2 (in this case, each character other than the hundreds character of the maximum blood pressure value). A partial image 728 is generated by cutting out a region including the character. The terminal device 10 stores the partial image 728 in the memory 154.

  Then, the terminal device 10 combines the partial image 724 and the partial image 728 stored in the memory 154 to generate a combined image 730. The terminal device 10 executes again the process of recognizing the character string indicating the display measurement value from the composite image. Since the composite image includes only the characters corresponding to the recognized characters having relatively high accuracy, the recognition accuracy of the character string indicating the display measurement value can be improved.

(Notification of imaging direction)
FIG. 9 is a diagram for explaining the imaging direction notification function. For ease of explanation, FIG. 9 shows only the captured image of the display 24 portion of the sphygmomanometer 20 among the captured images displayed on the screen of the display 158. Note that character strings other than the measured values displayed on the display 24 are also omitted.

  With reference to FIG. 9, the terminal device 10 compares the captured image 702 included in the screen 810 displayed on the display 158 with the reference image 700, and detects the inclination of the captured image 702 with respect to the reference image 700. For example, the inclination is detected by comparing the shape and size of the display 24 included in the reference image 700 and the captured image 702. In the example of FIG. 9, the depth direction, the short side direction, and the long side direction of the display 24 are defined as an X axis direction, a Y axis direction, and a Z axis direction, respectively.

  The terminal device 10 detects that the captured image 702 is tilted around the X axis with respect to the reference image 700. In order to reduce the inclination with respect to the reference image 700, the terminal device 10 displays an object 772 that prompts the terminal device 10 to rotate around the X axis on the screen 810. When the terminal device 10 is rotated, a screen 840 including the captured image 708 is obtained. Note that the terminal device 10 may use the three-axis posture information of the own device detected via the direction sensor 172 when calculating the rotation direction for reducing the inclination with respect to the reference image 700.

  Further, the terminal device 10 compares the captured image 704 included in the screen 820 with the reference image 700 and detects the inclination of the captured image 704 with respect to the reference image 700. Subsequently, the terminal device 10 detects that the captured image 704 is tilted around the Y axis with respect to the reference image 700. In order to reduce the inclination with respect to the reference image 700, the terminal device 10 displays an object 774 that prompts the terminal device 10 to rotate around the Y axis on the screen 820. When the terminal device 10 is rotated, a screen 840 including the captured image 708 is obtained.

  Further, the terminal device 10 compares the captured image 706 included in the screen 830 with the reference image 700, and detects the inclination of the captured image 706 with respect to the reference image 700. Subsequently, the terminal device 10 detects that the captured image 706 is tilted around the Z axis with respect to the reference image 700. In order to reduce the inclination with respect to the reference image 700, the terminal device 10 displays an object 776 that prompts the terminal device 10 to rotate around the Z axis on the screen 830. When the terminal device 10 is rotated, a screen 840 including the captured image 708 is obtained. Thereby, the recognition accuracy of the character string which shows a display measurement value can be improved.

(High dynamic range synthesis)
In the case of a captured image with a lot of whiteout or a captured image with a lot of blackout, there is a high possibility that only recognition with low accuracy can be performed from the captured image as a whole. Therefore, the terminal device 10 may generate an image using an HDR (high dynamic range imaging) function. In FIG. 10, only the captured image of the display 24 portion of the sphygmomanometer 20 is shown. Note that character strings other than the measured values displayed on the display 24 are also omitted.

  FIG. 10 is a diagram for explaining a method of generating a captured image using the HDR function. Referring to FIG. 10, when imaging the display 24 of the sphygmomanometer 20, the terminal device 10 adjusts the exposure amount to generate a captured image 742 with less whiteout and a captured image 744 with less blackout. To do. Then, the terminal device 10 generates a high dynamic range combined image 746 by combining the captured image 742 and the captured image 744.

<Others>
Terminal apparatus 10 according to the present embodiment further has the following functions.

(Specification of model etc.)
The terminal device 10 includes identification information (device ID) of the health device, characteristic information such as the shape of the housing of the health device, character information provided on the housing, the position of the display, the position of the button or switch, and the like. Are stored in the memory 154 in association with each other.

  The terminal device 10 compares the feature information extracted from the captured image of the health device with the feature information of the various health devices stored in the memory 154, so that the type and model of the health device included in the captured image It is possible to specify a brand or the like.

(Abnormal value judgment)
If each of the maximum blood pressure value, the minimum blood pressure value, and the pulse rate acquired by recognizing the character string of the display measurement value included in the captured image is an abnormal value, the terminal device 10 notifies the abnormality. Good. Specifically, when the acquired systolic blood pressure value is not included in the predetermined range Ra (for example, 40 to 260 mmHg), the terminal device 10 notifies that the systolic blood pressure value is abnormal. When the acquired minimum blood pressure value is not included in the predetermined range Rb (for example, 40 to 260 mmHg), the terminal device 10 notifies that the minimum blood pressure value is abnormal. When the acquired pulse rate is not included in the predetermined range Rc (for example, 40 to 180 beats / minute), the terminal device 10 notifies that the pulse rate is abnormal. In addition, when the minimum blood pressure value is higher than the maximum blood pressure value, the terminal device 10 notifies that these values are abnormal.

(Duplicate data elimination)
The terminal device 10 is configured not to repeatedly store the same measurement result in order to ensure the reliability of the measurement result and to effectively use the memory 154. Specifically, the terminal device 10 compares the acquired measurement result with the measurement result already stored in the memory 154 and determines whether or not they match. For example, the terminal device 10 compares the user number, the measurement date and time, and the measurement value included in each measurement result with the model specified by the image recognition. Then, the terminal device 10 does not store the acquired measurement result in the memory 154 when each piece of information of the acquired measurement result and each piece of information of the stored measurement result completely match. Thereby, the terminal device 10 can prevent the measurement results from being stored redundantly.

<Functional configuration>
FIG. 11 is a block diagram illustrating a functional configuration of the terminal device 10. Referring to FIG. 10, the terminal device 10 includes an imaging unit 202, an output control unit 204, a recognition unit 206, a determination unit 208, a determination unit 210, an abnormality determination unit 212, an image generation unit 214, A combining unit 216 and an inclination detecting unit 218 are included.

  The imaging unit 202 captures a subject at every predetermined period and generates a captured image. In the present embodiment, the imaging unit 202 generates a captured image including an image of the sphygmomanometer 20 as a subject. The imaging unit 202 is a function that is realized mainly by the cooperation of the processor 152 and the camera 163. The imaging unit 202 sequentially outputs the generated captured images to the output control unit 204, the recognition unit 206, and the image generation unit 214.

  The output control unit 204 causes the display 158 to display captured images generated by sequentially capturing images by the imaging unit 202. In one aspect, the output control unit 204 causes the display 158 to display a captured image generated by the imaging unit 202 and a guide for assisting the user's operation.

  The guide is displayed to assist the user in the imaging operation so that the recognition process is facilitated when the character string indicating the measurement result displayed on the display 24 of the sphygmomanometer 20 is recognized by the recognition unit 206. Is done. The guide includes an object (for example, a frame 510 and a translucent object 540) indicating the recognition range of the recognition unit 206. The guide further includes objects (for example, semi-transparent objects 542, 544, and 546) that indicate each of the seven segments that exist in the object that indicates the recognition range. Further, the guide may include an object representing the shape of the sphygmomanometer 20 (for example, the object 520).

  The recognition unit 206 recognizes a character string indicating the measurement result displayed on the display 24 of the sphygmomanometer 20 from the images of the sphygmomanometer 20 included in the captured image generated by the imaging unit 202. The measurement result includes, for example, a measurement value (maximum blood pressure value, minimum blood pressure value, pulse rate), measurement date and time, temperature, user number, various messages, information (symbol) indicating appropriateness of how to wind the cuff, and a pulse wave state. Contains information (symbols) to indicate. The recognizing unit 206 also recognizes a plurality of recognized characters included in a recognized character string that is a recognition result of a character string (for example, a character string indicating a systolic blood pressure value) indicating a measured value (displayed measured value) displayed on the display 24. For each of the above, an accuracy representing the recognition accuracy of the recognized character is calculated. The recognition unit 206 outputs a recognition result (recognized characters and accuracy) to the determination unit 208 and the determination unit 210.

  The determining unit 208 determines, for each of the plurality of recognized characters, whether or not the accuracy of the recognized character is less than a threshold Th1.

  When the determination unit 208 determines that the accuracy of at least one of the plurality of recognized characters is less than the threshold Th1, the recognition unit 206 further generates N (N is A process of recognizing a character string indicating a display measurement value from each of the captured images of an integer of 1 or more is further executed.

  The determination unit 210 determines a recognized character string that is a recognition result of the character string indicating the display measurement value as a measurement value of the sphygmomanometer 20 based on a predetermined condition.

  Specifically, when the determination unit 208 determines that the accuracy of at least one recognized character among the plurality of recognized characters is less than the threshold Th1, the determination unit 210 sets a character string ( For example, based on N + 1 recognition results (that is, N + 1 recognized character strings) of the character string “118” representing the systolic blood pressure value, each character (characters “1” and “1”) included in the character string. , “8”), among at least one recognized character obtained as a recognition result of the character (for example, two recognized characters “8” and “0” obtained as a recognition result of the character “8”), Identify the recognized character with the highest number of recognitions.

  The determination unit 210 determines the recognized character (for example, the recognized character “1” specified for the hundreds character “1”, the recognized character “1” specified for the tens character “1”). , A recognized character string (for example, recognized character string “118”) specified for the first character “8”) is determined as a measurement value of the sphygmomanometer 20.

  On the other hand, when the determination unit 208 determines that the accuracy for each of the plurality of recognition characters is equal to or greater than the threshold Th1, the determination unit 210 converts the recognition character string including the plurality of recognition characters into the sphygmomanometer 20. Determined as measured value. In other words, when the accuracy for each recognized character included in the recognized character string first recognized by the recognizing unit 206 is greater than or equal to the threshold Th1, the determining unit 210 determines the first recognized character string as It is determined as a measurement value of the sphygmomanometer 20.

  The determination unit 210 stores the determination result in the memory 154 and outputs it to the output control unit 204 and the abnormality determination unit 212. The output control unit 204 outputs the determination result of the determination unit 210. For example, the output control unit 204 causes the display 158 to display the determination result. Note that the output control unit 204 may output the determination result to an external device via the communication interface 166 (or the wireless communication unit 160), or output the determination result via the speaker 168 by voice. Also good.

  Abnormality determination unit 212 determines the presence or absence of abnormality based on the numerical value represented by the recognized character string determined by determination unit 210 as the measurement value of sphygmomanometer 20. For example, the abnormality determination unit 212 determines that there is no abnormality when the numerical value is within a predetermined range, and determines that there is an abnormality when the numerical value is outside the predetermined range. The abnormality determination unit 212 outputs the determination result to the output control unit 204. The output control unit 204 outputs abnormality information when the abnormality determination unit 212 determines that there is an abnormality. For example, the output control unit 204 causes the display 158 to display an error notification indicating that the numerical value is outside a predetermined range.

  Here, in another aspect, for each of a plurality of recognized characters included in a recognized character string that is a recognition result of the character string indicating the display measurement value, the determination unit 208 has a threshold Th2 (< You may further judge whether it is less than threshold value Th1). The determination unit 208 outputs the determination result to the image generation unit 214 or the inclination detection unit 218.

  The determination unit 208 has an accuracy of at least one recognized character (for example, recognized character “0”) out of a plurality of recognized characters (for example, recognized characters “1”, “2”, “0”) less than the threshold Th2. When the image generation unit 214 determines that the remaining recognition characters other than the at least one recognition character from the captured image generated by the imaging unit 202 (for example, the recognition characters “1” and “2”). A partial image (for example, partial images 724 and 728) obtained by cutting out an area including each character (for example, the characters “1” and “2”) corresponding to is generated. The image generation unit 214 outputs the partial image to the synthesis unit 216.

  The synthesizing unit 216 synthesizes a plurality of partial images generated by the image generating unit 214. The composition unit 216 outputs the composite image to the recognition unit 206. The recognition unit 206 executes processing for recognizing a character string indicating a display measurement value from the composite image (for example, the composite image 730) combined by the combining unit 216.

  In still another aspect, when the determination unit 208 determines that the accuracy of at least one recognition character of the plurality of recognition characters is less than the threshold Th2, the inclination detection unit 218 captures the reference image. The inclination of the captured image generated by the unit 202 is detected. The output control unit 204 causes the display 158 to display a rotation direction for reducing the inclination of the captured image with respect to the reference image.

  The output control unit 204 displays each of the plurality of recognized characters included in the recognized character string, which is the recognition result of the character string indicating the display measurement value, on the display 158 with a predetermined color according to the accuracy of the recognized character. It may be displayed. In addition, the output control unit 204 may display information indicating the accuracy of the recognized character on the display 158. For example, information indicating the accuracy is displayed at a predetermined position in the frame 510. Further, the output control unit 204 may display information indicating the accuracy of the recognized character in the vicinity of the recognized character.

<Processing procedure>
FIG. 12 is a flowchart illustrating an example of a processing procedure of the terminal device 10.

  Referring to FIG. 12, processor 152 of terminal device 10 generates a captured image of sphygmomanometer 20 captured using camera 163, and displays the captured image on display 158 (step S10). The processor 152 displays a guide for assisting the user's operation on the display 158 (step S12). The processor 152 recognizes a character string indicating the measurement value (display measurement value) displayed on the display 24 of the sphygmomanometer 20 from the captured image (step S14).

  In the processor 152, the accuracy of each of the plurality of recognized characters included in the recognized character string that is the recognition result of the character string indicating the display measurement value is greater than or equal to the threshold Th1 (that is, the accuracy of each recognized character is greater than or equal to the threshold Th1). Whether or not (step S16). When the accuracy for each of the plurality of recognized characters is equal to or greater than the threshold Th1 (YES in step S16), processor 152 determines a recognized character string formed of the plurality of recognized characters as a measurement value of sphygmomanometer 20. (Step S24), the determination result (the numerical value indicated by the recognized character string) is displayed on the display 158 (Step S26), and the process is terminated.

  When the accuracy of at least one of the plurality of recognized characters is less than the threshold Th1 (NO in step S16), the processor 152 determines whether the captured image is different from the captured image that is the target of the recognition process. The character string recognition process indicating the display measurement value is re-executed (step S18). The processor 152 determines whether or not the character string recognition process indicating the display measurement value has been performed a predetermined number of times (step S20).

  If the recognition process has not been executed a predetermined number of times (NO in step S20), processor 152 repeats the process of step S18. When the recognition process is executed a predetermined number of times (YES in step S20), the processor 152 converts the recognition character string composed of each recognition character specified based on the recognition result for the predetermined number of times to the sphygmomanometer 20 Is determined as a measured value (step S22). Specifically, for each character included in the character string indicating the display measurement value, the processor 152 selects the recognized character having the highest number of recognitions among at least one recognized character obtained by recognizing the character. Identify. Then, the processor 152 displays the determination result on the display 158 (step S26), and ends the process.

(Modification)
FIG. 13 is a flowchart illustrating a modification of the processing procedure of the terminal device 10.

  Referring to FIG. 13, the processes in steps S50 to S56 are the same as the processes in steps S10 to S16 in FIG. 12, respectively, and detailed description thereof will not be repeated. When the accuracy of each of the plurality of recognized characters included in the recognized character string that is the recognition result of the character string indicating the display measurement value is less than a threshold Th1 (for example, 90%) (NO in step S56), step S66, The process of S68 is executed. Since the processes in steps S66 and S68 are the same as the processes in steps S24 and S26 in FIG. 12, respectively, detailed description thereof will not be repeated.

  If the accuracy of at least one of the plurality of recognized characters is less than the threshold Th1 (YES in step S56), the processor 152 determines that the accuracy of each of the plurality of recognized characters is a threshold Th2 (for example, 40 %) Or more is determined (step S58). When the accuracy for each of the plurality of recognized characters is equal to or greater than threshold value Th2 (YES in step S58), the processes of steps S60 to S64 and S68 are executed. Since the processes of steps S60 to S64 and S68 are the same as the processes of steps S18 to S22 and S26 in FIG. 12, detailed description thereof will not be repeated.

  When the accuracy of at least one of the plurality of recognized characters is less than the threshold Th2 (NO in step S58), the processor 152 determines each remaining character other than at least one recognized character from the generated captured image. A partial image is generated by cutting out a region including each character corresponding to the recognized character (that is, a recognized character having a relatively high accuracy equal to or higher than the threshold Th2) (step S70). The processor 152 stores the generated partial image in the memory 154.

  The processor 152 determines whether or not a predetermined number of partial images are stored in the memory 154 (step S72). If the predetermined number of partial images is not stored (NO in step S72), processor 152 repeats the processing from step S50. If a predetermined number of partial images are stored (YES in step S72), processor 152 combines the predetermined number of partial images (step S74). The processor 152 executes processing for recognizing a character string indicating a display measurement value from the composite image (step S76).

  The processor 152 determines whether or not the accuracy of each of the plurality of recognized characters included in the recognized character string recognized from the composite image is greater than or equal to the threshold Th1 (step S78). When the accuracy for each of the plurality of recognized characters is equal to or greater than threshold value Th1 (YES in step S78), processor 152 executes the process of step S66. When the accuracy of at least one recognized character among the plurality of recognized characters is less than the threshold Th1 (NO in step S78), the processor 152 notifies an error (step S80) and ends the process.

<Advantages>
According to the present embodiment, when the terminal device 10 determines that the accuracy of the recognized character is low, the terminal device 10 executes the recognition process a predetermined number of times again, and recognizes a recognized character string composed of each recognized character adopted by majority vote. Is determined as a measured value, and the determined recognized character string is displayed on the display 158. Therefore, the recognized character with low accuracy is not displayed as it is on the display 158, and the accuracy of the measurement value displayed on the display 158 can be improved.

  Further, according to the present embodiment, when recognizing a character string indicating a display measurement value, a guide for assisting the user's operation is displayed. Moreover, since the object which shows a frame, the recognition range, each segment of a 7 segment display system, etc. is displayed as a guide, a user can be induced to image the sphygmomanometer 20 at an optimal position.

<Other embodiments>
(1) In embodiment mentioned above, the terminal device 10 may evaluate the reliability of a measured value using the various information contained in a measurement result. For example, the processor 152 (evaluation unit) of the terminal device 10 calculates a period from the measurement date to the imaging date and determines whether the calculated period is within a predetermined period. When the calculated period is not within the predetermined period, the processor 152 evaluates that the reliability of the measurement result is low. In this case, the processor 152 may output an error notification indicating that the calculated period is not within the predetermined period.

  (2) In the above-described embodiment, the configuration for notifying the user of an appropriate imaging direction based on the inclination of the captured image 702 with respect to the reference image 700 in FIG. 9 has been described, but the configuration is not limited thereto. Specifically, the following steps A to F may be executed.

  The terminal device 10 detects the inclination of the captured image 702 with respect to the reference image 700 (step A). Based on the inclination, the terminal device 10 corrects the captured image 702 so as to reduce the inclination with respect to the reference image 700 (step B). The terminal device 10 executes processing for recognizing a character string indicating a display measurement value from the corrected captured image 702 (step C). The terminal device 10 determines whether or not the accuracy for all recognized characters corresponding to the display measurement value is equal to or greater than a predetermined threshold (for example, threshold Th1) (step D).

  When the accuracy of all recognized characters corresponding to the display measurement value is determined as a predetermined threshold, the terminal device 10 displays a recognition character string formed of these recognition characters on the display 158 as a measurement value of the sphygmomanometer 20 ( Step E).

  On the other hand, when the accuracy of at least one of the plurality of recognized characters is less than the predetermined threshold, the terminal device 10 rotates the terminal device 10 in an appropriate direction in order to reduce the inclination with respect to the reference image 700. An object (for example, objects 772, 774, 776) that prompts the user to perform the operation is displayed on the display 158 (step F), and the processing from step A is repeated again.

  (3) In the above-described embodiment, it is possible to provide a program for causing a computer to function and executing control as described in the above flowchart. Such a program is recorded on a non-temporary computer-readable recording medium such as a flexible disk attached to the computer, a CD (Compact Disk Read Only Memory), a secondary storage device, a main storage device, and a memory card. It can also be provided as a program product. Alternatively, the program can be provided by being recorded on a recording medium such as a hard disk built in the computer. A program can also be provided by downloading via a network.

  The program may be a program module that is provided as a part of an operating system (OS) of a computer and that calls necessary modules in a predetermined arrangement at a predetermined timing to execute processing. In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module can also be included in the program according to the present embodiment.

  Further, the program according to the present embodiment may be provided by being incorporated in a part of another program. Even in this case, the program itself does not include the module included in the other program, and the process is executed in cooperation with the other program. A program incorporated in such another program can also be included in the program according to the present embodiment.

  (4) The configuration exemplified as the above-described embodiment is an example of the configuration of the present invention, and can be combined with another known technique, and a part thereof does not depart from the gist of the present invention. It is also possible to change and configure such as omitting. In the above-described embodiment, the processing and configuration described in the other embodiments may be adopted as appropriate.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  1 system, 10 terminal device, 20, 21 sphygmomanometer, 22 body part, 23 body composition meter, 24,158 display, 26 forward / back button, 27 lamp, 28 user number switch, 29 start / stop button, 41 brand information , 43 Model information, 152 processor, 154 memory, 156 input device, 160 wireless communication unit, 162 communication antenna, 163 camera, 164 memory interface, 165 storage medium, 166 communication interface, 168 speaker, 170 microphone, 172 direction sensor, 202 Image capturing unit, 204 output control unit, 206 recognition unit, 208 determination unit, 210 determination unit, 212 abnormality determination unit, 214 image generation unit, 216 synthesis unit, 218 inclination detection unit.

Claims (6)

A terminal device,
Imaging means for imaging a health device as a subject;
Recognition means for executing processing for recognizing a character string indicating a measurement value displayed on the display of the health device from the image of the health device included in the captured image generated by the imaging means;
Whether or not each of the plurality of recognized characters included in the recognized character string that is the recognition result of the character string indicating the displayed measurement value is less than a first threshold value representing the accuracy of recognition of the recognized character. And a judging means for judging
When the accuracy of at least one recognition character of the plurality of recognition characters is less than the first threshold, the recognition unit further generates N pieces (N is an integer of 1 or more) generated by the imaging unit. A process of recognizing a character string indicating the displayed measurement value from each of the captured images of
Based on the recognition result for N + 1 times of the character string indicating the displayed measurement value, among the at least one recognized character obtained by recognizing the character for each character included in the character string, Determining means for identifying a recognized character having a large number of recognition times, and determining a recognized character string composed of each identified recognized character as a measurement value of the health device;
A terminal device further comprising output control means for outputting a determination result of the determination means.   When the accuracy for each of the plurality of recognition characters is equal to or greater than the first threshold, the determination unit determines a recognition character string composed of the plurality of recognition characters as a measurement value of the health device. The terminal device according to claim 1.   The output control means displays each of a plurality of recognized characters included in a recognized character string that is a recognition result of the character string indicating the displayed measurement value in a predetermined color according to the accuracy of the recognized character. The terminal device according to claim 1, wherein the terminal device is displayed on a display of the device.   The output control means, for each of a plurality of recognized characters included in a recognized character string that is a recognition result of the character string indicating the displayed measurement value, displays information indicating the accuracy of the recognized character of the terminal device. The terminal device according to claim 1, wherein the terminal device is displayed on a display. The determination unit is configured such that, for each of a plurality of recognized characters included in a recognized character string that is a recognition result of the character string indicating the displayed measurement value, the accuracy of the recognized character is smaller than the first threshold value. Further determine whether it is less than two thresholds,
When the accuracy of at least one recognized character of the plurality of recognized characters is less than the second threshold value, each recognition of remaining other than the at least one recognized character from the captured image generated by the imaging unit Generating means for generating a partial image obtained by cutting out an area including each character corresponding to the character;
A synthesis unit that synthesizes a plurality of partial images generated by the generation unit;
The terminal device according to claim 1, wherein the recognition unit executes a process of recognizing a character string indicating the displayed measurement value from the image combined by the combining unit.   The terminal according to any one of claims 1 to 5, wherein the output control means outputs abnormality information when a numerical value represented by a recognized character string determined as a measurement value of the health device is outside a predetermined range. apparatus.