JP2014053693A - Information service system, electronic device, control method, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately read a measurement value displayed on a display section from an image in which the display section of a health measuring instrument is photographed.SOLUTION: An electronic device (a mobile phone 100, for example) has: a camera for capturing image data to be photographed which includes a display section for displaying a measurement value measured by a health measuring instrument which is provided in the health measuring instrument; and a controller for specifying an area in which an image of the display section which is included in the image data is projected on the basis of luminance distribution of the image data which is captured by the camera, performing photometric processing using the specified area as a target, and adjusting the exposure amount when an image to be photographed is acquired on the basis of the result of the photometric processing.

Description

  The present application relates to an information providing system, an electronic device, a control method, and a control program.

  For health management, various measuring instruments such as a weight scale that measures body weight, a sphygmomanometer that measures blood pressure, and a pedometer that measures the number of steps are used. Some of such measuring instruments have a communication function for transmitting measured values to an information processing apparatus (see, for example, Patent Document 1 and Patent Document 2). The value transmitted to the information processing apparatus is used for various types of analysis.

JP 2008-033834 A JP 2005-319216 A

  If each measuring instrument is provided with a communication function, the configuration becomes complicated, resulting in an increase in cost and difficulty in miniaturization. From the above, there is a need for an information providing system, an electronic device, a control method, and a control program that make it possible to use the values measured by the measuring device in other devices without complicating the configuration of the measuring device. .

  The information provision system which concerns on one aspect is an information provision system which has an electronic device and an information provision apparatus, Comprising: The said electronic device is provided in the said health measuring device which displays the measured value measured by a health measuring device. A camera that captures image data to be photographed including a display unit, a region where an image of the display unit included in the image data captured by the camera is displayed, and photometry for the specified region A controller that adjusts an exposure amount when acquiring the image to be imaged based on a result of the photometry process, and transmits information related to the measurement value included in the image to the information providing apparatus Communication unit.

  An electronic apparatus according to one aspect includes a camera that captures image data of a photographing target including a display unit provided in the health measuring instrument that displays a measurement value measured by the health measuring instrument, and is captured by the camera. An area in which the image of the display unit included in the image data is displayed is specified, a photometric process is performed on the specified area, and an image of the imaging target is acquired based on a result of the photometric process And a controller for adjusting the exposure amount at the time.

  A control method according to one aspect is a control method for controlling an electronic device, and image data of an imaging target including a display unit provided in the health measuring device for displaying a measurement value measured by the health measuring device. A capturing step, specifying an area in which the image of the display unit included in the image data is projected, performing a photometric process on the specified area, and based on a result of the photometric process, Adjusting an exposure amount when acquiring an image.

  The control program according to one aspect includes a step of capturing image data of an imaging target including a display unit provided in the health measuring device for displaying a measurement value measured by the health measuring device on an electronic device, and the image An area when the image of the display unit included in the data is projected, the exposure when the photometric process is performed on the identified area and the image to be captured is acquired based on the result of the photometric process Adjusting the amount.

FIG. 1 is a diagram illustrating a configuration of an information providing system according to an embodiment. FIG. 2 is a diagram showing an outline of the operation of the information providing system. FIG. 3 is a block diagram of a mobile phone. FIG. 4 is a diagram illustrating an example of a body composition meter included in an image. FIG. 5 is a diagram illustrating an example of an activity meter included in an image. FIG. 6 is a diagram illustrating an example of a sphygmomanometer included in an image. FIG. 7 is a block diagram of the information providing apparatus. FIG. 8 is a diagram illustrating an example of measurement value information. FIG. 9 is a flowchart illustrating a procedure of processing for reading and transmitting a measurement value. FIG. 10 is a diagram illustrating an example of screen transition. FIG. 11 is a diagram illustrating a modified example of the operation of the information providing system. FIG. 12 is a diagram illustrating another modified example of the operation of the information providing system. FIG. 13 is a diagram illustrating another modification of the operation of the information providing system. FIG. 14 is a diagram illustrating another configuration of the information providing system. FIG. 15 is a diagram illustrating an example of a state where image data is supplemented by a camera. FIG. 16 is a diagram showing an example of the luminance distribution of the image data shown in FIG. FIG. 17 is a diagram illustrating an example of the position in the image data of the pixel having the peak luminance value in the luminance distribution illustrated in FIG. 16. FIG. 18 is a diagram illustrating an example of a region cut out as a candidate for a region where an image of the display unit of the body composition meter is displayed. FIG. 19 shows an example of a processing procedure for specifying a region in which an image of the display unit of the health measuring device is displayed from image data captured by the mobile phone with a camera and performing photometric processing for the specified region. FIG. FIG. 20 is a diagram illustrating an example of a pattern design for specifying a region in image data in which an image of the display unit of the body composition meter is displayed.

  Embodiments disclosed in the present application will be described in detail with reference to the drawings. In the following embodiment, a description will be given using a mobile phone as an example of an electronic device.

(Embodiment 1)
In the first embodiment, an information providing system including an information providing apparatus that provides a service related to health management and a mobile phone of a user who uses the service related to health management will be described. The information providing apparatus manages measurement values measured by various measuring instruments such as a weight scale, a blood pressure monitor, and a pedometer for each user, analyzes the measurement values, and provides the analysis results to the users. The mobile phone reads the measurement value of the measuring instrument and transmits it to the information providing apparatus, and receives the analysis result from the information providing apparatus.

  The configuration of the information providing system according to the first embodiment will be described with reference to FIG. As shown in FIG. 1, the information providing system includes a mobile phone 100, a base station 200, information providing devices 300a and 300b, and a voice recognition device 400. In the following description, the information providing apparatus 300a and 300b may be collectively referred to as the information providing apparatus 300 without specifying which one.

  The mobile phone 100 has a function of reading the measurement value measured by the health measuring instrument and transmitting it to the information providing apparatus 300. The health measuring instrument is a measuring instrument that measures numerical values related to health. The health meter is, for example, a weight scale that measures body weight, a body composition meter that measures body fat percentage in addition to body weight, a thermometer that measures body temperature, a sphygmomanometer that measures blood pressure, a pulsometer that measures pulse, a step count A pedometer that measures the amount of activity and an activity meter that measures the amount of activity due to exercise including walking.

  The cellular phone 100 reads a measurement value displayed on a display unit included in the health measuring device by photographing the health measuring device with a camera. By photographing the measurement values displayed on the display unit with the camera in this way, the mobile phone 100 can read the measurement values measured by the health measuring device even if the health measuring device does not have a communication function. The read measurement value is transmitted to the information providing apparatus 300 by the communication function of the mobile phone 100. Details of reading the measurement values by the mobile phone 100 will be described later.

  Base station 200, information providing apparatus 300, and speech recognition apparatus 400 are communicably connected via network 500.

  Base station 200 establishes a wireless communication line with mobile phone 100 located within the communication area, and enables mobile phone 100 to communicate with other devices through this wireless communication line. Hereinafter, in order to simplify the description, when the communication with the mobile phone 100 is described, the description regarding the base station 200 may be omitted.

  The information providing apparatus 300 stores the measurement value transmitted from the mobile phone 100 in association with the ID (identification number) and date / time (time stamp) of the user of the mobile phone 100. The information providing apparatus 300 can store the measurement values of a plurality of types of health measuring devices in association with the identification numbers of the respective users. Then, the information providing apparatus 300 performs analysis processing based on the measurement values accumulated in this way, and transmits information such as health advice to the mobile phone 100. There are a plurality of information providing apparatuses 300, each of which is operated by a different operator.

  The speech recognition apparatus 400 analyzes the content of speech information transmitted from another device by speech recognition processing, and transmits the analyzed content to the transmission source device. For example, when the mobile phone 100 transmits voice information in which the user's voice is recorded to the voice recognition device 400, the voice recognition device 400 analyzes the content of the voice information by voice recognition processing and carries the analyzed content as text data. Send to phone 100. With such a mechanism, voice operation of the mobile phone 100 is realized.

  The configuration of the information providing system is not limited to the example shown in FIG. For example, the number of various devices included in the information providing system may be larger or smaller than the number shown in FIG. The information providing system may not include the voice recognition device 400. The information providing apparatus 300 may be a home personal computer, and the base station 200 may be a home wireless LAN router.

  An outline of the operation of the information providing system will be described with reference to FIG. Prior to the operation shown in FIG. 2, it is assumed that the health measuring device performs measurement and the measured value is displayed on the display unit of the health measuring device. In this state, the mobile phone 100 takes a picture of the health measuring device using the camera (step S11). Then, the mobile phone 100 determines the position and type of the measurement value included in the photographed image (Step S12). Shooting in the present application is not limited to acquiring an image when the shutter is released (when the release button is pressed), and is continuously displayed while displaying the image on the display unit as in a so-called live view. Including acquisition.

  After determining the position and type of the measurement value in this way, the mobile phone 100 reads the measurement value from the captured image (step S13). Reading of the measurement value included in the image can be realized by using a known character recognition technique. The mobile phone 100 transmits the read measurement value together with the additional information to the information providing apparatus 300 (step S14). The additional information is information necessary for the information providing apparatus 300 to use the measurement value for analysis. The additional information includes, for example, a value indicating the type of measurement value, a value indicating the unit of measurement value, and the like.

  When a plurality of measurement values are read from the captured image, the mobile phone 100 may transmit the measurement values separately or collectively. The mobile phone 100 may store measurement values acquired from a plurality of types of health measuring devices, and collectively transmit them to the information providing apparatus 300 at some timing. In this case, the mobile phone 100 stores the date and time when the measurement value is acquired in association with each measurement value, and includes the stored date and time in the additional information at the time of transmission.

  When there are a plurality of information providing apparatuses 300, the mobile phone 100 may transmit measurement values and additional information to a predetermined information providing apparatus 300, or the manufacturer, type, etc. acquired from the captured image Depending on the information, the destination information providing apparatus 300 may be determined.

  The information providing apparatus 300 stores the measurement value transmitted from the mobile phone 100 in association with the ID and time stamp of the user of the mobile phone 100 (step S15). The user ID may be acquired from the additional information, or may be acquired based on the value of the header of the packet that transmits the measurement value from the mobile phone 100 to the information providing apparatus 300. The time stamp may be acquired from the additional information, or may be acquired based on the value of the time measuring unit of the information providing apparatus 300 when the measured value is received. The time stamp indicates the date / time when the measurement value was read or the date / time when the measurement value was transmitted when acquired from the additional information. When the time stamp is acquired based on the value of the time measuring unit of the information providing apparatus 300, the measurement value is received. Indicates the date and time when

  Thereafter, the information providing apparatus 300 executes an analysis process based on the stored measurement value (step S16). The analysis process may be executed every time a measurement value is received, or may be executed every time a specific type of measurement value (for example, weight) is received. Alternatively, the analysis process may be executed at a predetermined time.

  In the analysis process, health analysis is performed based on changes in a plurality of measurement values over time regarding the user of the mobile phone 100. Since the information providing apparatus 300 stores measurement values related to a plurality of users, the information providing apparatus 300 may use information obtained by statistically processing the measurement values related to a plurality of users in the analysis process.

  The information providing apparatus 300 transmits the analysis result to the mobile phone 100 (step S17). The mobile phone 100 displays the transmitted analysis result (step S18). The analysis result may be transmitted to the mobile phone 100 as an e-mail, or may be transmitted to the mobile phone 100 in another predetermined format. The measurement value and the additional information may be transmitted as an HTTP (HyperText Transfer Protocol) request, and the analysis result may be returned in an HTML (HyperText Markup Language) format.

  Thus, by accumulating and analyzing the measurement values read from the health measuring device, it is possible to provide the user with information that is difficult to notice from the individual measurement values.

  The configuration of the mobile phone 100 will be described with reference to FIG. As shown in FIG. 3, the mobile phone 100 includes a touch screen display 2, a button 3, an illuminance sensor 4, a proximity sensor 5, a communication unit 6, a receiver 7, a microphone 8, a storage 9, and a controller. 10, a speaker 11, a camera 12, a camera 13, a connector 14, an acceleration sensor 15, an orientation sensor 16, a gyroscope 17, and a GPS (Global Positioning System) receiver 18.

  The touch screen display 2 includes a display 2A and a touch screen 2B. The display 2A includes a liquid crystal display (LCD: Liquid Crystal Display), an organic EL display (OELD: Organic Electro-Luminescence Display), or an inorganic EL display (IELD: Inorganic Electro-Luminescence Display). The display 2A displays characters, images, symbols, graphics, and the like.

  The touch screen 2B detects contact of a finger, a pen, a stylus pen, or the like with respect to the touch screen 2B. The touch screen 2B can detect a position where a plurality of fingers, a pen, a stylus pen, or the like is in contact with the touch screen 2B. In the following description, a finger, pen, stylus pen or the like that comes into contact with the touch screen 2B may be referred to as a “contact object (contact object)”.

  The detection method of the touch screen 2B may be any method such as a capacitance method, a resistive film method, a surface acoustic wave method (or an ultrasonic method), an infrared method, an electromagnetic induction method, and a load detection method. In the following description, for the sake of simplicity, it is assumed that the user uses his / her finger to touch the touch screen 2 </ b> B in order to operate the mobile phone 100.

  The mobile phone 100 is based on at least one of the contact detected by the touch screen 2B, the position at which the contact is detected, the change in the position at which the contact is detected, the interval at which the contact is detected, and the number of times the contact is detected. To determine the type of gesture. The gesture is an operation performed on the touch screen 2B. Gestures identified by the mobile phone 100 include, but are not limited to, touch, long touch, release, swipe, tap, double tap, long tap, drag, flick, pinch in, and pinch out, for example.

  The button 3 is operated by the user. The controller 10 detects an operation on the button 3 by cooperating with the button 3. The operation on the button 3 includes, but is not limited to, click, double click, triple click, push, and multi-push.

  The illuminance sensor 4 detects the illuminance of ambient light of the mobile phone 100. Illuminance indicates light intensity, brightness, or luminance. The illuminance sensor 4 is used for adjusting the luminance of the display 2A, for example. The proximity sensor 5 detects the presence of a nearby object without contact. The proximity sensor 5 detects the presence of an object based on a change in a magnetic field or a change in a feedback time of an ultrasonic reflected wave. The proximity sensor 5 detects that the touch screen display 2 is brought close to the face, for example. The illuminance sensor 4 and the proximity sensor 5 may be configured as one sensor. The illuminance sensor 4 may be used as a proximity sensor.

  The communication unit 6 communicates wirelessly. The communication method supported by the communication unit 6 is a wireless communication standard. Examples of wireless communication standards include cellular phone communication standards such as 2G, 3G, and 4G. As a cellular phone communication standard, for example, LTE (Long Term Evolution), W-CDMA (Wideband Code Multiple Access), CDMA2000, PDC (Personal Digital Cellular), GSM (registered trademark) (Globular Co., Ltd.) And PHS (Personal Handy-phone System). As wireless communication standards, for example, there are WiMAX (Worldwide Interoperability for Microwave Access), IEEE802.11, Bluetooth (registered trademark), IrDA (Infrared Data Association), NFC (NearCo), etc. The communication unit 6 may support one or more of the communication standards described above.

  The receiver 7 and the speaker 11 are sound output units. The receiver 7 and the speaker 11 output the sound signal transmitted from the controller 10 as sound. The receiver 7 is used, for example, to output the other party's voice during a call. The speaker 11 is used, for example, for outputting a ring tone and music. One of the receiver 7 and the speaker 11 may also function as the other. The microphone 8 is a sound input unit. The microphone 8 converts the user's voice or the like into a sound signal and transmits the sound signal to the controller 10.

  The storage 9 stores programs and data. The storage 9 is also used as a work area for temporarily storing the processing result of the controller 10. The storage 9 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage 9 may include a plurality of types of storage media. The storage 9 may include a combination of a portable storage medium such as a memory card, an optical disk, or a magneto-optical disk, and a storage medium reader. The storage 9 may include a storage device used as a temporary storage area such as a RAM (Random Access Memory).

  The programs stored in the storage 9 include an application executed in the foreground or the background and a control program that supports the operation of the application. For example, the application displays a screen on the display 2 </ b> A, and causes the controller 10 to execute a process according to a gesture detected by the touch screen 2 </ b> B. The control program is, for example, an OS. The application and the control program may be introduced into the storage 9 via wireless communication by the communication unit 6 or a non-transitory storage medium.

  The storage 9 stores, for example, a control program 9A, a mail application 9B, a browser application 9C, a health management application 9D, used device information 9X, determination information 9Y, and setting data 9Z. The control program 9A provides functions related to various controls for operating the mobile phone 100. For example, the control program 9A activates or terminates the application according to the detected operation.

  The mail application 9B provides an electronic mail function for creating, transmitting, receiving, and displaying an electronic mail. The browser application 9C provides a WEB browsing function for displaying a WEB page.

  The health management application 9D has a function of reading the measurement value displayed on the display unit of the health measuring instrument and transmitting it to the information providing apparatus 300, and a function of displaying the analysis result transmitted from the information providing apparatus 300 on the display 2A. provide.

  The used device information 9X includes information indicating the appearance characteristics of a health measuring device (hereinafter also referred to as “used device”) used by the user of the mobile phone 100, and measurement values displayed by the health measuring device. Information indicating the position and type of the. The information indicating the appearance characteristics of the health measuring instrument is, for example, the shape (the whole or part of the health measuring instrument), the color, the shape of the measurement value, the arrangement of the measurement value, the character string existing in the vicinity of the measurement value or It includes at least one of a symbol, other characters or symbols present on the surface of the health meter, and their positions. When a plurality of measurement values are displayed simultaneously on the display unit of the health measuring device, the information indicating the position and type of the measurement value includes information corresponding to each measurement value. When the measurement value displayed on the display unit of the health measuring device is switched, the information indicating the position and type of the measurement value includes information regarding the order in which the measurement values are displayed.

  As will be described later, information is added to the used device information 9X every time the mobile phone 100 reads a measurement value from a new health measuring device. The used device information 9X may be initially empty or may hold information corresponding to a pre-selected health measuring device.

  The determination information 9Y includes information for determining which measurement value exists in the captured image. Specifically, in the determination information 9Y, for each health measuring instrument existing in the market, information relating to appearance characteristics and information indicating the position and type of the measurement value displayed by the health measuring instrument are stored in association with each other. Is done. The information indicating the appearance characteristics of the health measuring instrument is, for example, the shape (the whole or part of the health measuring instrument), the color, the shape of the measurement value, the arrangement of the measurement value, the character string existing in the vicinity of the measurement value or It includes at least one of a symbol, other characters or symbols present on the surface of the health meter, and their positions. When a plurality of measurement values are displayed simultaneously on the display unit of the health measuring device, the information indicating the position and type of the measurement value includes information corresponding to each measurement value. When the measurement value displayed on the display unit of the health measuring device is switched, the information indicating the position and type of the measurement value includes information regarding the order in which the measurement values are displayed.

  In the determination information 9Y, a plurality of models having the same appearance characteristics and the same position and type of measurement values to be displayed may be collected and stored as one health measuring device. The determination information 9Y may not cover information on all health measuring instruments existing in the market.

  The determination information 9Y includes abstracted information so that the measurement value can be read from an unknown health measuring device. Specifically, the determination information 9Y stores information on characters or symbols that are likely to exist in the vicinity of the measurement value for each type of measurement value. Generally, at least one of the name or unit of the measurement value exists in the vicinity of the measurement value. By storing such a name or unit in association with the type of measurement value, it is possible to detect the measurement value in the vicinity using the name or unit as a clue and determine the type of the measurement value. Become. In this case, in order to easily detect the measurement value, the relative position of the measurement value corresponding to the name or unit may be stored. For example, when storing a unit, it may be further stored that a measurement value exists on the left side.

  In the determination information 9Y, information regarding the range of the measurement value is further stored as abstracted information for each type of measurement value. For example, when the measured value is body temperature, a range from 35 ° C. to 40 ° C. is stored as the range. For example, when the measured value is the maximum blood pressure, a range from 100 mmHg to 200 mmHg is stored as the range. When a numerical value is detected in the image, it is possible to determine whether the numerical value is a measured value and what type of measured value by comparing the numerical value with a range.

  In the determination information 9Y, information regarding the arrangement of measurement values is further stored as abstracted information. When a plurality of measurement values are displayed at the same time, the arrangement of the measurement values may have a law for each manufacturer or health measuring device. For example, in the case of a pulse meter, the highest blood pressure, the lowest blood pressure, and the pulse rate are often displayed vertically from the top. When multiple numerical values are detected in an image, the arrangement of the numerical values is checked against the law property to determine whether the numerical values are measured values and what type each is a measured value. be able to.

  The determination information 9Y includes information for determining the manufacturer and type of the health measuring device. Specifically, in the determination information 9Y, information on the character shape of the displayed measurement value is stored for each manufacturer. Further, the determination information 9Y stores information on the name or symbol of the company to be printed, stamped, or affixed on the surface of the health measuring device for each manufacturer. Further, the determination information 9Y stores information on the model number printed, stamped, or affixed on the surface of the health measuring device for each manufacturer and type of the health measuring device. In the determination information 9Y, information for determining the manufacturer and type of the health measuring device is stored in association with the information regarding the appearance characteristics of the health measuring device. The abstracted information included in the determination information 9Y may be stored for each manufacturer or type of health measuring instrument.

  The setting data 9Z includes information relating to various settings relating to the operation of the mobile phone 100.

  The controller 10 is an arithmetic processing device. The arithmetic processing unit includes, for example, a CPU (Central Processing Unit), an SoC (System-on-a-chip), an MCU (Micro Control Unit), and an FPGA (Field-Programmable Gate Array), but is not limited thereto. The controller 10 implements various functions by comprehensively controlling the operation of the mobile phone 100.

  Specifically, the controller 10 executes instructions included in the program stored in the storage 9 while referring to the data stored in the storage 9 as necessary. And the controller 10 controls a function part according to data and a command, and implement | achieves various functions by it. The functional unit includes, for example, the display 2A, the communication unit 6, the microphone 8, and the speaker 11, but is not limited thereto. The controller 10 may change the control according to the detection result of the detection unit. The detection unit includes, for example, a touch screen 2B, a button 3, an illuminance sensor 4, a proximity sensor 5, a receiver 7, a camera 12, a camera 13, an acceleration sensor 15, an orientation sensor 16, a gyroscope 17, and a GPS receiver 18. It is not limited to these.

  For example, the controller 10 executes a health management application 9D to read a measurement value displayed on the display unit of the health measuring instrument and transmit it to the information providing apparatus 300, and an analysis transmitted from the information providing apparatus 300. The function of displaying the result on the display 2A is realized.

  The camera 12 is an in camera that photographs an object facing the front face. The camera 13 is an out camera that captures an object facing the back face. The camera 13 is also used for photographing a health measuring device.

  The connector 14 is a terminal to which another device is connected. The connector 14 may be a general-purpose terminal such as USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), Light Peak (Thunderbolt (registered trademark)), and an earphone microphone connector. . The connector 14 may be a dedicated terminal such as a dock connector. Devices connected to connector 14 include, but are not limited to, external storage, speakers, and communication devices, for example.

  The acceleration sensor 15 detects the direction and magnitude of acceleration acting on the mobile phone 100. The direction sensor 16 detects the direction of geomagnetism. The gyroscope 17 detects the angle and angular velocity of the mobile phone 100. The detection results of the acceleration sensor 15, the azimuth sensor 16, and the gyroscope 17 are used in combination in order to detect changes in the position and orientation of the mobile phone 100. The GPS receiver 18 detects the position of the mobile phone 100.

  In FIG. 3, some or all of the programs and data stored in the storage 9 may be downloaded from another device through wireless communication by the communication unit 6. 3 may be stored in a non-transitory storage medium that can be read by a reading device included in the storage 9. 3 may be stored in a non-transitory storage medium that can be read by a reading device connected to the connector 14. Non-transitory storage media include, for example, optical disks such as CD (registered trademark), DVD (registered trademark), and Blu-ray (registered trademark), magneto-optical disks, magnetic storage media, memory cards, and solid-state storage media Including, but not limited to.

  The configuration of the mobile phone 100 shown in FIG. 3 is an example, and may be changed as appropriate without departing from the scope of the present invention. For example, in the example illustrated in FIG. 3, the mobile phone 100 includes two cameras, but the mobile phone 100 may include only the camera 13. In the example shown in FIG. 3, the mobile phone 100 includes four types of sensors for detecting the position and orientation, but the mobile phone 100 may not include some of these sensors. Alternatively, the mobile phone 100 may include another type of sensor for detecting at least one of a position and a posture.

  An example of reading a measurement value by the mobile phone 100 will be described with reference to FIGS. FIG. 4 is a diagram illustrating an example of the body composition meter 40 included in the image photographed by the camera 13. The body composition meter 40 includes a display unit 41, a manufacturer name 44 printed on the surface, and an electrode 45 for measuring a body fat percentage. The display unit 41 displays a measured value 42a indicating weight and a measured value 43a indicating body fat percentage. Further, a unit 42b indicating that the unit of weight is Kg is displayed on the right side of the measured value 42a, and a symbol indicating that the body fat percentage is displayed on the right side of the measured value 43a. 43b is displayed.

  When such a body composition meter 40 is included in the image, the mobile phone 100 includes the measured values 42a and 43a that are numerical values, a unit 42b in the vicinity of the measured value 42a, and a symbol 43b in the vicinity of the measured value 43a. Is detected. Furthermore, the mobile phone 100 acquires information related to the appearance characteristics of the body composition meter 40 including the measurement value 42a and the measurement value 43a. For example, the shape 40a of the body composition meter 40, the color of the body composition meter 40, the position of the display unit 41, the font and arrangement of information displayed on the display unit 41, the characters constituting the manufacturer name 44, the font of the manufacturer name 44, and The position, the position of the electrode 45, the position of the electrode 45, and the like are acquired as information relating to features on appearance.

  When the mobile phone 100 collates the acquired information with at least one of the used device information 9X and the determination information 9Y, the measured value 42a indicates the weight and the measured value 43a indicates the body fat percentage. judge. Then, the mobile phone 100 acquires the weight and body fat percentage as numerical data by the character recognition process.

  FIG. 5 is a diagram illustrating an example of an activity meter 50 included in an image photographed by the camera 13. The activity meter 50 includes a display unit 51, a model number 54 printed on the surface, a button 55, and a button 56. The display unit 51 displays a measurement value 52a indicating the number of steps and a measurement value 53a indicating the calorie consumption. Further, a unit 52b indicating that the unit of the number of steps is a step (STEP) is displayed to the right of the measured value 52a, and a unit of calorie consumption is Kcal to the right of the measured value 53a. Unit 53b is displayed.

  When such an active mass meter 50 is included in the image, the mobile phone 100 includes a measured value 52a and a measured value 53a that are numerical values, a unit 52b in the vicinity of the measured value 52a, and a unit 53b in the vicinity of the measured value 53a. Is detected. Furthermore, the mobile phone 100 acquires information related to appearance features of the activity meter 50 including the measurement value 52a and the measurement value 53a. For example, the shape 50a of the activity meter 50, the color of the activity meter 50, the position of the display unit 51, the font and arrangement of information displayed on the display unit 51, the characters constituting the model number 54, the font and position of the model number 54, The shapes, positions, and the like of the buttons 55 and 56 are acquired as information related to appearance characteristics.

  The mobile phone 100 determines that the measured value 52a indicates the number of steps and the measured value 53a indicates the calorie consumption by comparing the acquired information with at least one of the used device information 9X and the determination information 9Y. To do. Then, the cellular phone 100 acquires the number of steps and the calorie consumption as numerical data by the character recognition process. The timing at which the mobile phone 100 determines that the measured value 52a indicates the number of steps and the measured value 53a indicates calorie consumption may be after the character recognition process is performed.

  FIG. 6 is a diagram illustrating an example of a sphygmomanometer 60 included in an image photographed by the camera 13. The sphygmomanometer 60 includes a display unit 61 and a button 65. The display unit 61 displays a measurement value 62a indicating the maximum blood pressure, a measurement value 63a indicating the minimum blood pressure, and a measurement value 64a indicating the pulse rate. Further, a name 62b indicating that the measured value 62a indicates the maximum blood pressure is printed on the left side of the measured value 62a, and a name 63b indicating that the measured value 63a indicates the minimum blood pressure is displayed on the left side of the measured value 63a. Is printed on the left side of the measured value 64a, and a name 64b indicating that the measured value 64a indicates the pulse rate is printed.

  When such a sphygmomanometer 60 is included in the image, the mobile phone 100 uses the measured value 62a, the measured value 63a, and the measured value 64a that are numerical values, the name 62b in the vicinity of the measured value 62a, and the vicinity of the measured value 63a. And a name 64b in the vicinity of the measured value 64a are detected. Furthermore, the mobile phone 100 acquires information regarding the appearance characteristics of the sphygmomanometer 60 including the measurement value 62a, the measurement value 63a, and the measurement value 64a. For example, the shape 60a of the sphygmomanometer 60, the color of the sphygmomanometer 60, the position of the display unit 61, the font and arrangement of information displayed on the display unit 61, the characters constituting the names 62b to 64b, the fonts of the names 62b to 64b, and The position, the shape and position of the button 65, and the like are acquired as information relating to appearance characteristics.

  The mobile phone 100 collates the acquired information with at least one of the used device information 9X and the determination information 9Y, so that the measured value 62a indicates the maximum blood pressure, and the measured value 63a indicates the minimum blood pressure. It is determined that the measured value 64a indicates the pulse rate. Then, the cellular phone 100 acquires the systolic blood pressure, the diastolic blood pressure, and the pulse rate as numerical data by character recognition processing.

  The configuration of the information providing apparatus 300 will be described with reference to FIGS. 7 and 8. As illustrated in FIG. 7, the information providing apparatus 300 includes a communication unit 301, a controller 302, and a storage 303. The communication unit 301 enables communication with other devices based on a predetermined protocol.

  The controller 302 is an arithmetic processing unit such as a CPU (Central Processing Unit). The controller 302 implements various functions by comprehensively controlling the operation of the information providing apparatus 300. Specifically, the controller 302 executes instructions included in the program stored in the storage 303 while referring to the data stored in the storage 303 as necessary. Then, the controller 302 executes various types of information processing according to data and instructions.

  The controller 302 executes an analysis process based on the accumulated measurement value by executing the analysis program 304, for example.

  The storage 303 stores programs and data. The storage 303 is also used as a work area for temporarily storing the processing result of the controller 302. The storage 303 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage 303 may include a plurality of types of storage media. The storage 303 may include a combination of a portable storage medium such as a memory card, an optical disk, or a magneto-optical disk and a storage medium reading device. The storage 303 may include a storage device used as a temporary storage area such as a RAM (Random Access Memory).

  For example, the storage 303 stores an analysis program 304 and measurement value information 306. The analysis program 304 provides a function for executing analysis processing based on accumulated measurement values.

  In the measurement value information 306, measurement values transmitted from the mobile phone 100 are accumulated. An example of the measurement value information 306 is shown in FIG. As illustrated in FIG. 8, the measurement value information 306 includes items such as a user ID, a time stamp, and a type, and a plurality of items in which the measurement value and a name indicating the type are stored in association with each other. In the item of user ID, an identifier for identifying the user of the mobile phone 100 is stored. The time stamp item stores the date and time when the measurement value was read, the date and time when the measurement value was transmitted, or the date and time when the measurement value was received. In the type item, a value indicating the type of the health measuring device from which the measured value is read is stored.

  As in the example illustrated in FIG. 8, the measurement value information 306 is configured so that a plurality of types of measurement values can be stored in time series. For this reason, the information provision apparatus 300 can perform the advanced analysis regarding health based on the time-dependent change of several types of measured values.

  With reference to FIG. 9 and FIG. 10, a procedure of processing in which the mobile phone 100 reads and transmits a measurement value will be described. FIG. 9 is a flowchart illustrating a procedure of processing for reading and transmitting a measurement value. FIG. 10 is a diagram illustrating an example of screen transition when a measured value is read and transmitted. The operations shown in FIGS. 9 and 10 are realized by the controller 10 of the mobile phone 100 executing the health management application 9D.

  In order to execute the procedure shown in FIG. 9, the controller 10 first activates the health management application 9D. The health management application 9D is activated, for example, triggered by detection of a user operation. The operation for starting the health management application 9D may be an operation for a menu or an operation for an icon similar to the operation for starting another application. Alternatively, the operation for starting the health management application 9D may be an operation on the lock screen.

  The lock screen is a screen on which an operation for shifting the mobile phone 100 from the standby state to the normal state is performed. In step S21 shown in FIG. 10, the lock screen 31 is displayed on the display 2A. On the lock screen 31, an unlock icon 31a, a camera icon 31b, and a measured value reading icon 31c are arranged.

  When an operation for moving the unlock icon 31a in the direction of the camera icon 31b by flicking or dragging is detected by the touch screen 2B, the controller 10 activates an application for taking a picture. When an operation for moving the unlock icon 31a in the direction of the measurement value reading icon 31c by flicking or dragging is detected by the touch screen 2B, the controller 10 activates the health management application 9D. When an operation for moving the unlock icon 31a in the other direction by flicking or dragging is detected by the touch screen 2B, the controller 10 displays on the display 2A the screen that was displayed before the transition to the home screen or standby state. Let

  In this way, by configuring the health management application 9D to be activated by an operation on the lock screen, the user can quickly start reading the measurement value from the health measuring instrument.

  When the health management application 9D is activated, as shown in FIG. 9, the controller 10 first activates the camera 13 (step S101). Then, the controller 10 captures an image using the camera 13 (step S102), and displays the captured image on the display 2A (step S103).

  At this stage, as shown in step S22 of FIG. 10, a shooting screen 32 for shooting an image is displayed on the display 2A. When the user puts the camera 13 on the body composition meter in a state where it is placed on the body composition meter, as shown in step S22, the body composition meter displaying the weight and body fat percentage on the display unit is photographed.

  Subsequently, the controller 10 detects a number in the image captured by the camera 13 (step S104). When the number in the image cannot be detected, that is, when the health measuring instrument displaying the measurement value is not captured (No at Step S105), the controller 10 re-executes Step S102 and the subsequent steps.

  When the number in the image photographed by the camera 13 can be detected (step S105, Yes), the controller 10 detects a character near the number (step S106). Furthermore, the controller 10 acquires the shape and color of the device including numbers (step S107), and detects other characters and symbols included in the device (step S108). And the controller 10 acquires arrangement | positioning of the detected number, a character, and a symbol (step S109). The order of steps S106 to S108 is not limited to this.

  After acquiring information about the appearance characteristics of the health measuring instrument in this way, the controller 10 collates the acquired information with the used device information 9X (step S110). When there is a health measuring device having a degree of coincidence of information regarding appearance features higher than a predetermined value, that is, when a device used in the image is detected (Yes in step S111), the controller 10 performs step S115 and subsequent steps. Run.

  When the used device in the image is not detected (step S111, No), the controller 10 uses the acquired information on the appearance characteristic of the health measuring device as the determination information 9Y in order to detect the health measuring device other than the used device. Collation is performed (step S112). When the health measuring device in the image cannot be specified by collation (No at Step S113), the controller 10 re-executes Step S102 and the subsequent steps.

  When the health measuring device in the image is identified by the collation (step S113, Yes), the controller 10 adds the information of the identified health measuring device to the use device information 9X (step S114). Specifically, the information on the appearance characteristics acquired in steps S104 to S109 is information indicating the position and type of the measurement value of the data in the determination information 9Y corresponding to the health measuring instrument specified in step S113. And is added to the used device information 9X. As described above, by storing the information of the newly recognized health measuring device in the use device information 9X, the processing related to the health measuring device is accelerated from the next time. Thereafter, the controller 10 executes step S115 and subsequent steps.

  In step S115, the controller 10 displays the detection result display screen on the display 2A (step S115). Then, the controller 10 displays the manufacturer name and type of the health measuring instrument included in the photographed image on the detection result display screen (step S116).

  Subsequently, the controller 10 captures an image using the camera 13 (step S117), and displays the captured image on the detection result display screen (step S118). And the controller 10 detects the measured value in an image (step S119), and displays the detected measured value on a detection result display screen (step S120).

  At this stage, for example, a detection result display screen 33 as shown in step S23 of FIG. 10 is displayed on the display 2A. The detection result display screen 33 includes an image display area 33a, a transmission button 33b, and a detection result display area 33c. The image display area 33a is an area where a captured image is displayed. The controller 10 enlarges the area where the measurement value is displayed in the photographed image and displays it in the image display area 33a. For this reason, even in a situation where it is difficult to see the display unit of the health measuring device, the user can easily grasp the measured value. In step S120, the mobile phone 100 may be configured so that the detection result can be corrected using the touch screen 2B or the button 3 in step S120.

  The transmission button 33b is a button for instructing execution of transmission of the measurement value and the additional information to the information providing apparatus 300. The detection result display area 33c is an area where information obtained by analyzing a captured image is displayed.

  Subsequently, the controller 10 determines whether an instruction to transmit the measurement value and the additional information has been given (step S121). When the transmission instruction is not given (No at Step S121), the controller 10 re-executes Step S117 and the subsequent steps. In this way, the controller 10 repeats image acquisition, image display, measurement value detection, and detection value display until a transmission instruction is issued. For this reason, the mobile phone 100 can reflect the change in the detection result when the measurement value included in the image changes.

  For example, in the case of a body composition meter, it may take a while for the measured value to stabilize. Even in this case, since the controller 10 repeatedly executes the measurement value detection, the latest measurement value can be reflected in the detection result. Furthermore, in the case of a body composition meter, the type of measurement value displayed on the display unit may change according to the user's operation or over time. Even in this case, since the controller 10 repeatedly executes the measurement value detection, the newly displayed measurement value can be reflected in the detection result as shown in step S24 of FIG. In order to determine the type of the newly displayed measurement value, in step S119, the controller 10 performs detection of a character or symbol near the measurement value and matching of the detected character or symbol with the determination information 9Y. May be.

  Furthermore, since the controller 10 repeatedly executes the process of enlarging the area where the measurement value is displayed in the captured image and displaying it in the image display area 33a, even if the image is blurred due to camera shake, You can continue to display the same part of. For this reason, the mobile phone 100 can improve the visibility of the displayed image.

  Further, since the analysis of the image is repeatedly executed, the mobile phone 100 is made to read the model number of the health measuring device, and the type of the health measuring device and the position and type of the measurement value displayed on the display unit of the health measuring device are determined. The user can also perform an operation of confirming and then causing the mobile phone 100 to read the measurement value.

  When a transmission instruction is made (step S121, Yes), the controller 10 instructs the communication unit 6 to transmit the detected measurement value and additional information to the information providing apparatus 300 (step S122). The transmission instruction is performed, for example, when the user who has confirmed the measurement value displayed on the detection result display screen 33 taps the transmission button 33b using the finger F1 as shown in step S24 of FIG.

  Thereafter, when the analysis result is transmitted from the information providing apparatus 300, the analysis result is displayed on the display 2A. For example, the controller 10 displays the analysis result screen 34 shown in step S25 of FIG. 10 on the display 2A, for example. The analysis result screen 34 includes a history display area 34a that shows changes over time in a plurality of types of measurement values, and an advice display area 34b that displays advice obtained by analysis processing.

  Embodiment which this application discloses can be changed in the range which does not deviate from the summary and range of invention. Furthermore, the embodiments disclosed in the present application and modifications thereof can be appropriately combined. For example, the above embodiment may be modified as follows.

  For example, each program shown in FIGS. 3 and 7 may be divided into a plurality of modules, or may be combined with other programs.

  The arrangement of the functions of the mobile phone 100 and the information providing apparatus 300 may be changed as appropriate. For example, the mobile phone 100 may execute the function of the information providing apparatus 300. That is, the mobile phone 100 may execute accumulation and analysis of measurement values in addition to reading of measurement values.

  Alternatively, the information providing apparatus 300 may execute some of the functions of the mobile phone 100. When the information providing apparatus 300 executes a part of the functions of the mobile phone 100, the information providing apparatus 300 may execute a process for determining the position and type of the measurement value included in the image. In this case, as shown in FIG. 11, the mobile phone 100 transmits an image to the information providing apparatus 300, and the information providing apparatus 300 executes both the collation with the used device information 9 </ b> X and the collation with the determination information 9 </ b> Y. It may be configured. Alternatively, as shown in FIG. 12, when the mobile phone 100 executes collation with the used device information 9 </ b> X and determination of the position of the measurement value or the like is impossible, the mobile phone 100 transmits an image to the information providing apparatus 300. You may comprise so that the information provision apparatus 300 may perform collation with the determination information 9Y. In the configuration shown in FIG. 11, the load on the mobile phone 100 is reduced, and it is easy to cope with a new model of the health measuring device. With the configuration shown in FIG. 12, it is possible to reduce the time required for determining the position of the measurement value and the like while facilitating the correspondence to the new model of the health measuring instrument.

  When the information providing apparatus 300 executes a part of the functions of the mobile phone 100, the mobile phone 100 transmits an image and additional information to the information providing apparatus 300 as shown in FIG. In addition to the determination of the position or the like, the measurement value may be read. In the configuration shown in FIG. 13, the load on the mobile phone 100 is further reduced.

  Alternatively, some of the functions of the mobile phone 100 may be executed by a device other than the information providing device 300. For example, as illustrated in FIG. 14, a device determination apparatus 600 that determines the position of a measurement value or the like may be added to the information providing system.

  In the above embodiment, the example in which the mobile phone 100 reads the measurement value displayed on the display unit of the health measuring device has been described, but the method of reading the measurement value is not limited to this. For example, the mobile phone 100 may read the measurement value from the positional relationship between the needle and the scale of the analog health measuring device.

  When reading a measurement value of a measuring instrument that measures body weight such as a weight scale or a body composition meter, it is considered that the user is placed on the measuring instrument while holding the mobile phone 100. Therefore, in order to eliminate the influence of the weight of the mobile phone 100 on the measurement value, the weight of the mobile phone 100 is stored in advance, and the device displaying the measurement value is a measuring instrument that measures the body weight. The mobile phone 100 may be configured so as to subtract the weight of the mobile phone 100 from the measurement value indicating the body weight.

  The health measuring device may be configured to display the measured value not only as a numerical value but also as a code that the mobile phone 100 can easily read. In this case, not only the measured value but also the type and unit of the measured value may be encoded. The encoding can be realized using, for example, a one-dimensional barcode or a two-dimensional barcode. Or when the display part of a health measuring device is a 7 segment display, you may implement | achieve encoding by assigning the blink pattern of the element which comprises each character to a code | symbol.

  When personal information of users such as age, height, gender, etc. is registered in the health meter, the health meter may be configured to display not only the measured value but also the personal information on the display unit. . By configuring in this way, personal information that is not registered in the mobile phone 100 can be acquired from the health measuring device by the mobile phone 100 and transmitted to the information providing apparatus 300 to be useful for analysis. The personal information may be displayed as characters on the display unit of the health measuring device, or may be encoded and displayed as described above. The personal information may be stored in advance in the information providing apparatus 300.

(Embodiment 2)
In the first embodiment, the example has been described in which the mobile phone 100 reads numerical data corresponding to the measurement value by the character recognition process from the image obtained by photographing the measurement value displayed on the display unit of the health measuring device. At this time, an object other than the measurement value included in the image data being captured by the camera affects the adjustment of the exposure amount when the image is acquired from the image data, and thus the measurement value read from the image by the mobile phone 100 There is a risk of variations in accuracy. In particular, when the display unit 41 of the health measuring device is configured with a 7-segment display that displays black characters on a grayscale liquid crystal panel, the health measuring device 41 is obtained from the image data according to the ratio of white objects included in the image data. The measured value becomes unclear in the image. If the accuracy of the measurement value is poor, the information providing apparatus 300 cannot correctly analyze the user's health condition, and the quality of the service related to health management also deteriorates. Therefore, hereinafter, an example of control for the mobile phone 100 to accurately read the measurement value displayed on the display unit from the image obtained by photographing the display unit of the health measuring device will be described.

  The health management application 9D provides the functions described below in addition to the functions described in the first embodiment. The health management application 9D has a function of specifying a region in the image data on which the image (image) of the display unit of the health measuring device is displayed based on the luminance distribution of the image data of the health measuring device being captured by the camera 13. provide. Furthermore, the health management application 9D provides a function of executing photometric processing for the specified area.

  The controller 10 executes the health management application 9D, thereby displaying an image (image) of the display unit of the health measuring device based on the luminance distribution of the image data of the health measuring device being captured by the camera 13. Identify areas in the data. Further, the controller 10 executes the health management application 9D to execute a photometric process for the specified area.

FIG. 15 is a diagram illustrating an example of a state where image data is supplemented by a camera. FIG. 15 shows a case where image data of the body composition meter 40, which is a health measuring device, is supplemented. The user operates the camera 13 and the mobile phone 100 while confirming that the image data of the display unit 41 of the body composition meter 40 that is a subject to be photographed is displayed on the touch screen display 2 (display 2A). As shown in FIG. 15, the controller 10 displays the image data I ₁ of the body composition meter 40 that can be captured by the camera 13 on the display 2 </ b> A in accordance with a user operation.

FIG. 16 is a diagram showing an example of the luminance distribution of the image data shown in FIG. In FIG. 16, the horizontal axis indicates the luminance value, and the vertical axis indicates the frequency of the luminance value. The controller 10 calculates a luminance distribution of the image data _{I 1} shown in FIG. 15. As shown in FIG. 16, the image data I ₁ shown in FIG. 15 includes a luminance value “α” and a luminance value “β” that are peaks. The user focuses on the display unit 41 of body composition analyzer 40 whose images are to be captured, since the operation of the camera 13 and the cellular phone 100, the image data I _1, the display unit 41 of body composition analyzer 40 Many images are included. Furthermore, when the display unit 41 of the body composition meter 40 is configured by a liquid crystal panel that displays character information in gray scale, the luminance values of the pixels that configure the display unit 41 are approximately the same value. Therefore, it can be determined that there is a high possibility that one of the peak luminance value “α” and the luminance value “β” is the display unit 41.

FIG. 17 is a diagram illustrating an example of a position in the image data I ₁ of a pixel having a peak luminance value in the luminance distribution illustrated in FIG. FIG. 17 shows an example of the position in the image data I ₁ of the pixel P _α having the luminance value “α” that is the peak in the luminance distribution shown in FIG.

FIG. 18 is a diagram illustrating an example of a region that is cut out as a candidate for a region in which the image of the display unit 41 of the body composition meter 40 is displayed. The user focuses on the display unit 41 of body composition analyzer 40 whose images are to be captured, since the operation of the camera 13 and the cellular phone 100, the image data I _1, the display unit 41 of body composition analyzer 40 It can be predicted that many images are included. Therefore, the controller 10 refers to the position (FIG. 17) of the pixel P _α having the luminance value “α” that is the peak in the luminance distribution in the image data I ₁ , and the pixels P _α shown in FIG. The region A _α is cut out from the image data I ₁ as a candidate for a region where the image of the display unit 41 of the body composition meter 40 is projected. For example, the controller 10, when cutting out the candidate region where the image of the display unit 41 of body composition monitor 40 is projected from the image data I _1, for example, the pixel P _alpha having a luminance value becomes a peak is concentrated Cut out the smallest rectangle that contains the area. The controller 10 is not limited to cutting out in a rectangular shape, and can be cut out in various shapes such as a circle, an ellipse, and a triangle according to the shape of the region where the pixels _Pα are concentrated.

The controller 10 performs a character recognition process on the area A _α cut out from the image data I ₁ as a candidate area where the image of the display unit 41 of the body composition meter 40 is projected. Then, the controller 10, when the recognized character information in the area A _alpha identifies the region A _alpha as an area where the image of the display unit 41 of body composition monitor 40 is displayed.

If that could identify areas A _alpha as an area where the image of the display unit 41 of body composition monitor 40 is displayed, the controller 10 executes photometry processing intended for the area A _alpha. The controller 10, when performing photometric processing intended for the area A _alpha, segment metering, center-weighted metering, may be either a spot metering. When performing the photometry process using the divided photometry, the controller 10 divides the area _Aα into a plurality of areas, and calculates a control value for determining the exposure amount from the average of the light amounts of the divided areas. Controller 10, when performing photometric processing using a center-weighted metering, with emphasis on the light amount of the central part of the area of region A _alpha, calculates a control value for determining the exposure amount. Controller 10, when performing photometric processing using a spot metering, based on the small amount of a portion of the region A _alpha, calculates a control value for determining the exposure amount.

  FIG. 19 shows a procedure of processing for specifying a region where the image of the display unit of the health measuring device is displayed from the image data captured by the mobile phone 100 by the camera 13 and performing photometric processing for the specified region. An example will be described. The processing procedure shown in FIG. 19 is started when the controller 10 activates the health management application 9D. The health management application 9D is activated in the same way as in the first embodiment, for example, when a user operation is detected.

After starting the health management application 9D, the controller 10 starts the camera 13 and calculates the luminance distribution (see FIG. 16) of the image data I ₁ (see FIG. 15 and the like) being captured by the camera 13 (step S201). . Subsequently, the controller 10 selects one luminance value (for example, α) having a peak in the luminance distribution (step S202).

Subsequently, the controller 10 cuts out a region (for example, A _α , see FIG. 18) of the image data I ₁ where pixels having a peak luminance value (for example, P _α ) concentrate (step S203).

Subsequently, the controller 10 performs character recognition processing on the region (for example, A _α ) cut out in step S203, and determines whether or not character information can be detected in the region (step S204).

If the controller 10 can detect character information in the area cut out in step S203 as a result of the determination (step S204, Yes), the controller 10 displays the area cut out in step S203 as an image of the display unit 41 of the body composition meter 40. identifying a region in the image data _{I 1} that is displayed (step S205).

  Subsequently, the controller 10 performs a photometric process for the area specified in step S205 (step S206). Then, the controller 10 adjusts the exposure amount when the image data is captured and the image is acquired based on the result of the photometry process (step S207), and the processing procedure shown in FIG.

  In step S204, if the controller 10 cannot detect the character information in the area cut out in step S203 as a result of the determination (No in step S204), the controller 10 has not yet selected among the peak luminance values. It is determined whether there is something (step S208). As a result of the determination, if there is an unselected one (step S208, Yes), the controller 10 returns to step S202, selects one luminance value from the unselected luminance values, and executes the above-described procedure. To do. On the other hand, if the result of determination is that there is no unselected one (step S208, No), the controller 10 ends the processing procedure shown in FIG.

  As described above, in the second embodiment, the mobile phone 100 executes the health management application 9D, and thereby the health measuring device is configured based on the luminance distribution of the image data of the body composition meter 40 captured by the camera 13. An area in the image data in which the image (image) of the display unit is projected is specified. Further, the controller 10 executes the health management application 9D to execute a photometric process for the specified area. For this reason, according to Embodiment 2, the measured value currently displayed on the display part can be accurately read from the image which the display part of the health measuring device image | photographed.

In the second embodiment, the mobile phone 100 cuts out a region in the image data corresponding to the luminance value that is a peak in the luminance distribution as a candidate for a region where the image of the display unit is displayed. Then, when the cellular phone 100 can detect character information (for example, 7 segments) in the region cut out as a candidate, the image on the display unit displays the region in the image data corresponding to the peak luminance value. Identifies the area being projected. The user focuses on the display unit 41 of body composition analyzer 40 whose images are to be captured, since the operation of the camera 13 and the cellular phone 100, the image data I _1, the display unit 41 of body composition analyzer 40 It can be predicted that many images are included. Therefore, it is also effective to proceed with the processing of the corresponding area in the image data corresponding to the luminance value peaking in the luminance distribution as the area where the display unit is projected, but if character information can be recognized in the corresponding area, The area where the image of the display part is projected can be specified reliably.

  In the second embodiment, since the region of the display unit 41 is specified based on the luminance distribution of the image data of the body composition meter 40, for example, the position of the camera 13 is at a position not facing the body composition meter 40. Even so, the measurement value displayed on the display unit 41 can be read with high accuracy. That is, even when the image of the display unit 41 is taken from an oblique direction, since the photometric process for the display unit 41 is executed, the effect of light reflected from the display unit 41 or reflection on the display unit, etc. The measured value displayed on the display unit 41 can be read without receiving.

  In the second embodiment, the example in which the region in the image data in which the image of the display unit of the body composition meter is displayed is specified based on the luminance distribution of the image data of the body composition meter being captured by the camera. For example, a pattern design may be provided for the body composition meter so as to specify an area in the image data in which the image of the display unit of the body composition meter is displayed.

  FIG. 20 is a diagram illustrating an example of a pattern design for specifying a region in image data in which an image of the display unit of the body composition meter is displayed. As shown in FIG. 20, the pattern design includes three quadrangles 41 a arranged so as to be inscribed in three corners of the display portion 41 of the body composition meter 40 among the four corners of the display portion 41 of the body composition meter 40. And a straight line 41 b surrounding the display unit 41. When the controller 10 supplements the image data of the body composition meter 40 with the camera 13, the controller 10 recognizes the pattern design shown in FIG. 20 so that the image on the display unit 41 of the body composition meter 40 is displayed. Specify the area. The controller 10 may read the pattern design shown in FIG. 20 by pattern matching, or may read the pattern design shown in FIG. 20 by extracting an edge from the image data.

  By using the pattern design shown in FIG. 20, not only specifying the region where the image of the display unit 41 of the body composition meter 40 is projected, but also detecting the display content of the display unit 41 and changes in the display content, The area of the display unit 41 may be specified. For example, when the controller 10 executes character recognition processing of the image data of the body composition meter being captured by the camera 13 and can detect the character “under measurement”, the display unit 41 uses the position of the character as a guide. The area where the image of is displayed. Alternatively, when the controller 10 can detect a change in display content displayed on the display unit 41 during measurement by taking a difference between frames of the image data of the body composition meter 40 being captured by the camera 13, A region where the image of the display unit 41 is projected is specified by using the changed portion as a guide. Changes in display content include changes in numbers until the measured weight value is determined, changes in the case of displaying body fat from weight display, and the like.

  Alternatively, when the camera 13 captures image data of the display unit 41 of the body composition meter 40, a guide frame or the like for matching with the display unit 41 may be displayed on the display 2A. In this case, the area inside the guide frame in the image data supplemented by the camera 13 is specified as the area where the image of the display unit 41 is projected.

  In the second embodiment, the example of specifying the region of the display unit 41 of the body composition meter 40 has been described. However, a thermometer other than the body composition meter 40, a blood pressure meter that measures blood pressure, a pulse meter that measures a pulse, and the number of steps are measured. The display units of other health measuring devices such as a pedometer and an activity meter that measures the amount of activity due to exercise including walking can also be specified in the same manner as in the case of the body composition meter 40.

  In the second embodiment, when the color temperature of the display unit 41 of the body composition meter 40 and the type of light source are known in advance, the mobile phone 100 performs white according to the color temperature of the region of the display unit 41 and the type of light source. You can also adjust the balance.

  In the second embodiment, the mobile phone 100 may change the ISO sensitivity according to the result of the photometric process for the area where the image of the display unit 41 is displayed.

  In the second embodiment, the mobile phone 100 uses the used device information 9X, the determination information 9Y, and the like, so that a region where the display unit 41 is displayed in the image data of the body composition meter 40 being captured by the camera 13 to some extent. After narrowing down, the area where the display unit 41 is projected may be specified based on the luminance distribution of the narrowed area.

  The mobile phone 100 obtains a measurement value from an image obtained by adjusting the exposure amount based on the result of the photometry process for the area of the display unit 41, and then uses the used device information 9X and the determination information 9Y. The same processing as in the first embodiment (step S105 in FIG. 9, Yes to step S122) may be executed.

  In the above embodiment, a mobile phone has been described as an example of an electronic device, but the mobile electronic device according to the appended claims is not limited to a mobile phone. The electronic device according to the appended claims may be a portable electronic device other than a mobile phone. Examples of the portable electronic device include, but are not limited to, a tablet, a portable personal computer, a digital camera, a media player, an electronic book reader, a navigator, and a game machine. The electronic device according to the appended claims may be an electronic device other than the portable electronic device.

  The characterizing embodiments have been described in order to fully and clearly disclose the technology according to the appended claims. However, the appended claims should not be limited to the above-described embodiments, but all modifications and alternatives that can be created by those skilled in the art within the scope of the basic matters shown in this specification. Should be configured to embody such a configuration.

DESCRIPTION OF SYMBOLS 100 Mobile phone 2 Touch screen display 2A Display 2B Touch screen 3 Button 4 Illuminance sensor 5 Proximity sensor 6 Communication unit 7 Receiver 8 Microphone 9 Storage 9A Control program 9B Mail application 9C Browser application 9D Health management application 9X Device information 9Y Determination information 9Z Setting data 10 Controller 11 Speaker 12 Camera 13 Camera 14 Connector 15 Acceleration sensor 16 Direction sensor 17 Gyroscope 18 GPS receiver 200 Base station 300a, 300b Information providing device 301 Communication unit 302 Controller 303 Storage 304 Analysis program 306 Measurement value information 400 Voice recognition Device 500 Network 600 Device determination device

Claims (5)

An information providing system having an electronic device and an information providing device,
The electronic device is
A camera that captures image data of an imaging target including a display unit provided in the health measuring instrument for displaying a measurement value measured by the health measuring instrument;
An area in which the image of the display unit included in the image data captured by the camera is displayed is specified, a photometric process is performed on the specified area, and the imaging is performed based on a result of the photometric process A controller that adjusts the exposure when acquiring the target image;
An information providing system comprising: a communication unit that transmits information related to the measurement value included in the image to the information providing apparatus.   The controller of the electronic device can detect the character information in a region in the image data corresponding to the peak luminance value in the luminance distribution of the image data, and the controller corresponds to the peak luminance value. The information providing system according to claim 1, wherein an area in image data is specified as an area in which an image of the display unit included in the image data is projected. A camera that captures image data of an imaging target including a display unit provided in the health measuring instrument for displaying a measurement value measured by the health measuring instrument;
An area in which the image of the display unit included in the image data captured by the camera is displayed is specified, a photometric process is performed on the specified area, and the imaging is performed based on a result of the photometric process An electronic device comprising: a controller that adjusts an exposure amount when acquiring a target image. A control method for controlling an electronic device,
Capturing image data of an imaging target including a display unit provided in the health measuring instrument for displaying a measurement value measured by the health measuring instrument;
When an area in which the image of the display unit included in the image data is displayed is specified, photometric processing is performed on the specified area, and the image to be captured is acquired based on the result of the photometric processing Adjusting the amount of exposure of the control method. Electronic equipment,
Capturing image data of an imaging target including a display unit provided in the health measuring instrument for displaying a measurement value measured by the health measuring instrument;
When an area in which the image of the display unit included in the image data is displayed is specified, photometric processing is performed on the specified area, and the image to be captured is acquired based on the result of the photometric processing A control program that executes a step of adjusting the exposure amount of and.

Images