JP5809682B2 - Data detection system - Google Patents

Description

  The present invention relates to a data detection system that detects data from a device appearance image obtained by photographing the appearance of a device that displays data.

  In recent years, there is a system in which data output with a QR code (registered trademark) is read by a mobile phone with a camera and the data is registered in a database of a server via the Internet. According to such a system, manual input of data becomes unnecessary, so that the work burden on the user can be reduced.

JP 2008-269304 A

  By the way, as a system for detecting data from an image, the following is conceivable. In other words, this is a system in which a device that displays data on a liquid crystal screen or the like is photographed for each appearance, and data is detected from the obtained image (device appearance image).

  A user who uses this system first takes a picture of the device displaying the data with a smartphone or the like. A device appearance image obtained by shooting is displayed on the smartphone. The user designates the position of data in the displayed device appearance image. Thereby, the smart phone transmits an image of the designated position to the server. The server detects and stores data from the image using a technique such as optical character recognition.

  However, if a position other than the data display position is designated, it becomes difficult to detect data from the image at that position, and the accuracy of the detected data decreases.

  The present invention has been made in view of the above problems, and an object of the present invention is to improve the accuracy of data detected from a device appearance image obtained by photographing the appearance of a device that displays data.

  In order to solve the above-described problem, the data detection system of the present invention includes, for each device appearance image obtained by photographing the appearance of a device, a device feature amount that is a feature amount of the appearance detected from the device appearance image, the device Data display frame position information indicating the position specified in the device appearance image as the position of the data display frame of the device by the user using the data, and the data display frame at the position of the data display frame position information in the device appearance image A DB server having a device appearance image table for storing a record including success / failure indicating whether or not data has been correctly detected from an image, and a device feature for detecting a device feature amount from the device appearance image of the device displaying the data The amount was detected correctly from the amount detection unit and the device appearance image table. A control unit that retrieves a record including success / failure indicating the device feature and the device feature amount, and reads data display frame position information from the record, from the data display frame image at the position of the data display frame position information in the device appearance image And an analysis server including a data detection unit for detecting data.

  According to the data detection system of the present invention, it is possible to improve the accuracy of data detected from a device appearance image obtained by photographing the appearance of a device displaying data.

It is a figure which shows the structure of the data detection system which concerns on this Embodiment. It is the figure which illustrated the appearance of a device. 2 is a functional block diagram showing a schematic configuration of an analysis server 1. FIG. 3 is a functional block diagram showing a schematic configuration of a DB server 2. FIG. It is a figure which shows the information contained in each record of the device external appearance image table. It is a figure which shows the information contained in each record of the device feature-value table. It is a figure which shows the information contained in each record of the data table. It is a figure which shows the sequence in the case of the first data detection. It is a figure for demonstrating an example of the detection method of a device feature-value. It is a figure which shows the sequence in the case of the data detection after the 2nd time.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing a configuration of a data detection system according to the present embodiment.

  The data detection system according to the present embodiment includes an analysis server 1 and a DB server 2, and both are connected to a communication network N.

  In many cases, the data detection system is used by a large number of users, and each user has a device 3 for displaying data, and a communication device for photographing the appearance of the device 3 and transmitting the captured image. 4 is used. The communication device 4 is used to supplement a communication function that the device 3 does not have.

  FIG. 1 illustrates two devices 3 used by a user. One is a health care device that can measure and display the maximum blood pressure, the minimum blood pressure, and the pulse rate, and the other is a pedometer (registered trademark) that can measure and display the number of steps. Moreover, FIG. 1 illustrates the communication apparatus 4 called what is called a smart phone.

  As shown in FIG. 2A, the device 3 which is a health care device has, for example, a data display frame A1 for displaying the maximum blood pressure, a data display frame A2 for displaying the minimum blood pressure, and a data display for displaying the pulse rate. And a frame A3. As shown in FIG. 2B, the device 3 that is a pedometer includes a data display frame B1 that displays the number of steps.

  FIG. 3 is a functional block diagram illustrating a schematic configuration of the analysis server 1.

  The analysis server 1 includes a device feature amount detection unit 11 that detects a device feature amount that is a feature amount of the appearance of the device 3, and a data display frame image that is an image of a data display frame in a device appearance image obtained by photographing the appearance of the device 3. A data display frame position information detection unit 12 for detecting data display frame position information indicating the position of the data, a data detection unit 13 for detecting data displayed on the device 3 from the data display frame image, and control within the analysis server 1; And a control unit 14 that is responsible for communication with the DB server 2 and information acquisition from the DB server 2.

  FIG. 4 is a functional block diagram showing a schematic configuration of the DB server 2.

  A device appearance image table 21 that stores a record for each device appearance image obtained by photographing the appearance of the device 3, a device feature amount table 22 that stores a record for each identical device feature amount, and a record that includes detected data. And a control unit 24 that handles reading and writing of each table, communication with the communication device 4 and the analysis server 1, and the like.

  FIG. 5 is a diagram illustrating information included in each record of the device appearance image table 21.

  Each record of the device appearance image table 21 is a device appearance image ID that uniquely indicates the corresponding device appearance image, a shooting date and time when the device appearance image was taken, and a feature amount of the appearance of the device 3 detected from the device appearance image. Feature amount, data display frame position information indicating the position of the data display frame image specified in the device appearance image as the device data display frame image by the user using the device, and the position of the data display frame position information in the device appearance image And success / failure indicating whether or not the data is correctly detected from the data display frame image in FIG.

  FIG. 6 is a diagram illustrating information included in each record of the device feature amount table 22.

  Each record of the device feature table 22 includes a device feature ID uniquely indicating the corresponding device feature, a device feature, one or more data display frame position information obtained from a device appearance image corresponding to the device feature, The data display frame ID associated with each data display frame position information is included. The data display frame ID indicates a data display frame corresponding to the corresponding data display frame position information.

  FIG. 7 is a diagram illustrating information included in each record of the data table 23.

  Each record of the data table 23 includes one or more data detected from the corresponding device appearance image, a data display frame ID corresponding to each data, a user ID that uniquely indicates the user who captured the device appearance image, and a device It includes the device feature ID corresponding to the appearance image and the date and time of shooting.

  The figure shows a data display frame ID “HIGH” indicating a data display frame A1 for displaying the maximum blood pressure, a data display frame ID “LOW” indicating a data display frame A2 for displaying the minimum blood pressure, and a data display for displaying the pulse rate. The data display frame ID “PULSE” indicating the frame A3 is illustrated.

  FIG. 8 is a diagram showing a sequence in the first data detection.

  For example, the user operates the device 3 which is a healthcare device, and for example, the device 3 measures and displays the user's systolic blood pressure, diastolic blood pressure, and pulse rate (S1).

  The user captures the appearance of the device 3 with the communication device 4 that is a smartphone, for example (S3).

  The communication device 4 transmits a device appearance image that is a captured image to the analysis server 1 (S5).

  Alternatively, the device 3 that is a pedometer displays the number of steps (S1), and the communication device 4 captures the device 3 that displays the number of steps and transmits a device appearance image of the pedometer to the analysis server 1 (S5). .

  The device feature quantity detector 11 of the analysis server 1 detects a device feature quantity that is a feature quantity of the appearance from the device appearance image (S11).

  As illustrated in FIG. 9, the device feature amount detection unit 11 converts, for example, feature points in the device appearance image I into numerical values such as an X coordinate, a Y coordinate, a size, a Laplacian, a vector, a color, and the numerical values. The feature value.

  Returning to FIG.

  The control unit 14 of the analysis server 1 transmits the device feature amount and the device appearance image to the DB server 2 (S13).

  The DB server 2 stores the device appearance image in a table (not shown).

  On the screen of the communication device 4 on which the device appearance image is displayed, for example, for each data display frame A1, A2, A3, the user touches the position of the data display frame with a finger or the like and designates the position (S21).

  In addition, the user inputs data display frame IDs corresponding to the data display frames A1, A2, and A3 to the communication device 4 (S23).

  Thereby, the communication device 4 cuts out the data display frame image for each of the data display frames A1, A2, and A3 from the device appearance image based on the touched position (S25), and the data display frame image and the data display frame ID are extracted. The set is transmitted to the DB server 2 (S27).

  For the data display frame image, similar information may be extracted from a database (not shown) of device feature points, and thereby distortion correction may be performed. The distortion correction is, for example, processing for converting a data display frame image so that the data is not diagonal in a data display frame image in which displayed data is slanted.

  Further, so-called noise removal processing for removing unnecessary characters may be performed on the data display frame image. The accuracy of data detection can be improved by distortion correction and noise removal.

  The control unit 24 of the DB server 2 generates a record of each data display frame image in the device appearance image table 21, and in each record, the device appearance image ID corresponding to the received device appearance image, the shooting date and time of the device appearance image, The received device feature value is stored (S31).

  Further, the control unit 24 of the DB server 2 transmits each received data display frame image to the analysis server 1 (S33).

  The data display frame position information detection unit 12 of the analysis server 1 detects data display frame position information indicating the position of the data display frame image from the previously received device appearance image for each received data display frame image (S41). ).

  Further, the data detection unit 13 of the analysis server 1 detects data from each data display frame image (S43). For example, the data detection unit 13 detects numbers and characters using a technique of optical character recognition.

  For each data display frame image, the control unit 14 of the analysis server 1 transmits the corresponding data display frame position information and data set to the DB server 2 (S45).

  The control unit 24 of the DB server 2 stores each data display frame position information in the corresponding record of the device appearance image table 21 (S51).

  Further, the control unit 24 of the DB server 2 transmits each data to the communication device 4 (S53).

  The communication device 4 displays each data (S61).

  The user determines whether or not each data is correctly detected, that is, whether or not the data detection is successful. Specifically, the user determines whether or not the data displayed on the communication device 4 matches the data displayed on the device 3. Then, the user inputs success / failure to the communication device 4 (S63). The communication device 4 transmits success or failure for each data to the DB server 2 (S65).

  The control unit 24 of the DB server 2 stores each success / failure in the corresponding record of the device appearance image table 21 (S71).

  In addition, if the success data is 1 or more, the control unit 24 of the DB server 2 generates one record in the device feature table 22 and the device feature ID corresponding to the corresponding device feature, device feature, and success. The data display frame position information corresponding to the processed data and the data display frame ID are stored in association with each data display frame position information (S73).

  Further, if there is one or more success data, the control unit 24 of the DB server 2 generates one record in the data table 23, each data, a data display frame ID corresponding to each data, and a user ID that uniquely identifies the user. Then, the device feature amount ID corresponding to the device appearance image and the shooting date and time are stored (S75). The user ID may be obtained, for example, by performing user authentication at the time of use.

  FIG. 10 is a diagram illustrating a sequence in the second and subsequent data detection. It is assumed that the first data detection is successful.

  The user operates the device 3, and for example, the device 3 measures and displays the user's systolic blood pressure, diastolic blood pressure, and pulse rate (S1). This user may be a user mentioned in the description of FIG. 8 or another user.

  For example, the user captures the appearance of the device 3 with the communication device 4 that is a smartphone (S3).

  The communication device 4 transmits a device appearance image that is a captured image to the analysis server 1 (S5).

  The device feature quantity detector 11 of the analysis server 1 detects a device feature quantity that is a feature quantity of the appearance from the device appearance image (S11).

  The control unit 14 of the analysis server 1 transmits the device appearance image to the DB server 2 (S15).

  The DB server 2 stores the device appearance image in a table (not shown).

  The control unit 24 of the analysis server 1 searches the device appearance image table 21 of the DB server 2 for a record including success / failure indicating that the data has been correctly detected and the device feature amount, and data display frame position information from the record. Is read (S101).

  The data detection unit 13 of the analysis server 1 cuts out a data display frame image that is an image at a position indicated by the read data display frame position information from the device appearance image (S103), and detects data from the data display frame image (S103). S105). Since this data is detected using data display frame position information indicating the position of the data display frame image that has been successfully detected in the past, it can be said to be correct data.

  For each data display frame image, the control unit 14 of the analysis server 1 transmits the corresponding data display frame position information and data set to the DB server 2 (S107).

  For each data display frame position information, the control unit 24 of the DB server 2 searches the device feature amount table 22 for a record including the data display frame position information and the corresponding device feature amount, and uses the data display frame ID from the record. And device feature-value ID is read (S111).

  In addition, the control unit 24 of the DB server 2 generates one record in the data table 23 and stores each data, each read data display frame ID, a user ID, a read device feature ID, and a shooting date and time (S113). ). In this record, each data is associated with a corresponding data display frame ID.

  Further, the control unit 24 of the DB server 2 transmits each data to the communication device 4 (S115).

  The communication device 4 displays each data (S121).

  The user confirms that the detection of each data is successful. Specifically, the user confirms that the data displayed on the communication device 4 matches the data displayed on the device 3.

  Such data confirmation is performed for the sake of completeness, and may be omitted if it is not necessary.

  As described above, according to the data detection system according to the present embodiment, for each device appearance image obtained by photographing the appearance of the device 3, the device feature amount and device that are the feature amounts of the appearance detected from the device appearance image are obtained. Data display frame position information indicating the position specified in the device appearance image as the position of the device data display frame by the user, and data from the data display frame image at the position of the data display frame position information in the device appearance image Device feature amount detection for detecting a device feature amount from the device appearance image of the DB server 2 having the device appearance image table 21 for storing a record including success / failure indicating whether or not the data can be detected and the device 3 displaying the data Unit 11 and the device appearance image table 21 indicate that the data has been detected correctly. A control unit 14 that retrieves a record including availability and device feature amount, reads data display frame position information from the record, and a data detection unit that detects data from the data display frame image at the position of the data display frame position information in the device appearance image Therefore, the accuracy of the detected data can be improved.

  Further, once data detection is successful, it is unnecessary to specify the position of the data display frame thereafter. Therefore, the system can be operated early. In addition, since pre-registration of the data display frame is not required, there is a great merit especially when operating various types of devices. In addition, the device type can be distinguished by the device appearance image, and therefore, operation with a plurality of devices is possible.

  The DB server 2 also includes, for each device feature quantity, a device feature quantity ID indicating a device feature quantity, a device feature quantity, and a plurality of data display frame position information obtained from a device appearance image corresponding to the device feature quantity, A device feature amount table 22 that stores a record including a data display frame ID associated with each data display frame position information is provided. If data is detected, the device feature amount corresponding to the data is detected from the device feature amount table 22. And a record including data display frame position information is retrieved, and a device feature amount ID indicating a device feature amount and a data display frame ID associated with the data display frame position information are read from the record (S111). Since the quantity ID and the data display frame ID are stored in pairs (S113), they are displayed on the same device. A plurality of data can be differentiated management of the. That is, for example, the maximum blood pressure, the minimum blood pressure, and the pulse rate can be recognized without confusion.

  In the present embodiment, a smartphone is exemplified as the communication device. However, the communication device is not limited thereto, and the communication device may be, for example, a personal computer having a communication function. At that time, the image may be taken by a camera connected to a personal computer. At that time, the position of the data may be specified with a mouse or the like in the device appearance image displayed on the display.

  In the present embodiment, a health care device and a pedometer are illustrated as devices for displaying data. However, the present invention is not limited to this, and various devices such as home appliances and meters can be used for displaying data. Can be considered.

  In this embodiment, optical character recognition is exemplified as an example of a method for detecting data from an image. However, data detection is not limited to this. For example, data detection in which the weight is detected from the position of the scale needle is also possible.

  Moreover, in this Embodiment, although the analysis server 1 and DB server 2 were divided | segmented, you may comprise these in one server.

  Further, in the present embodiment, it is assumed that the data detection is successful in the first time, but the detection success may be performed after the second time.

DESCRIPTION OF SYMBOLS 1 ... Analysis server 2 ... DB server 3 ... Device 4 ... Communication apparatus 11 ... Device feature-value detection part 12 ... Data display frame position information detection part 13 ... Data detection part 14, 24 ... Control part 21 ... Device external appearance image table 22 ... Device feature table 23 ... Data table

Claims (2)

For each device appearance image obtained by capturing the appearance of the device, the device feature amount, which is the feature amount of the appearance detected from the device appearance image, and the position of the data display frame of the device by the user using the device Data display frame position information indicating the position specified in the image and a record including success / failure indicating whether or not the data was correctly detected from the data display frame image at the position of the data display frame position information in the device appearance image A DB server including a device appearance image table for storing
A device feature amount detection unit for detecting a device feature amount from a device appearance image of a device displaying data, a record indicating success or failure indicating that the data has been correctly detected from the device appearance image table, and the device feature amount And an analysis server provided with a data detection unit for detecting data from the data display frame image at the position of the data display frame position information in the device appearance image. A data detection system comprising: The DB server
For each same device feature amount, a device feature amount ID indicating the device feature amount, the device feature amount, a plurality of data display frame position information obtained from a device appearance image corresponding to the device feature amount, and each data display A device feature amount table for storing a record including a data display frame ID associated with the frame position information; if data is detected, a device feature amount and data display corresponding to the data are detected from the device feature amount table; A record including the frame position information is searched, and a device feature amount ID indicating the device feature amount and a data display frame ID associated with the data display frame position information are read from the record, and the data and the device feature amount are read. 2. The data according to claim 1, wherein the ID and the data display frame ID are stored in pairs. Detection system.

Images