JP6375084B1 - Computer system, hand-drawn object specifying method and program - Google Patents

Abstract

An object of the present invention is to provide a computer system, a hand-drawn object specifying method, and a program capable of specifying a hand-drawn object. A computer system for identifying an object drawn by hand-drawing accepts input of hand-drawn data, stores the object data to be hand-drawn, and extracts feature points of the received hand-drawn data by image analysis. The extracted feature points are collated with the stored object data, the object drawn in the hand-drawn data is identified from the collation result, and the identified result is displayed together with the probability according to the degree of specificity. . [Selection] Figure 1

Description

  The present invention relates to a computer system for specifying a hand-drawn object, a method for specifying a hand-drawn object, and a program.

  2. Description of the Related Art Conventionally, a target image shown in a captured image is specified by performing image analysis on a captured image captured by a camera or other imaging device.

  As a specific usage method of such an object, a key image as a search condition is subjected to image analysis, and an ID of the object shown in the key image is specified. A configuration is disclosed in which an image showing an object belonging to the same cluster as the object shown in the key image is selected by specifying the cluster to which this ID is assigned (see Patent Document 1).

JP 2011-107997 A

  However, the configuration of Patent Document 1 is effective for a captured image obtained by photographing an actual target, but it is difficult to specify the target depicting the real target.

  An object of the present invention is to provide a computer system, a hand-drawn object specifying method, and a program capable of specifying an object drawn by hand.

  The present invention provides the following solutions.

The present invention is a computer system for identifying an object drawn by hand,
Data input receiving means for receiving input of hand-drawn data;
Storage means for storing target data to be hand-drawn;
Extraction means for extracting feature points of the received hand-drawn data by image analysis;
Collating means for collating the extracted feature points with the stored target data;
A specifying means for specifying an object drawn in the hand-drawn data from the collation result;
Display means for displaying the identified result together with the probability according to the degree of specificity;
A computer system is provided.

  According to the present invention, a computer system for identifying an object drawn by hand drawing accepts input of hand-drawn data, stores the object data to be hand-drawn, and performs image analysis on the feature points of the received hand-drawn data. The extracted feature point is collated with the stored target data, and the target drawn in the hand-drawn data is identified from the collation result, and the identified result is combined with the probability according to the degree of specificity. indicate.

  Here, the present invention is a category of a computer system. However, in other categories such as a method or a program, the same actions and effects according to the category are exhibited.

  According to the present invention, it is possible to provide a computer system, a hand-drawn target specifying method, and a program that can specify a hand-drawn target.

FIG. 1 is a diagram showing an outline of the hand-drawn object specifying system 1. FIG. 2 is an overall configuration diagram of the hand-drawn object specifying system 1. FIG. 3 is a functional block diagram of the information terminal 100. FIG. 4 is a diagram illustrating a hand-drawn target specifying process executed by the information terminal 100. FIG. 5 is a diagram illustrating an example of hand-drawn data that the information terminal 100 has received input. FIG. 6 is a diagram illustrating an example of a state in which the result and probability specified by the information terminal 100 are displayed. FIG. 7 is a diagram illustrating an example of a state in which the result, probability, and basis specified by the information terminal 100 are displayed. FIG. 8 is a diagram illustrating an example of a state in which the result, probability, and highlight of target data specified by the information terminal 100 are displayed. FIG. 9 is a diagram illustrating an example of a state where the result, probability, and attribute information specified by the information terminal 100 are displayed.

  The best mode for carrying out the present invention will be described below with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

[Outline of hand-drawn object identification system 1]
An outline of a preferred embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram for explaining an outline of a hand-drawn object specifying system 1 which is a preferred embodiment of the present invention. The hand-drawing target specifying system 1 is configured from an information terminal 100.

  In FIG. 1, the number of information terminals 100 is not limited to one and may be plural. Further, the information terminal 100 is not limited to a real device, and may be a virtual device. The hand-drawing target specifying system 1 may include an external device such as a computer or a terminal device (not shown), and these may be connected to the communication terminal 100 so that data communication is possible.

  The information terminal 100 is a terminal device that can specify a hand-drawn target. The information terminal 100 is, for example, a cellular phone, a portable information terminal, a tablet terminal, a personal computer, an electric appliance such as a netbook terminal, a slate terminal, an electronic book terminal, a portable music player, a smart glass, a head mounted display, or the like Wearable terminals and other items.

  The information terminal 100 stores a plurality of different target data in advance (step S01). The information terminal 100, as target data, for example, captured image data obtained by capturing a part or all of various objects (artifacts, natural objects, etc.), other user, and other hand-drawn data that the user has performed hand-drawn input in the past Remember. The information terminal 100 stores text such as the name, category, description, and location of the target included in the captured image data and other hand-drawn data in association with the target data as attribute information. The target data and attribute information may be input by the user in advance or may be acquired from an external device.

  The information terminal 100 accepts input of hand-drawn data (step S02). For example, the information terminal 100 accepts an input as a hand-drawn image of a hand-drawn image drawn by a user and drawn by a user using an input device such as a touch panel. The information terminal 100 reads a hand-drawn image by a reading device such as a scanner connected to the information terminal 100 and accepts input as hand-drawn data. The hand-drawn data is, for example, data obtained by hand-drawing part or all of a target that the user desires to specify.

  The information terminal 100 extracts feature points of the hand-drawn data that has been accepted by image analysis (step S03). The information terminal 100 extracts characteristic points on the handwritten data. The information terminal 100 may be configured to extract feature amounts in addition to feature points, or may be configured to extract feature amounts instead of feature points.

  The information terminal 100 collates the extracted feature points with the stored target data (step S04). The information terminal 100 collates the extracted feature points with the feature points of the stored target data. The information terminal 100 may be configured to change the collation procedure depending on whether the data to be collated with the extracted feature points is captured image data or other hand-drawn data. For example, when the data to be collated is photographed image data, the information terminal 100 collates the entire feature points of the photographed image data, and thereafter collates the feature points of each part. Further, when the data to be collated is other hand-drawn data, the information terminal 100 collates the feature points of the respective parts. The change of the verification procedure executed by the information terminal 100 is not limited to the above-described configuration, and can be changed as appropriate.

  When the information terminal 100 collates the feature points with the target data, the information terminal 100 receives text input of a search category in advance, and further extracts target data from the category and the category associated with the target data. You may collate.

  The information terminal 100 specifies the target drawn in the hand-drawn data from the collation result (step S05). As a result of the collation, the information terminal 100 calculates a probability corresponding to the degree of specificity for the identified target. For example, the information terminal 100 obtains the degree of coincidence of each of the plurality of parts as the degree of specificity, and calculates the probability that the target drawn in the hand-drawn data matches the specified target according to the degree of specificity. To do.

  The information terminal 100 displays the identified result and the probability according to the degree of specificity together (step S06). When the information terminal 100 specifies a plurality of targets, the information terminal 100 displays a probability corresponding to the degree of specification on each target. When the information terminal 100 identifies one target, the information terminal 100 displays a probability corresponding to the degree of specificity on the target. The information terminal 100 may be configured to display a probability corresponding to the degree of specificity for each part of the identified target. Further, the information terminal 100 may be configured to display the basis for specifying the target together with the target. The information terminal 100 may be configured to display attribute information such as the name, category, description, and location of the identified target.

  In the processing described above, the storage of the target data, the extraction of the feature points, the matching of the feature points, and the specific combination of the targets need not necessarily be executed by the information terminal 100. For example, the information terminal 100 transmits hand-drawn data to an external device such as a computer or other terminal device (not shown), and the external device performs any one or a combination of the above-described processes instead of the information terminal 100. The configuration may be such that the external device transmits the identified result to the information terminal 100.

  The outline of the hand-drawn object specifying system 1 has been described above.

[System configuration of hand-drawn object identification system 1]
Based on FIG. 2, the system configuration of the hand-drawing target specifying system 1 which is a preferred embodiment of the present invention will be described. FIG. 2 is a diagram showing a system configuration of the hand-drawing target specifying system 1 which is a preferred embodiment of the present invention. The hand-drawing target specifying system 1 is a computer system that includes the information terminal 100. Note that the number of information terminals 100 is not limited to one and may be plural. Further, the information terminal 100 is not limited to a real device, and may be a virtual device. Further, it may be communicably connected to an external device such as a computer or a terminal device (not shown) through a public line network or the like.

  The information terminal 100 is the above-described terminal device having the functions described later.

[Description of each function]
Based on FIG. 3, the function of the hand-drawing target specifying system 1 which is a preferred embodiment of the present invention will be described. FIG. 3 is a functional block diagram of the information terminal 100.

  The information terminal 100 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like as the control unit 110, and enables communication with other devices as a communication unit (not shown). For example, a device compatible with WiFi (Wireless Fidelity) compliant with IEEE 802.11. In addition, the information terminal 100 includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card as the storage unit 130. The information terminal 100 stores, in the storage unit 130, target data such as other hand-drawn data and captured image data, which will be described later, and attribute information (name, category, description, location, etc.) associated with the target data. In addition, the information terminal 100 includes, as the input / output unit 140, a display unit that outputs and displays data and images controlled by the control unit 110, an input unit such as a touch panel, a keyboard, and a mouse that receives input from the user, and various images. Various devices such as an analysis device for performing analysis, an extraction device for extracting feature points and feature amounts of hand-drawn data, and a specific device for specifying an object included in the data are provided.

  In the information terminal 100, when the control unit 110 reads a predetermined program, the storage module 160 is realized in cooperation with the storage unit 130. Further, in the information terminal 100, the control unit 110 reads a predetermined program, so that the input receiving module 170, the extraction module 171, the matching module 172, the specifying module 173, and the display module 174 are cooperated with the input / output unit 140. Realize.

[Hand-drawn target identification process]
Based on FIG. 4, the hand-drawn object specifying process executed by the hand-drawn object specifying system 1 will be described. FIG. 4 is a diagram illustrating a flowchart of the hand-drawn target specifying process executed by the information terminal 100. Processing executed by each module described above will be described together with this processing.

  The storage module 160 stores a plurality of different target data in advance (step S10). In step S10, the storage module 160 captures image data obtained by capturing a part or all of various objects (artifacts or natural objects) as object data, other users, or other handwritten input by users in the past. Store hand-drawn data. At this time, the storage module 160 stores the attribute information (text such as the target name, category, description, and location) of the captured image data and other hand-drawn data in association with the target data. The other hand-drawn data is hand-drawn drawing data depicting a part or all of the object. The photographed image data is photographed image data obtained by photographing a part or all of the object. The storage module 160 stores, for example, target data received from an external device and attribute information of the target data, target data received by the input receiving module 170, and attribute information of the target data.

  The input reception module 170 receives input of hand-drawn data (step S11). In step S <b> 11, the input reception module 170 receives input as hand-drawn data to be specified by a user-drawn hand-drawn image using an input device such as a touch panel, for example. The input receiving module 170 reads a hand-drawn image by a reading device such as a scanner, for example, and receives an input of the read hand-drawn image as hand-drawn data.

  Note that the input receiving module 170 is not limited to the above-described configuration, and may receive input of hand-drawn data by other configurations. In addition, the hand-drawn image is not limited to the configuration described above, and any hand-drawn image may be used as long as the whole or part of the object is drawn. However, in order to specify with sufficient accuracy, it is desirable that the whole object or a characteristic part is drawn.

  The input receiving module 170 receives an input of attribute information of hand-drawn data that has been accepted (step S12). In step S12, the input reception module 170 receives input such as a category, a name, and a location as attribute information related to a target to be specified.

  Based on FIG. 5, a state in which the input receiving module 170 receives input of hand-drawn data and attribute information of the hand-drawn data will be described. FIG. 5 is a diagram illustrating an example of hand-drawn data that the information terminal 100 has received input. In FIG. 5, the input reception module 170 displays a hand-drawn data input area 210 and an attribute information input area 220 in the input area 200 and receives input respectively. The hand-drawn data input area 210 is an area that accepts hand-drawn input based on a tap operation from the user and displays the accepted hand-drawn image as hand-drawn data. The attribute information input area 220 is an area that accepts input of attribute information based on an input by a virtual keyboard operation from the user and displays the accepted attribute information. In the present embodiment, an input of a tower-like hand-drawn image is accepted as the hand-drawn data, and a state in which the observation platform is accepted as a text input as a category in the attribute information is shown.

  Note that the type and content of the hand-drawn image can be changed as appropriate, and input other than the category may be accepted as attribute information. Further, the input receiving module 170 may receive an input of a captured image instead of a hand-drawn image.

  The extraction module 171 extracts feature points of the hand-drawn data that has received the input by image analysis (step S13). In step S13, the extraction module 171 extracts characteristic points by extracting characteristic points of the hand-drawn data. For example, the extraction module 171 extracts each corner present in the hand-drawn data, and extracts the position of the corner and the positional relationship between the corners as feature points.

  The extraction module 171 extracts the feature amount of the hand-drawn data (step S14). In step S <b> 14, the extraction module 171 extracts feature quantities of each part such as line length, corner size, and line segment ratio of hand-drawn data. For example, the extraction module 171 extracts each part of the target existing in the hand-drawn data, and extracts feature quantities such as the size, position, and angle of each part.

  Note that only one of the processes in step S13 or step S14 may be performed. In this case, the information terminal 100 may be configured to execute each process described later based on the feature point or the feature amount after executing the process of either step S13 or step S14.

  The collation module 172 determines whether the type of the target data to be collated with the extracted feature points and feature amounts is other hand-drawn data or captured image data (step S15). The other hand-drawn data means hand-drawn image data that has been hand-drawn by another user or a user in the past. In step S15, the collation module 172 may determine the type of target data to be collated by receiving an input of which target data is collated from the user, or stores only one of the target data. If it is, it may be determined to collate the stored target data, or the type of target data may be determined based on a preset condition. In step S15, the collation module 172 may be configured to collate either the extracted feature point or feature quantity.

  In step S15, the collation module 172 performs the process of step S16 described later when the type of the target data is other hand-drawn data (YES in step S15). On the other hand, in step S15, when the type of the target data is captured image data (NO in step S15), the collation module 172 executes a process in step S17 described later. In other words, the information terminal 100 performs a matching procedure by executing different processes depending on whether the target data to be collated with the extracted feature points and feature amounts is other hand-drawn data or captured image data. change.

  In step S15, when the type of the target data is other hand-drawn data (YES in step S15), the matching module 172 is the target data storing the feature points and feature amounts extracted from the hand-drawn data. It collates with other hand-drawn data (step S16). In step S16, the collation module 172 collates the entire feature points of the other hand-drawn data, and thereafter collates the feature points and feature amounts of the respective parts. For example, the collation module 172 collates the feature points of the entire target included in the other hand-drawn data, and further collates the feature points and feature amounts of each part of the target.

  On the other hand, in step S15, if the type of the target data is captured image data (NO in step S15), the verification module 172 uses the stored target data to store the feature points and feature quantities extracted from the hand-drawn data. It is collated with certain captured image data (step S17). In step S <b> 17, the collation module 172 collates the feature points and feature amounts of each part of the captured image data. For example, the collation module 172 collates the target included in the captured image data, and collates the feature points and feature amounts of the respective parts of the target.

  Thus, in step S16 and step S17, the collation module 172 performs different collation procedures.

  In addition, the process of step S16 and step S17 mentioned above can be changed suitably. For example, the information terminal 100 may be configured to execute the same processing procedure in the processing of step S16 and step S17. Moreover, in the process of step S16 and step S17, the structure collated with object data by a different structure may be sufficient. For example, in step S16 and step S17, the information terminal 100 may be configured to collate based on only one of feature points or feature amounts.

  The collation module 172 further collates the target data based on the attribute information that has received the input in the process of step S12 (step S18). In step S18, the collation module 172 collates the attribute information associated with the target data and the attribute information received as hand-drawn data. In this embodiment, since the input of a category is received as attribute information, target data associated with a category corresponding to the received category is collated.

  The identification module 173 calculates, based on the collation result, a probability according to the degree of identification and the similarity of the attribute information between the target included in the hand-drawn data and the target included in the target data (step S19). In step S19, the degree of specificity means how much the target included in the hand-drawn data matches the target included in the target data. In step S19, the specifying module 173 calculates a probability according to the specific degree for each part in addition to the probability according to the overall specific degree of the target. Further, the specifying module 173 calculates a probability based on the similarity according to the attribute information for which the input has been received, based on the attribute information associated with the target data. In step S19, for example, the specifying module 173 calculates a probability according to the specific degree for each part, such as an angle, a line segment ratio, and a partial shape. Based on the probabilities of these parts, the identification module 173 calculates a probability according to the degree of specificity of the entire object. For example, when the line segment ratio is 90% and the other probabilities are 80%, the specifying module 173 adopts a low specificity and calculates that the probability is 80% as a whole. Further, the specifying module 173 recognizes the attribute information that has received the input, and calculates a probability based on the similarity between the attribute information and the attribute information associated with the target data. The identification module 173 calculates the probability based on the probability based on the specificity and the probability based on the similarity.

  In step S19, the calculation of the probability according to the degree of specificity executed by the identification module 173 is not limited to the configuration described above, and can be changed as appropriate. That is, the identification module 173 may be configured to calculate the target probability based on the partial position, may be configured to calculate the target probability based on a plurality of parts, or may be set in advance. The configuration may be such that the probability of the target is calculated based on the specific part, or a configuration other than that.

  The identification module 173 identifies the target drawn on the hand-drawn image based on the collation result (feature point, feature amount, and attribute information) and the calculated probability (step S20). In step S20, the identification module 173 determines whether the overall probability is equal to or higher than a predetermined value, whether there is even a whole or part probability or attribute information probability, or whether the attribute information probability is higher than a predetermined value. The object drawn on the hand-drawn image is specified based on the determination criteria such as the above. For example, if the identification module 173 matches the hand-drawn data among the target data to be stored, and if there is a slight probability of the whole or a partial position or the probability of attribute information, all the target data having this probability is It is specified as an object drawn in a hand-drawn picture. At this time, the specifying module 173 specifies one target data or a plurality of target data as a target drawn on a hand-drawn image. Further, for example, the identification module 173 identifies, among the target data to be stored, data whose overall probability is a predetermined value or more as a target drawn on a hand-drawn image. Further, the identification module 173 identifies an object having a probability of attribute information equal to or higher than a predetermined value as an object drawn on a hand-drawn image. In addition, the specifying module 173 specifies, as an object drawn on a hand-drawn image, one in which each of the probability based on the feature points and the feature amount and the probability based on the attribute information is a predetermined value or more.

  The display module 174 displays the identified result together with the probability according to the degree of specificity or similarity (step S21). In step S21, the display module 174 displays the identified target and the probability of this target. When there are a plurality of targets as a result of the identification, the display module 174 displays each target and the probability of each target. The display module 174 displays one or both of other hand-drawn data and captured image data of the specified target.

  Based on FIG. 6, a state in which the identified result displayed by the display module 174 and the probability according to the degree of specificity and the degree of similarity are displayed together will be described. FIG. 6 is a diagram illustrating an example of a state in which the identified result displayed by the display module 174 and the probability according to the degree of specificity and the degree of similarity are displayed together. In FIG. 6, the display module 174 displays a data display area 300, a target data display area 310, a probability display area 320, and an attribute information display area 330. The display module 174 displays a plurality of target data display areas 310, a plurality of probability display areas 320, and a plurality of attribute information display areas 330 in the data display area 300. The display module 174 displays the target data of the specified target in the target data display area 310. This target data is one or both of other hand-drawn data and captured image data stored in the storage module 160. Further, the display module 174 displays the probability display area 320 at the lower stage of each target data display area 310. The display module 174 displays in the establishment display area 320 the probabilities corresponding to the degree of specification and similarity of the target data displayed in each target data display area 310. The display module 174 displays the target data side by side in descending order of probability. The display module 174 displays the attribute information display area 330 in the lower part of each target data display area 310. The display module 174 displays attribute information associated with the target data in the attribute information display area 330. At this time, the attribute information displayed by the display module 174 displays information related to the attribute information received by the processing in step S12 described above.

  In FIG. 6, the display module 174 displays the target data A in the target data display area 310, and displays 80%, which is the probability corresponding to the degree of specification and similarity of the target data A, in the probability display area 320. In the attribute information display area 330, the observation platform X which is the name of the target data A is displayed. The display module 174 displays the target data B in the target data display area 310, displays 10% that is the probability according to the degree of specificity and similarity of the target data B in the probability display area 320, and displays attribute information. In the area 330, the transmission line that is the name of the target data B is displayed. The display module 174 displays the target data C in the target data display area 310, displays 5% which is a probability corresponding to the degree of specificity and similarity of the target data C in the probability display area 320, and displays attribute information. In the area 330, the tower X which is the name of the target data C is displayed. The display module 174 displays others in the target data display area 310, and displays 5%, which is the probability corresponding to the other specificities and similarities, in the probability display area 320. “Other” is displayed when there is no corresponding target data or when there is more than a predetermined number of corresponding target data. As the target data A to C described above, captured image data is displayed.

  Note that the number, position, shape, and contents of the target data display area 310, the probability display area 320, and the attribute information display area 330 displayed by the display module 174 can be changed as appropriate. Further, the target data A to C are not limited to photographed image data, but may be other hand-drawn data.

  Further, in step S21, the display module 174 may be configured to display the probability of each identified part of the target and the probability of attribute information. For example, the display module 174 is configured to display the calculated probability in the vicinity of each part, to display the probability calculated in a state in which the probability of the part is clearly indicated by a leader line, etc. The structure which displays the probability of a part, the structure which displays the probability of attribute information, etc. may be sufficient. Moreover, the structure which displays a probability by structures other than the structure mentioned above may be sufficient. For example, in FIG. 6 described above, the probability of this part may be displayed on the corresponding part of the target data.

  Further, in step S21, the display module 174 may be configured to display the basis for specifying the target together with the target. For example, the display module 174 is configured to display a notification that the target has been specified because the probability of each part exceeds a predetermined value, and the probability of some parts exceeds a predetermined value. Therefore, a notification is displayed to indicate that this target has been identified, and the probability of each part does not exceed a predetermined value, but the target with the highest overall probability compared to other identified targets Such as a configuration that displays a notification that the target has been identified, because the probability of the attribute information that has received the input exceeds or matches a predetermined value. May be. Moreover, the structure which displays a basis by structures other than the structure mentioned above may be sufficient. For example, in FIG. 6 described above, the target data or probability may be displayed and the basis of the target data or probability may be displayed.

  Based on FIG. 7, a description will be given of a state in which each part of the specified target displayed by the display module 174 and the probability of the attribute information and the basis for specifying the target are displayed. FIG. 7 is a diagram illustrating an example of a state in which each part of the specified target displayed by the display module 174 and the probability of the attribute information and the basis for specifying the target are displayed. In FIG. 7, the target data A in FIG. 6 is shown as an example. The display module 174 displays, for each target data in each target data display area 310 in FIG. 6 described above, the probability of each part and attribute information and the basis for specifying this target. In FIG. 7, the display module 174 displays the target data A in the target data display area 310 of the target data A, and the probability display area 320 has a probability corresponding to the degree of specification or similarity of the target data A 80. % Is displayed. Moreover, the display module 174 shows a leader line in each part of the target data A, for example, the ratio and outline of the line segment, and displays the probability corresponding to the degree of specification of this part. The display module 174 displays the line segment ratio of 80% and the outline specification degree of 90%. Further, the display module 174 displays the probability of the category of the target data A with respect to the input category as the similarity, and displays that the similarity is 90%. Moreover, the display module 174 displays that the probability according to the specific degree of each part and the similarity degree of attribute information exceeds a predetermined value as a basis for specifying the captured image data of the target data A. The display module 174 displays, as a basis, “the ratio and outline of the line segment exceed a predetermined probability. The same category is included.” Further, the display module 174 displays “Category matches or is similar” as the basis.

  In addition, the display mode of each part and the probability of attribute information displayed by the display module 174, the display content thereof, the display mode of the basis, and the display content thereof can be appropriately changed, and are not limited to the above-described configuration. The display module 174 may be configured to display any one or a combination of the probability of each part, the probability of attribute information, or the basis.

  In addition, the display module 174 has a low probability according to the overall specificity, but the probability according to the specificity of each part is high, and the similarity of the target data and attribute information with a high proportion of coincidence is very high. A configuration for highlight display may be used. That is, even if the probability according to the degree of specificity is low, the display module 174, for example, when the line segment ratios or outlines completely match, or when the input attribute information completely matches For example, when the specificities of the parts and the similarity of the attribute information are completely matched, the target data may be highlighted.

  Based on FIG. 8, a state in which the target probability specified by the display module 174, target attribute information, and highlights of target data are displayed will be described. FIG. 8 is a diagram illustrating an example of a state in which the target probability specified by the display module 174 and the highlight of the target data are displayed. The display module 174 has substantially the same display as that shown in FIG. 6, but the target data B is highlighted and displayed. The display module 174 has a lower probability according to the degree of specificity and similarity compared to the target data A, but the probability of other parts is substantially the same (close to 100%), and the attribute information is abbreviated. If they match (close to 100%), it is estimated that the target data B is a target drawn in a hand-drawn image, and the target data B is highlighted and displayed.

  Note that the display mode of the highlight displayed by the display module 174 can be changed as appropriate. The display module 174 may be configured to highlight the target data in a display mode other than highlight. For example, the display module 174 may be configured to display corresponding target data in an enlarged manner than other target data, or may be configured to display other target data in a reduced size. Other configurations such as blinking display and audio output may be used.

  Further, in step S21, the display module 174 may be configured to display a plurality of specified attribute information of the target together with the target. For example, the display module 174 may be configured to display the attribute information of the identified target in the vicinity of the target, or to display the attribute information in a state where the target is clearly indicated by a leader line or the like. . Moreover, the structure which displays attribute information by structures other than the structure mentioned above may be sufficient.

  Based on FIG. 9, a state in which the target probability specified by the display module 174 and the target attribute information are displayed will be described. FIG. 9 is a diagram illustrating an example of a state in which the target probability specified by the display module 174 and target attribute information are displayed. In FIG. 9, the display module 174 shows the target data A in FIG. 6 as an example. The display module 174 displays the target attribute information for each target data in each target data display area 300 in FIG. 6 described above. In FIG. 9, the display module 174 displays a name, a category, a description, and a location as attribute information.

  The attribute information is not limited to the configuration described above, and may be information regarding other objects. Further, the display mode and display position of the attribute information are not limited to the above-described configuration, and can be changed as appropriate.

  In each process described above, any one or a combination of the storage module 160, the extraction module 171, the collation module 172, or the identification module 173 does not necessarily have a configuration included in the information terminal 100. For example, a configuration in which an external device (not shown) executes these configurations may be used. In this case, the information terminal 100 may be configured to transmit hand-drawn data to an external device, and the external device executes each of these processes and transmits the execution result to the information terminal 100. Further, when the external device executes a part of each of these modules, the external device may execute the corresponding process and the execution result may be transmitted to the information terminal 100.

  The display module 174 described above may be configured to display any one or more of the above-described configurations in combination. For example, the configuration may be such that the target data, the probability of each part of the target data, the probability according to the degree of specificity, the probability according to the degree of similarity, the specified basis, and the attribute information are displayed. The target data, the probability according to the degree of specificity, the specified basis, the highlight display, and the attribute information may be displayed, or other combinations may be displayed. .

  The above is the hand-drawn object specifying process.

  The means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program. The program is provided, for example, in a form (SaaS: Software as a Service) provided from a computer via a network. The program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.). In this case, the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it. The program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

  1 Hand-drawn object identification system, 100 information terminals

Claims (7)

A computer system for identifying an object drawn by hand,
Data input receiving means for receiving input of hand-drawn data;
Storage means for storing target data to be hand-drawn;
Extraction means for extracting feature points of the received hand-drawn data by image analysis;
Collating means for collating the extracted feature points with the stored target data;
A specifying means for specifying an object drawn in the hand-drawn data from the collation result;
Display means for displaying the identified result together with the probability according to the degree of specificity;
A computer system comprising: The target data is other hand-drawn data stored in advance.
The computer system according to claim 1. The target data is photographed image data photographed and stored in advance.
The computer system according to claim 1. The target data is other hand-drawn data stored in advance and captured image data captured and stored in advance,
The collation means changes the collation procedure depending on whether the target data to be collated with the extracted feature points is the other hand-drawn data or the captured image data.
The computer system according to claim 1. Search text accepting means for accepting text input of a category for search;
With
The data input receiving means receives an input of the category of the hand-drawn data,
The storage means stores the hand-drawn data and the category of the hand-drawn data in association with each other,
The collation means collates the target data from the received category for search and the category of the stored hand-drawn data.
The computer system according to claim 1. A hand-drawn object identification method executed by a computer system for identifying an object drawn by hand,
A step of accepting input of hand-drawn data;
Storing target data to be hand-drawn;
Extracting the feature points of the received hand-drawn data by image analysis;
Collating the extracted feature points with the stored target data;
A step of identifying an object drawn in the hand-drawn data from the collation result;
Displaying the identified results together with the probability depending on the degree of specificity;
A hand-drawn object specifying method comprising: A computer system that identifies objects drawn by hand,
Accepting hand-drawn data input,
Storing target data to be hand-drawn;
Extracting the feature points of the received hand-drawn data by image analysis;
Collating the extracted feature points with the stored target data;
A step of identifying an object drawn in the hand-drawn data from the collation result;
Displaying the identified results with probabilities depending on the degree of specificity;
A computer-readable program for executing the program.

Images