JP6424756B2 - Data processing apparatus and data processing method - Google Patents

Description

  The present invention relates to a data processing apparatus, and more particularly to a data processing apparatus that associates items of new data whose item specification is unknown with known data whose item specification is known.

  When importing new data whose item specification is unknown into a master DB (database) whose item specification is known, it is desirable to determine which item corresponds to which item, and to collect it as data of the same item. If the item names (also called field names and attribute names) are similar, such as "vehicle speed", "speed", and "speed", it can be said that there is a high possibility of corresponding items, but not necessarily the same items. Not exclusively. Also, when an item name such as "s1" is given, it is difficult to determine from the item name that this is the same as "vehicle speed".

  In Patent Document 1, an item of a master DB having similar characteristics of the new data and the value is determined, and when there are a plurality of items having similar characteristics of the value, one of them is based on the similarity of the item name. It is stated to choose.

JP, 2006-099236, A JP 2001-155025 A JP, 2011-248661, A

  It is not always possible to determine the corresponding item precisely on the basis of the features of the data values. When there are a plurality of items having similar characteristics in the known data, it is difficult to determine which item corresponds to.

  In Patent Document 1, when there are a plurality of items having high similarity based on the features of data values, the corresponding items are selected based on the similarity of item names. However, if there is an item that is difficult to determine by the similarity of data values, there is a high possibility that an erroneous determination will occur. In addition, there is a high possibility that an erroneous determination may occur even when there are a plurality of items with high data value similarity and item names are not appropriately assigned.

  In view of the problems as described above, the present invention aims to provide a technique capable of accurately matching items with new data whose item specification is unknown and known data whose item specification is known. .

  In order to achieve the above object, a data processing apparatus according to the present invention holds information on discrimination accuracy by characteristics of data values for each item of known data. Then, the degree of similarity with data items of known data is obtained for new data, and among the items with high degree of similarity, known data corresponding to new data depending on whether or not the item with high determination accuracy by data values exists. Switch the decision method of the item candidate of.

More specifically, the data processing apparatus according to the present invention is a data processing apparatus that associates items with new data whose item specification is unknown and known data whose item specification is known. Storage means for storing information on discrimination accuracy according to the characteristics of data values of a plurality of items, acquisition means for acquiring item names and data values of new data, items of the known data corresponding to the new data Processing means for obtaining a candidate and outputting the item name of the candidate, wherein the processing means obtains, for the plurality of items of the known data, similarity of features of data values with the new data If there is an item with high determination accuracy due to the feature of the data value among the high-order predetermined items with high similarity of data value, the item with high determination accuracy due to the feature of the data value Are output together with the ranking according to the degree of similarity of the features of the data value, and when there are no items with high determination accuracy due to the features of the data value among the upper predetermined number of items with high degree of similarity of data values. Determines the similarity between the item name of the upper predetermined number of items and the item name of the new data, and outputs the item names of the upper predetermined number of items together with the ranking according to the similarity of the item names To be characterized.

  Known data whose item specification is known is data that is known as what data is stored in or to be stored in each item. The new data whose item specification is unknown is data whose item name (text) is known but whose data stored therein is what (representative) data is.

  The information on the determination accuracy based on the data value feature includes information indicating whether the determination accuracy based on the data value is high, and may be, for example, binary data indicating whether the determination accuracy based on the data value is high. It may be data that indicates the determination accuracy as a numerical value.

  The characteristics of the data values are, for example, the maximum value, the minimum value, the average value, the variance, or the maximum value, the minimum value, the average value within a predetermined period of time difference (time change) of data values within a predetermined period. It can be determined based on one or more of the variances. The predetermined period may be a predetermined and fixed period, or may be a period that changes according to the situation. For example, when the data to be handled is data concerning a vehicle, it is conceivable to set a period of one trip as a predetermined period.

  The processing means in the present invention calculates the degree of similarity between the feature of the data value of the new data and the feature of the data value of the known data, and obtains items of upper predetermined pieces of known data having a high degree of similarity. Here, the upper predetermined number of items may be a predetermined number of items counted from the one with the highest degree of similarity, or may be an item whose similarity is equal to or higher than a threshold.

  The processing means determines whether or not there is an item with high determination accuracy based on the feature of the data value among items of upper predetermined pieces of known data having high similarity. This determination is performed based on the information on the determination accuracy. If there is an item with high determination accuracy among the high-order predetermined items having high similarity of data values, the processing means ranks the items in the order according to the similarity of the data values from the items with high determination accuracy. Above, the candidate of the item of the known data corresponding to the new data is output. On the other hand, when there is no item with high determination accuracy among the items of high-order predetermined pieces of known data with high degree of similarity of data values, the processing means selects the high-order predetermined number of item names with high degree of similarity of data values. After ranking based on the degree of similarity of the item name (text), it is output as a candidate of items of known data corresponding to new data.

  According to such a configuration, for the item having high determination accuracy based on the data value, the corresponding candidate can be determined based on only the data value, so that the erroneous determination depending on the item name can be prevented. On the other hand, for an item having a low determination accuracy based on data values, it is possible to determine correspondence candidates using both data values and item names, so it is possible to prevent an erroneous determination depending only on data values. That is, according to the present invention, it becomes possible to accurately determine the candidate of the item of known data corresponding to the new data whose item specification is unknown.

  The processing means in the present invention can obtain the similarity of the feature of the data value using a learning device (classifier, classifier) previously learned using data whose item name is known. When creating a learning device, learning (supervised learning) processing is performed using data whose corresponding items are known as learning data. The learning algorithm may employ any known algorithm including Random Forest, SVM, and the like. In addition, it is determined whether or not the discrimination based on the data value can be correctly performed for each item of the known data using the learning device obtained as a result of the learning processing, and according to the result, the discrimination accuracy by the feature of the data value Information can be generated.

  In the present invention, it is also preferable to further include input means for receiving an input of the item of the known data corresponding to the new data, and performing re-learning of the learning device using the input to the input means. The item (a candidate may be included in the candidate or any other item) input by the user may be the correct answer to the corresponding item, and thus, the result may be used to perform relearning. Can improve the accuracy of the

  In the present invention, the processing means may obtain the degree of similarity of item names using an edit distance such as Levenshtein distance or may use an Euclidean distance.

  In the present invention, the processing means can obtain, as the upper predetermined number of items, items having a similarity of the data value equal to or higher than a predetermined threshold value. Further, preferably, the processing means outputs that there is no similar item, when there is no item whose similarity of the data value is equal to or more than the predetermined threshold value.

  The present invention can be grasped as a data processing apparatus provided with at least a part of the above-mentioned means. Further, the present invention can also be understood as a method of executing at least a part of the processing performed by the above means. Also, the present invention can be regarded as a computer program for causing a computer to execute this method, or a computer readable storage medium storing this computer program non-temporarily. Each of the above means and processes can be combined with one another as much as possible to constitute the present invention.

  According to the present invention, it is possible to accurately associate an item of new data whose item specification is unknown with an item of known data whose item specification is known.

FIG. 1 is a diagram showing a functional configuration of a data processing apparatus according to the embodiment. FIG. 2A is a flowchart showing the flow of prior learning processing, FIG. 2B is a conceptual diagram for explaining discrimination accuracy based on feature amounts, and FIG. 2C shows an example of discrimination accuracy information. FIG. 3 is a flowchart showing the flow of data integration processing according to the embodiment. FIG. 4 is a flowchart showing the flow of the corresponding item candidate determination process according to the embodiment.

<System outline>
The data processing apparatus 1 according to the present embodiment has a master DB (database) whose item specifications are known, and supports the association of items when capturing new data whose item specifications are unknown. Although the following description is given using data relating to the state of a vehicle (hereinafter also referred to as vehicle data) as a specific example of data, this does not limit the types of data to which the present invention can be applied.

  The data whose item specification is known is data in which the data stored for each item or data to be stored is known. Since the specifications of the database generated and managed by itself are clear, the data contained in the master DB have known item specifications.

  On the other hand, for new data whose item specification is unknown, although the text of the item name (also referred to as a field name or attribute name) is known, it is unknown whether the data stored therein is what data It is. Typically, data generated and managed by a third party (third party) is relevant.

  The data processing apparatus 1 according to the present embodiment acquires new data, and presents the user (operator) with candidate items of the master DB corresponding to the respective items included in the new data. Thus, the process of associating data items performed by the user can be facilitated. The data processing apparatus 1 according to the present embodiment stores, for each item in the master DB, whether or not the determination accuracy based on the feature of the data value is high, and uses this information to obtain a candidate for the corresponding item. Switch the person.

<Configuration>
The data processing apparatus 1 according to the present embodiment includes a CPU (central processing unit) and an MP.
Processor (operation processing unit) such as U (Micro Processing Unit), main storage device such as RAM (Random Access Memory), auxiliary storage device such as semiconductor memory, magnetic disk, optical disk, magneto-optical disk, keyboard and various pointing devices Input devices such as a mouse, touch pad, touch panel, pen tablet, etc., microphones, output devices such as display devices (liquid crystal displays, CRT displays, projectors, etc.) and audio output devices, communication for wired communication and wireless communication It is configured as a general-purpose computer (information processing apparatus) configured to include an apparatus and the like. The data processing device 1 provides the following functions by being executed by a processor in which a computer program stored in the auxiliary storage device is loaded on the main storage device. However, some or all of the following functions may be realized using an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or the like. In addition, the data processing device 1 does not necessarily have to be configured as a single computer, and the function may be provided by cooperation of a plurality of computers.

  FIG. 1 is a diagram showing functional blocks included in the data processing apparatus 1 according to the present embodiment. As shown in FIG. 1, the data processing apparatus 1 has a new data input unit 10, a data processing unit 20, an input / output unit 30, a master DB 40, and a discrimination accuracy information storage unit 50 as its functional units.

  The new data input unit 10 is a functional unit that acquires new data generated by another device. The method of acquiring data is not particularly limited, and acquisition by wired communication or wireless communication or acquisition via a storage medium is possible.

  The data processing unit 20 has a function of determining the candidate of the item of the master DB corresponding to each item of the new data, and also has a learning function of learning data required when determining the corresponding candidate. More specifically, the data processing unit 20 includes sub-function units of a data value similarity calculation unit 21, an item name similarity calculation unit 22, a correspondence item candidate determination unit 23, a learning processing unit 24, and a feature amount calculation unit 25. . The functions of the data processing unit 20 will be described in detail together with the flowchart below.

The input / output unit 30 has a function of presenting to the user an item candidate corresponding to each item of the new data determined by the data processing unit 20, and a function of accepting the selection of the corresponding item by the user.

  The master DB 40 stores data whose item specifications are known. Data of one record stored in the master DB 40 is, for example, “steering angle” “accelerator opening” “speed” “yaw rate” “right blinker” “left blinker” “right rear distance sensor value” “left rear distance sensor value Has a data item with "" as an item name. The specifications of these data items are known to (the manufacturer / administrator / user of) the data processing device 1. For example, the data item "steering angle" is data representing the rotation angle of the steering wheel in a unit of one degree, with the right direction (clockwise) as positive and the left direction (counterclockwise) as negative. Specifications are defined.

  The information included in the master DB 40 is not limited to the data items described above. For example, the identifier of the device or user who created the data, the creation date of the data, the registration date, etc. are included. In addition, since vehicle data is handled in the present embodiment, each record also includes a trip identifier, and vehicle data in the same trip can be acquired. In addition, a trip means the movement from a certain point (start point) to another point (end point). Here, the start point and the end point can be determined by various methods.

  In the present embodiment, in the discrimination accuracy information storage unit 50, information on whether or not each item of the master DB 40 can be accurately discriminated (classified and identified) based on the characteristics of data values (hereinafter referred to as discrimination accuracy information) Is stored. The determination accuracy information is generated by learning processing by the data processing unit 20, and is referred to in the determination processing of the corresponding item candidate by the data processing unit 20.

<Processing>
The processing performed by the data processing apparatus 1 according to the present embodiment is roughly divided into two: pre-learning processing performed before data integration processing, and data integration processing including processing for determining corresponding item candidates using a learning result . Below, the content of each process is demonstrated in detail.

(1. Prior learning process)
FIG. 2A is a flowchart illustrating the flow of the prior learning process performed by the data processing unit 20. In this prior learning process, it is determined using machine learning whether or not the discrimination accuracy based on the features of the data value is high for each item of the master DB 40.

  In step S10, the data processing unit 20 acquires data from the master DB 40, and uses the feature amount calculation unit 25 to calculate feature amounts of data values for each trip and each item. The feature value of the data value is, for example, the maximum value, the minimum value, the average value, the variance of the data value within one trip, or the maximum value, the minimum value, the average value of the time change (time difference) of the data value within one trip. , Represented as one or more combinations of variances. The time change may be a difference between data adjacent in time series or may be a difference between data separated by a predetermined period.

  The data used for learning need not be limited to the data stored in the master DB 40, and any data may be used as long as it is data of which item (including the case where it is not data of any item) is known. It can be used for learning.

  In step S11, the learning processing unit 24 generates a classifier (a classifier, a learning device) for identifying which item of data is data from the feature of the data value using a machine learning algorithm. The learning algorithm is not particularly limited, and any algorithm can be adopted as long as it can generate a classifier capable of the above classification. For example, an algorithm such as Random Forest or SVM can be adopted.

In step S12, the learning processing unit 24 acquires evaluation data of each item, calculates the feature amount of the data value, and evaluates the determination accuracy using the generated learning device. The evaluation data may be data stored in the master DB 40 or other data as long as the correct answer of which item of data is known. It is also preferable to use cross-validation such as k-fold cross-validation for evaluation. Moreover, although what combined one or more of a precision (Recision), a recall (Recall), and F value (F-measure) can be used as an evaluation index, you may use the other evaluation index. .

  In step S13, the learning processing unit 24 determines whether the evaluation value for each item is equal to or greater than a predetermined threshold, and determines that the item whose evaluation value is equal to or greater than the threshold is an item that can be accurately determined. An item whose evaluation value is less than the threshold value is determined to be an item which can not be determined with high accuracy. The learning processing unit 24 stores the determination result in the determination accuracy information storage unit 50 as determination accuracy information.

  Classification (identification) using a classifier will be described with reference to FIG. 2B. FIG. 2B is a conceptual diagram in which feature quantities obtained for each trip for items of “steering angle”, “right turn signal”, and “left turn signal” are plotted in a feature amount space. In the figure, triangular marks indicate steering angle feature amounts, circle marks indicate right blinker feature amounts, and cross marks indicate left blinker feature amounts. Here, since the feature quantities of the steering angle are separated on the feature quantity space from the feature quantities of other data, it is possible to accurately determine based on the feature quantities of data values. On the other hand, the feature quantities of the right blinker and the left blinker are mixed at relatively close positions in the feature space, and accurate determination is difficult based on the feature quantities of data values.

  FIG. 2C is an example of the determination accuracy information stored in the determination accuracy information storage unit 50. Here, it is determined that "steering angle", "accelerator opening", "speed" and "yaw rate" can be accurately determined based on the characteristics of data values, and "right blinker" "left blinker" "right rear distance sensor The values “left rear distance sensor value” are determined to be difficult to accurately determine based on the characteristics of the data values.

(2. Data integration process)
Next, data integration processing for integrating new data into the master DB 40 in the data processing apparatus 1 will be described with reference to the flowcharts of FIGS. 3 and 4.

  In step S20, new data to be integrated is acquired from the new data input unit 10. The new data input unit 10 acquires new data to be integrated, for example, from a device that has generated data or from a device that aggregates (collects) data from a plurality of devices by communication (wired communication or wireless communication). . Here, it is assumed that the new data includes a plurality of items in one record, and is input in a format in which records belonging to the same trip can be grasped.

  In step S21, the data processing unit 20 selects one data item for which a candidate of the corresponding item of the master DB 40 is to be obtained from the data items included in the new data. Hereinafter, the data item selected here is referred to as a target data item, and data of the data item is referred to as target data.

  In step S22, the data processing unit 20 determines an item candidate of the master DB 40 corresponding to the item of the target data. Details of the corresponding item candidate determination process of step S22 are shown in FIG.

In step S30, the data value similarity calculation unit 21 calculates the similarity of data values between the target data item and each item of the master DB 40. Specifically, the feature amount of the target data is obtained by the feature amount calculation unit 25, and the degree of similarity of the feature amount with each item of the master DB 40 by the classifier generated by the learning processing unit 24 by the prior learning processing To calculate the similarity of The method of calculating the feature amount of the target data is the same as the prior learning process (step S10), and thus the detailed description is omitted.

  In step S31, the correspondence item candidate determination unit 23 determines whether or not there is an item with high determination accuracy based on the data value among the high-order data items with high value similarity. The upper data item having high value similarity may be determined as a data item whose value similarity is equal to or higher than a predetermined threshold, or as a predetermined number of data items determined in advance from the one having high value similarity. You may decide. Whether the item has a high determination accuracy based on the data value can be determined using the determination accuracy information stored in the determination accuracy information storage unit 50.

  If there is an item having a high determination accuracy based on the data value among the upper data items having high value similarity (S31-YES), the process proceeds to step S32. In step S32, the correspondence item candidate determination unit 23 determines an item having a value similarity value equal to or higher than the threshold value and having a high determination accuracy based on the data value as a candidate for the item corresponding to the target data item. . When there are a plurality of data items corresponding to this condition, the corresponding item candidate determination unit 23 determines the higher the degree of priority (priority) as the similarity of the value is higher, and then the plurality of data items correspond to the corresponding items. Decide as a candidate.

  On the other hand, if there is no item with high determination accuracy based on the data value among the upper data items with high value similarity (S31-NO), the process proceeds to step S33. In step S33, the item name similarity calculation unit 22 calculates the similarity of the item name (text) between the target data item and each item of the master DB 40. Euclidean distance may be used as an index of the degree of similarity of item names, or editing distance such as Levenshtein distance may be used. In addition, a dictionary storing the degree of similarity of vocabulary may be provided, and the degree of similarity of the item name may be calculated with reference to this dictionary.

  In step S34, the correspondence item candidate determination unit 23 determines a data item whose value similarity is equal to or more than a threshold value as a candidate for the item corresponding to the target data item. At this time, the correspondence item candidate determination unit 23 determines the order (priority) as a candidate to be higher as the similarity of the item name is higher.

  This is the end of the process of determining the corresponding item candidate in step S22.

  The description will return to the flowchart of FIG. In step S23, the input / output unit 30 presents the user with the correspondence item candidate determined by the correspondence item candidate determination unit 23. This presentation is preferably presented to the user in a form in which the order (priority) of each candidate is known, and in a form in which the candidates can be selected. For example, it is possible to present in the order according to the order, or to present with a numerical value indicating the order. In addition, when presenting the corresponding item candidate, whether these candidates (and priorities) are determined based on the similarity of the values (determined in the process of step S32) or based on the similarity of the item name It is also preferable to perform presentation in a form that shows whether it has been determined (determined in the process of step S34). In addition, when the candidate is determined based on the similarity of the value, the similarity of the value is displayed together, or the candidate with particularly high similarity of the value is highlighted (color is changed or bolded) , Highlighting, etc.). Similarly, when the candidate is determined based on the similarity of the item names, the similarity of the item names is displayed together, or the item name having a particularly high similarity or value similarity is emphasized. It may be displayed.

In addition, when presenting item candidates to the user, it is preferable to present any of them in a format that allows the user to select. It is also preferable that the user can input the corresponding item by himself. Thereby, the data processing apparatus 1 can acquire the corresponding item selected by the user.

  In addition, in the determination process of the corresponding item candidate in step S22, it is also assumed that there is no item corresponding to the candidate. Specifically, there is no item in which the value similarity is equal to or higher than the threshold. In such a case, in step S23, it may be suggested to the user that there is no corresponding item candidate.

  The user can select any of the corresponding item candidates presented in step S23 or can select any of the items not presented as candidates. In step S24, the data processing unit 20 acquires the input of the item name corresponding to the target data item from the user via the input / output unit 30.

  In step S25, the data processing unit 20 takes in target data as data of an item input by the user into the master DB 40 and integrates the data. In addition, if format conversion of data etc. is necessary at the time of taking in of data, the conversion method may be acquired from the user, and after performing the conversion, the data may be taken into the master DB 40.

  In step S26, it is determined whether there is an unprocessed data item in the integration target data (new data), and if there is an unprocessed data item (S26-YES), the process returns to step S21, and the next data item Repeat the above process for. If the processing of all data items is completed (S26-NO), the process proceeds to step S27.

  In step S27, the learning processing unit 24 performs relearning of the classifier using each data item of the new data and the corresponding item input from the user in step S24. The learning processing unit 24 can improve the classification accuracy of the classifier by performing the relearning process with the item input by the user as the correct answer of the corresponding item. It is also preferable to calculate discrimination accuracy information anew in accordance with the relearning of the classifier.

<Benefit of this embodiment>
In the data processing apparatus 1 according to the present embodiment, items included in the master DB 40 by prior learning are classified into items having high determination accuracy based on the features of data values and items having low determination accuracy. Therefore, when obtaining the corresponding item candidate, it is possible to make a determination using the information on the determination accuracy.

  Further, in the present embodiment, for the item having high determination accuracy based on the feature of the data value, the corresponding item candidate is determined based on the feature of the data value without considering the item name. That is, if it can be determined with sufficient accuracy based on the characteristics of the data value, the item name is not used. Thereby, the determination accuracy of the corresponding item candidate can be improved. This is because the naming of item names is arbitrary, and when considering item names for items that can be accurately determined based on data values, the accuracy of determination may be degraded.

  Further, in the present embodiment, if the item having high determination accuracy based on the characteristic of the data value does not exist in the upper item with the value similarity, the corresponding item candidate is determined in the order according to the item name similarity. ing. That is, the item name is used when the determination can not be made with sufficient accuracy based on only the feature of the data value. In such a situation, since it is not possible to determine the corresponding item candidate with high accuracy from the feature of the data value, it is possible to improve the determination accuracy of the corresponding item candidate by using the item name.

  Since the data processing apparatus according to the present embodiment can present to the user the candidate of the item corresponding to the data to be taken in when taking in the data generated / managed by another person or another company, the data processing apparatus needs to associate the data items. The burden on the user can be reduced. In the current situation where technology for deriving useful information and knowledge based on a large amount of data (big data) has been developed, processing for preparing a large amount of data (before analysis) using the data processing apparatus according to the present embodiment Processing) becomes easy.

  Further, in the present embodiment, since the item corresponding to the target data is received from the user and the relearning process of the classifier is performed using this result, the accuracy of the classifier is further improved as the use progresses. I can expect it.

<Modification>
Although vehicle data is taken as an example in the above description, it will be apparent to those skilled in the art that the present invention is applicable to any data regardless of the type of data.

  Further, although the feature amount is obtained from data in one trip, the method for obtaining the feature amount is not particularly limited as long as the feature of the data can be expressed and the identification using the feature amount is possible. For example, the feature amount may be calculated using data for a predetermined period determined in advance, not data for one trip. Alternatively, the feature amount may be calculated using a predetermined number of data adjacent in time. Further, it is also preferable to appropriately determine the method of calculating the feature amount according to the type of data to be handled.

  In the above, an example has been described in which the determination accuracy information is information indicating whether it is possible or not to perform accurate determination based on the characteristics of data values. Discrimination accuracy information does not have to be such binary information, and is information represented by numerical values (for example, 1 to 10 or 1 to 100) even if it is information obtained by dividing the height of discrimination accuracy by three or more steps. It may be. In this case, in the determination in step S31 whether the item is high in the determination accuracy, it may be determined that the item in which the determination accuracy information is equal to or more than a predetermined value is high in the determination accuracy. The threshold may be a predetermined fixed value, or may be a value determined in consideration of the degree of similarity with each item obtained in step S30.

  The above description of the embodiment and the modification is merely an example for describing the embodiment of the present invention, and is not intended to limit the present invention to the scope of the disclosure. In addition, the present invention can be implemented by combining the element techniques described in the above-described embodiment and the respective modifications, as long as no technical contradiction arises.

1 data processing apparatus 10 new data input unit 20 data processing unit 30 input / output unit 40 master database (DB)
50 Discrimination accuracy information storage unit

Claims (11)

A data processing apparatus for associating an item with new data whose item specification is unknown and known data whose item specification is known,
Storage means for storing information on discrimination accuracy according to features of data values for a plurality of items of known data;
Acquisition means for acquiring item names and data values of new data;
Processing means for obtaining candidates of items of the known data corresponding to the new data, and outputting item names of the candidates;
Equipped with
The processing means
For the plurality of items of the known data, the similarity of features of data values with the new data is determined;
If there is an item with high determination accuracy by the feature of the data value among the high-order predetermined number of items with high similarity of the data value, the item name of the item with high determination accuracy by the feature of the data value is Output along with the ranking according to the feature similarity of data value,
When there is no item with high determination accuracy due to the feature of the data value among the high-order predetermined items having high similarity of data values, the item name of the high-order predetermined number of items and the item name of the new data And the item names of the upper predetermined number of items are outputted together with the ranking according to the similarity of the item names,
Data processing apparatus characterized by the above. The processing means determines the degree of similarity of the features of the data value using a learning device previously learned using data whose item name is known.
The data processing apparatus according to claim 1. It further comprises an input unit for receiving an input of the item of the known data corresponding to the new data,
Re-learn the learner using the input to the input means,
The data processing device according to claim 2. The processing means determines the similarity of the item name based on the edit distance of the item name.
The data processing apparatus according to any one of claims 1 to 3. The processing means obtains, as the upper predetermined number of items, items having a similarity of the data value equal to or more than a predetermined threshold value.
The data processor according to any one of claims 1 to 4. The processing means outputs that there is no similar item when there is no item whose similarity of the data value is equal to or more than the predetermined threshold value.
The data processing device according to claim 5. The processing device outputs the item name of the candidate in a user-selectable manner.
The data processing device according to any one of claims 1 to 6. The characteristic of the data value is one of the maximum value, the minimum value, the average value, the variance, or the maximum value, the minimum value, the average value, or the variance within the predetermined period of the data value within the predetermined period. Required based on multiples,
The data processing device according to any one of claims 1 to 7. The known data and the new data are data related to a vehicle,
The predetermined period is one trip period,
The data processing apparatus according to claim 8. A data processing method performed by a data processing apparatus that associates an item with new data whose item specification is unknown and known data whose item specification is known,
The data processor
Pre-storing information about discrimination accuracy according to features of data values for a plurality of items of known data;
Obtaining item names and data values of new data;
Obtaining a candidate of the item of the known data corresponding to the new data, and outputting the item name of the candidate;
Run
In the processing step, the data processing device
For the plurality of items of the known data, the similarity of features of data values with the new data is determined;
If there is an item with high determination accuracy by the feature of the data value among the high-order predetermined number of items with high similarity of the data value, the item name of the item with high determination accuracy by the feature of the data value is Output along with the ranking according to the feature similarity of data value,
When there is no item with high determination accuracy due to the feature of the data value among the high-order predetermined items having high similarity of data values, the item name of the high-order predetermined number of items and the item name of the new data And the item names of the upper predetermined number of items are outputted together with the ranking according to the similarity of the item names,
A data processing method characterized in that. A program for causing a computer to execute the method according to claim 10.

Images