JP2014199504A - Customer specific data cleansing processing system and customer specific data cleansing processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a service providing system equipped with a data cleansing processing control unit which can systematically perform data cleansing processing even in the system where the number increases in the way that the number on a data provider side is N and the number on a data receiver side is M.SOLUTION: A device for a foundation entrepreneur has a data cleansing processing control unit (data cleansing processing server) which performs data cleansing processing on the basis of a data cleansing processing policy master including a processing policy with respect to a pattern of a wrong value of original data provided by a device for a data providing device.

Description

  The present invention relates to a customer-specific data cleansing processing system and a customer-specific data cleansing processing method.

  More specifically, various data provided by a plurality of data providers, for example, a plurality of service providers, etc. are collected, and for example, the result of executing the aggregation, analysis, data mining, etc. of the data is the data receiver side. The present invention relates to a customer-specific data cleansing processing system and a customer-specific data cleansing processing method suitable for application to a data providing system that can be provided to a customer.

  Along with the development of the Internet, various service providers that provide effective services based on the type of business and content of businesses, such as service providers that provide product information such as popular menus to the food and beverage industry There are an increasing number of service providers, retail service providers, and food producers who provide foods to service providers.

  For example, in recent years, a service platform serving as a basis for a specific business service for a service provider that provides a specific business service to a service receiving business such as a restaurant using the Internet or the Web. There is a service providing system that provides services and supports the business of the service receiving business with business services based on the service infrastructure.

  Such a service providing system includes a service platform operator device, a service provider operator device, a service receiving operator device, and a user terminal device.

  The service platform operator device is a system or device that provides a service provider device with a function (service) based on the service platform to be provided, for example, a SaaS (Software as a Service) type business system.

  The service provider device is a system or device that receives a business system (business SaaS) based on a service platform provided from a service platform operator device. Further, for example, it is a system or apparatus that customizes the business system according to the request of the service receiving business operator (or directly from the user) and provides it to the service receiving business operator.

  The service receiving provider device is a system or device that receives a service provided from a service provider. Further, for example, it is a device that provides user information provided by the user to the service provider device side.

The user terminal device is, for example, a terminal that inputs user information and provides it to the service receiving provider device side. In addition, the user terminal device may be a device that can automatically collect information on the mobile terminal by holding the user's own mobile terminal.
And the original user information which a service receiving provider and a service provision provider own contains useful information. If the service platform provider is provided with this useful original user information and can analyze the information, it can be expected to provide better business support.

  As background arts in this technical field, there are JP-A-2009-93569 (Patent Document 1), JP-A-9-153094 (Patent Document 2), and JP-A 2012-141847 (Patent Document 3).

  In Patent Document 1, when changing from the SaaS environment to the SI environment, in the business service that has started to be used in the SaaS mode, it is possible to reduce the occurrence of burial costs and opportunity loss. In the case of transition from the company) to the SI environment (city B), a service address list for recording business service addresses is provided in city B, and the addresses in the service address list are updated in accordance with the business service transition. In addition, when business data stored in Company A is also migrated, a data address list for recording business data addresses is provided in Company A and City B, and the addresses in both data address lists are updated as business data is migrated. To do. In these updates, a technique for changing an access method so that addresses are logically the same is disclosed.

  Patent Document 2 discloses a sales promotion effect analysis method for analyzing a sales promotion effect of a product from information related to the customer, information on the product referred to when the customer considered purchasing the product, and information on the product purchased by the customer. And an implementation system thereof.

  In Patent Document 3, a definition file that defines processing conditions used in each subsequent process is created, and a definition file creation unit that is stored in a storage area and a definition file that is read from the storage area are used before data cleansing processing or Data stored in the destination DB after loading using the check processing unit that performs the validity check processing of the data in the later source DB and outputs the check processing result, and the definition file read from the storage area A data processing system that displays a check processing result and a verification processing result on a display device is described. The verification processing unit includes a verification processing unit that verifies the validity and outputs a verification processing result.

JP 2009-93569 A Japanese Patent Laid-Open No. 9-153094 JP 2012-141847 A

  According to the service providing system described in Patent Document 1, the service infrastructure provider can support the service provider. Further, the service provider can support the business of the service enjoyer. And by these business support, the business of the whole system can be made efficient and the business efficiency can be improved.

  In addition, each service provider and service infrastructure provider that provides such a service usually holds (accumulates) and manages data related to each user, for example, data such as preference and behavior history.

  Therefore, this user data is analyzed, and useful information and original data based on the analysis results are provided to service users (stores, general companies, etc.) that receive services from service providers. If there is a system that can do this, it can be expected to improve operational efficiency.

  Here, the “user” in the present invention is an individual, group, or company that uses the system, and a specific user (one person, one group, one user that can identify one company) and an unspecified user (one user, one group, one company can be specified). Individuals, organizations, and companies that cannot or cannot be specified). A terminal or the like used by the user in this system is referred to as a “user side terminal”. Hereinafter, in the present invention, the term “individual” with respect to the user includes organizations and companies as the user.

  In addition, the “service enjoying operator” is an entity that receives the original business support service (first service) by this system, and the system component device provided on the service enjoying operator side is referred to as “service enjoying operator”. Say "business equipment". The meaning of the term “provided on the side” here does not mean that the term “provided in the service receiving service provider” is provided. The term “provided on the side” is used in the same manner hereinafter.

  A “service provider” is an entity that provides an original business support service (first service) using this system, and a system component device provided on the service provider side is referred to as a “service provider business”. Say the device.

  In addition, a “service infrastructure provider” is a company that bundles a plurality of service providers, and provides service infrastructure such as business and reservation functions (business services and reservation services) and / or service providers. It is a business entity provided to the enjoying business. Then, information is collected from time to time based on the contract from the service receiving provider device and / or the service providing provider device, and the information is analyzed, for example, and analyzed to the service receiving provider side and / or the service providing provider side. Provided as (second service). Here, the information to be provided need not be limited to analysis information, and for example, in response to a request from a data provider, non-analysis information with the original data as it is may be provided. A system component device provided on the service infrastructure provider side is referred to as a “service infrastructure provider apparatus”.

In addition, “data provider” and “data receiver” refer to a service provider and a service receiver. For example, the original user information Ia and Ib are transmitted as raw data as they are or as data to be provided to the service platform operator as anonymized data XIa and XIb satisfying k-anonymity, and the data is also transmitted to the service platform provider. It is received as analysis data or raw data analyzed on the side. A system component device provided on the data provider side is referred to as a “data provider unit”, and a system component unit provided on the data receiver side is referred to as a “data receiver unit”.
The data provider side is a “data provider” and the data receiver side is a “data receiver”. However, the data provider is not limited to a “provider” but may be a “user” individual. Therefore, in the embodiments described later, there may be simply referred to as “data provider” and “data receiver”.

  In addition, since a service infrastructure provider is a company that bundles multiple service providers, the service infrastructure provider integrates or integrates various data stored in multiple service providers. If it can be analyzed, higher value-added analysis becomes possible, and as a result, higher value-added analysis results for users including service providers and / or service users Can be provided.

  In the present situation where an IT environment capable of analyzing a large amount of data is being prepared, it is technically possible to perform the above-described high added value analysis from a large amount of data.

  “High-value-added analysis” means personalization (sending information by e-mail, etc.) according to personalization (customers and prospective users' preferences and attributes), and recommendations for simultaneous sales (recommendation: user preferences). This is an analysis for providing basic data that can be realized by analyzing and selecting information that is likely to be interesting for each user.

  At the service provider side and the service infrastructure provider side, various data are collected from the service provider side. For example, when collecting and analyzing the collected data, the data is resistant to aggregation and analysis (analysis). In order to achieve this, it is necessary to perform a data cleansing process. The reason is that a part of the data may become an illegal value due to an input error of the data input person or a defect in data processing.

When performing this data cleansing process, the following two methods are generally used.
First, data (original data) provided from the service provider (hereinafter referred to as the data provider) device 10 side is subjected to data cleansing processing on the data provider device side, and the data cleansing processing is performed. This is a method of providing already-completed data (data after data cleansing processing) to the device 20 for data beneficiaries (hereinafter referred to as data beneficiaries).
In the case of this method, it is required to decide between the two companies whether the data cleansing processing policy conforms to the data provider side policy or the data receiver side policy.

The second is a method of performing data cleansing processing of the original data on the data receiver device side that receives the original data provided from the data provider device 10 side.
In this method, (1) it is difficult for the data recipient device 20 side to determine whether the original data is a normal value or an illegal value. In addition, it is also necessary to determine whether to follow the data cleansing process policy on the data provider apparatus 10 side or the data cleansing process policy on the data receiver apparatus 20 side. (1) and (2) Is required to be agreed between the data provider side and the data receiver side.

  By the way, it is the data provider who is the owner of the original data that knows whether the value of the original data is a normal value or an incorrect value (value to be subjected to data cleansing processing). is there.

  On the other hand, when the data provider side provides the original data to the data receiver side, the data receiver side uses the provided data, and the data cleansing process is the data receiver side that uses the data. It is preferable to follow the data cleansing processing policy.

  As described above, the method of performing the data cleansing process on the data provider side or the data provider side can be easily realized by an agreement between the two companies when the data provider side and the data receiver side are one-to-one. is there.

  However, a system targeting a plurality of data providers and a plurality of data receivers having different data cleansing processing policies, for example, the number of data providers is N, and the number of data receivers is M. It is not considered for application to a system with an increasing number.

  That is, in the case of application to the system, for example, in a system in which the number of data providers is N and the number of data receivers is M, the system configuration becomes complicated, There is a problem like this.

  When processing data cleansing values on the N data provider side, the data cleansing process is performed according to each data cleansing policy on the M data receiver side. The data cleansing process must be performed M times for each person.

  Conversely, when the data cleansing process is performed on the data receiver side where the M person is present, the original data is acquired from the data provider side where the N person is present, and at the same time the data cleansing process is performed from the data provider side where the N person is present (illegal Value) is set, the data receiver side with M persons must perform data cleansing processing N times.

  That is, when there are N persons on the data provider side and M persons on the data receiver side, there is a problem that an efficient data cleansing apparatus is necessary.

v Further, Patent Document 1 has a data cleansing processing unit that performs data cleansing processing on the migration source DB based on an instruction from the client, and the user can determine the validity of the data of the migration destination system. A simple data migration system is described. However, it is not taken into consideration that data cleansing processing is systematically performed in a system that receives and collects multiple data from multiple data providers and provides data cleansing processing to multiple data recipients. . The same applies to Patent Documents 2 and 3.

  Therefore, the present invention provides a data cleansing process for each customer that can perform the data cleansing process systematically even in a system in which the number of data providers is N and the number of data beneficiaries is M. A system and customer-specific data cleansing processing method are provided.

  In order to solve the above problems, the present invention is based on the data cleansing process policy for the illegal type of the original data provided from the data cleansing process policy master and the master data provider apparatus on the infrastructure provider apparatus side. The data cleansing processing unit (data cleansing processing server) for each customer that performs data cleansing processing is provided, and the data cleansing processing policy can be set from the data receiver device.

For example, the customer-specific data cleansing processing system of the present invention is:
A data providing system having at least one service infrastructure company device, a plurality of data provider devices, and a plurality of data receiver devices,
The service platform operator device is:
Input / output means, storage means, control means,
The input / output means includes
Send and receive data between the data provider device and the data beneficiary device,
The storage means includes a data cleansing processing policy master that performs data cleansing processing on data provided from the data provider device.
The control means has a function of performing a data cleansing process of the data with a data cleansing process policy of the data cleansing process policy master,
A receiving unit that receives data provided from the data provider device and field type attribute information of the data;
A data cleansing process policy receiving unit for receiving the data cleansing process policy;
Data cleansing processing means for performing data cleansing processing on data provided from the data provider device according to the data cleansing processing policy;
Means for transmitting the data cleansing data after the data cleansing processing by the data cleansing processing means to the data receiver device;
It is characterized by having.

The control means further includes
Default data cleansing processing policy of the data receiver device, data receiver data cleansing processing policy for data provided from the data provider device, default data cleansing processing policy of the data provider device, and the data provider device Means for receiving a data provider data cleansing processing policy for data provided from and reflecting the data on the data cleansing processing policy master;
It is characterized by having.

The control means further includes
A field type attribute master, an illegal value master, a processing method master, and a means for creating a data cleansing processing policy for the service infrastructure provider device from the field type attribute master, the illegal value master, and the processing method master. The means for reflecting in the data cleansing process policy master reflects the data cleansing process policy of the service infrastructure provider apparatus in the data cleansing process policy master.

The data cleansing process policy master includes an original data field name of data provided from the data provider device, a processing method corresponding to each illegal value that may occur in the field type attribute of the field name,
The illegal values are “blank (NULL)”, “full-width half-width inconsistency”, “illegal character”, “abnormal value”, “hiragana character inconsistency”, “kana character inconsistency”, “English letter inconsistency”, “ 1 of "Numeric Ununiformity", "Unidentified Symbol", "Unidentified Kanji Character", "Denormalized Phone Number", "Denormalized Postal Code", "Denormalized Address", "Denormalized Amount" Two or more
The type attribute of the field is one or more of “primary key”, “name”, “zip code”, “address”, “quantity”, “amount”, “date”,
The processing methods are “do nothing”, “delete all records”, “fit to data provider format”, “fit to data provider normalization”, “blank”, “replace with average value”, One or more of the above.

The customer-specific data cleansing processing system of the present invention is
Data cleansing processing for data shared between multiple providers, including a base provider device that receives original data provided from the provider's data provider device and provides the original data to the data receiver device In customer-specific data cleansing processing system,
The infrastructure provider device is
A data cleansing processing control device that performs data cleansing processing on original data from the data provider device, an illegal value master that defines an illegal value that can occur in each field of the original data, and an illegal value that can occur in each field of the original data A data cleansing processing policy master having a processing method for
The data provider device comprises:
When the value of the data is an illegal value with respect to the original data, there is means for specifying which item of the illegal value master the illegal value corresponds to,
The data receiver device is:
Means for setting a processing policy for illegal values for field type attribute types prepared comprehensively for the original data;
The data cleansing process control device includes:
Reading means for reading the processing policy set on the data receiver device side from the data cleansing processing policy master for the field type attribute information of the original data set on the data provider device side, and the reading Control means for performing data cleansing processing on the original data in accordance with a data cleansing processing policy.

The customer-specific data cleansing processing method of the present invention includes:
A data cleansing processing method for each customer in a data providing system having at least one service infrastructure provider device, a plurality of data provider devices, and a plurality of data receiver devices,
The service platform operator device is:
Input / output means, storage means, control means,
The input / output means includes
Send and receive data between the data provider device and the data beneficiary device,
The storage means includes a data cleansing processing policy master that performs data cleansing processing on data provided from the data provider device.
The control means has a function of performing a data cleansing process of the data with a data cleansing process policy of the data cleansing process policy master,
Receiving data provided from the data provider device and field type attribute information of the data;
Receiving a data cleansing process policy of the data cleansing process policy master;
Performing data cleansing processing on data provided from the data provider device according to the data cleansing processing policy;
Transmitting data cleansing-processed data after data cleansing processing by the data cleansing processing means to the data receiver device;
It is characterized by having.

The control means further includes
Default data cleansing processing policy of the data receiver device, data receiver data cleansing processing policy for data provided from the data provider device, default data cleansing processing policy of the data provider device, and the data provider device Receiving a data provider data cleansing process policy for data provided from and reflecting it in the data cleansing process policy master;
It is characterized by having.

The control means further includes
A field type attribute master, an illegal value master, a processing method master, and a step of creating a data cleansing processing policy for the service infrastructure provider device from the field type attribute master, the illegal value master, and the processing method master. The means for reflecting in the data cleansing process policy master has a step of reflecting the data cleansing process policy of the service infrastructure provider apparatus in the data cleansing process policy master.

The data cleansing process policy master includes: an original data field name of data provided from the data provider device; a field type attribute corresponding to each illegal value that may occur in the field type attribute of the field name; and a process for the incorrect value Including methods,
The illegal values are “blank (NULL)”, “full-width half-width inconsistency”, “illegal character”, “abnormal value”, “hiragana character inconsistency”, “kana character inconsistency”, “English letter inconsistency”, “ 1 of "Numeric Ununiformity", "Unidentified Symbol", "Unidentified Kanji Character", "Denormalized Phone Number", "Denormalized Postal Code", "Denormalized Address", "Denormalized Amount" Two or more
The type attribute of the field is one or more of “primary key”, “name”, “zip code”, “address”, “quantity”, “amount”, “date”,
The processing methods are “do nothing”, “delete all records”, “fit to data provider format”, “fit to data provider normalization”, “blank”, “replace with average value”, One or more of the above.

  According to the present invention, even if the number of data providers is N, and the number of data beneficiaries is M, such as a system in which the number increases, the data cleansing process for each customer can perform the data cleansing process systematically. A system and customer-specific data cleansing processing method can be provided.

  In addition, by consolidating the data cleansing process to the device on the base provider side and holding the “data cleansing process policy master”, the data provider side once designates the “illegal value” of the “original data”, and The data receiver side only needs to set a “data cleansing processing policy” for the “illegal value” once. It can also be changed or added to the data cleansing processing policy designated on the data provider device or the infrastructure provider device side.

  Problems, configurations, and effects other than those described above will become apparent from the following description of embodiments.

It is a figure which shows the whole service provision system which employ | adopted the data cleansing processing system classified by customer of this invention. It is a block diagram which shows the structure of the server for service infrastructure providers of FIG. 1, and the relationship between the server for data providers, and the server for data beneficiaries. It is a block diagram which shows the structural example of the arithmetic unit (control means) of the server for service infrastructure providers of FIG. It is a figure which shows an example of the data (original data) provided from the server for data providers. It is an example of the master table which shows the original data prepared in the server for service infrastructure providers of the data (Ia) provided from the server for data providers. It is an example of the master table which shows the information of the data provider (Company A) in the server for service infrastructure providers. It is an example of the master table which shows the original data field type | mold attribute information prepared in the server for service infrastructure providers of the data provided from the server for data providers. It is an example of the master table which shows the field type attribute of the data provided from the server for data providers. It is an example of the master table which shows the illegal value prepared comprehensively in the server for service infrastructure providers. It is an example of the master table which shows the processing method prepared comprehensively in the server for service infrastructure providers. It is a figure which shows the example of the data cleansing process policy setting screen which sets a data cleansing process policy in the server for service beneficiaries. It is a figure which shows the example of the data provider (A company) data (Ia) default data cleansing process policy master. It is a figure which shows the example of the data beneficiary (P company) master in the server for service infrastructure providers. It is a figure which shows the example of the data cleansing process policy master in the server for service infrastructure providers. The data (Ia and Ib) provided from the data provider (Company A and Company B) is subjected to data cleansing processing on the service infrastructure provider server side, and the data cleansing processing (data cleansing processing) It is a figure which shows the flowchart which transmits back data) to the server for data beneficiaries. A state in which data (Ia), default data cleansing processing policy (CaIa) of the data, and data provider default data cleansing processing policy (Ca) are transmitted from the data provider (Company A) side to the base provider side. It is a figure shown typically. From the data beneficiary (Company P) side to the base provider side, the data beneficiary (P Company) side default data cleansing policy (Cp), the data provider side data (Ia) data beneficiary (P Co.) data cleansing process policy (CpIa), and the base provider default data cleansing process policy Cg and the data provider data cleansing process policy (CaIa) and the data provider default data cleansing process policy (Ca) of the data It is a figure which shows typically a mode that a data cleansing process is performed using at least one. It is a figure which shows the flowchart for displaying and producing | generating the data cleansing process policy setting screen in the server for data beneficiaries. It is a figure which shows the process flowchart in FIG. It is a figure which shows the process flowchart in FIG. It is a figure which shows the process flowchart regarding the data in FIG. 20, a record, and a field. It is a figure which shows one process flowchart according to a processing method. It is a figure which shows the other process flowchart according to a processing method. It is a figure which shows the other process flowchart according to a processing method. It is a figure which shows the other process flowchart according to a processing method. It is a figure which shows the other process flowchart according to a processing method.

The embodiment will be described below with reference to the drawings. First, an outline of a service providing system that supports a business of a service receiving business will be described.
The service providing system is a service infrastructure provider system or apparatus (hereinafter referred to as a service infrastructure provider apparatus) that provides an application service or SI, that is, a service (first service S1) such as a reservation function or business SaaS to the service provider. And a service provider system or device (hereinafter referred to as a service provider device) that customizes the first service and provides it to the service receiver in accordance with the request of the service receiver (such as a restaurant). And).

  A service receiving business operator (such as a restaurant) provides a user (consumer) with food and drinks and merchandise, and receives user information (original information) from the user.

  In addition to SaaS, the service infrastructure provider device may be a device that provides a PaaS (Platform as a Service), an ASP (Application Service Provider) service, or the like that lends a platform including an OS, for example.

Hereinafter, the restaurant industry (restaurants), service providers, and service infrastructure operators will be mainly described.
First, an outline of the service providing system will be described. The service platform operator apparatus of the service providing system has an application server (hereinafter referred to as a service platform provider server) that provides a reservation management service (application service) or the like to the service provider apparatus.

  The service provider device customizes this reservation management service according to the request of the service receiving provider, and accepts and manages the reservation information of the user (consumer) from the service enjoying provider (hereinafter referred to as the service provider). Server). In other words, this is a mechanism for supporting the business of the service receiving business operator in cooperation with the service infrastructure business operator device and the service providing business operator device.

A system with such a mechanism generally corresponds to an individual project basis for each service receiving company.
Therefore, the service platform operator device provides the first service S1 to the service provider device, and the service provider device provides the customized first service (service start) to the service receiving operator. ) And also manages user information (original information) Ia, Ib, etc. based on the provided first service.

  An example of this service providing method is SaaS. SaaS means that the function (first service) provided by the information system (software) is used by the service provider and service receiving operator via the network without constructing or introducing a new system. It is an ICT utilization environment that makes possible.

  In general, SaaS has an advantage that required services can be used quickly without the need for special system construction. In SaaS, it is not necessary to prepare a special environment in the company, so that the operation start time of the function (service) can be advanced and the cost can be reduced. In addition, although the Internet and WAN are used, the operation is almost the same as that using an in-house system due to technological innovation of the network and browser.

  In addition, service providers and service users can expect to reduce costs and expand sales by using SaaS, SI, and operation / maintenance provided by service infrastructure providers.

  The present invention provides a result (data) of data mining data from such data provider's devices (service provider devices A, B, service receiving provider devices a1, a2, b1, b2, etc.) Data cleansing processing is performed for each data receiver, and the data can be provided to the device side of the data receiver as post-data cleansing data.

  Embodiments of the present invention will be described below with reference to the drawings. In the present embodiment, even if the data cleansing process is not complete, it can be accepted, for example, it is possible to efficiently analyze by integrating or integrating the plurality of received data, and the analysis data or original data Data cleansing processing is performed on the (original data) according to the settings of the data receiver device, and is transmitted in response to a distribution request from the data receiver side.

In this example,
(1) When the number of data providers is N and the number of data beneficiaries is M, data is received through the infrastructure provider.
(2) A “data cleansing processing means” that performs data cleansing processing on the original data of the data provider is mounted on the server of the base provider. In other words, the system of the present invention is implemented mainly by the base business operator.
(3) Also, a “data cleansing process policy master” (FIG. 14) is provided on the server of the base business operator.
(4) “Data Cleansing Process Policy Master” (FIG. 14) is set as a default value by the value of “Default Data Cleansing Process Master” (FIG. 12) set by the data provider. (5) The data provider For each field of “original data” (FIG. 4), the “attribute” of the field is registered in “original data field type attribute information master” (FIG. 7). “Original data” is managed by “original data source master” (FIG. 2). (6) The data provider sets “field type ID” in “original data field type attribute information master” (FIG. 7) to “field”. Select from “Type Attribute Master” (FIG. 8).
(7) The “field type attribute master” (FIG. 8) prepares field types comprehensively. If there is no type corresponding to the “field type attribute master”, the data provider adds the field type to the “field type attribute master”.
(8) “Illegal value” is managed by “Illegal value master (FIG. 5)”. The “illegal value master (FIG. 5)” prepares illegal values comprehensively. If there is no illegal value corresponding to the “illegal value master (FIG. 5)”, the data provider adds an illegal value to the “illegal value master”.
(9) The “processing method” of invalid values is managed by the “processing method master” (FIG. 10). The “processing method master” prepares processing methods comprehensively. If there is no processing method corresponding to the “processing method master”, the data provider adds the processing method to the “processing method master” (FIG. 10).
(10) The data provider designates the “processing method” for the “field type attribute” on the “data cleansing process policy setting screen” (FIG. 11), and each ID of the “default data cleansing process master” (FIG. 12) ( Value). The data provider ID is “0” as a default value.
(11) As shown in FIG. 11, the “data cleansing process policy setting screen” (FIG. 11) includes “illegal value” (“blank”, “double-byte ununiformity”, “illegal character”, “non-format”, "Process ID" is set as the value of the field type attribute ("Primary key", "Name", "Zip code", "Address", "Quantity", "Amount", ...) .
(12) The data receiver can set the “data cleansing process policy” on the “data cleansing process policy setting screen” (FIG. 11). The set “data cleansing process policy” is reflected in the “data cleansing process policy master” (FIG. 14). Here, the data receiver is managed by the “data receiver master” (FIG. 13).
(13) As the value of the “data cleansing process policy master” (FIG. 14), the data receiver once sets the process policy for the “illegal value” of the data, that is, the “processing method”. In this way, by setting the processing policy for the “illegal value” of the data provider's original data, as a result, the data receiver can set the processing policy once for the original data of the N data provider. Data cleansing processing policy can be set.
(14) The data provider and the data receiver can set a data cleansing processing policy for specific data of the data provider. When a data cleansing processing policy is set for specific data, it is used in preference to the default data cleansing processing policy.
(15) The data cleansing processing server of the infrastructure provider sets the “processing method” designated by the data receiver to “data” for the “illegal value” designated on the data provider side (the illegal value ID in FIG. 12). Read from "cleansing process policy master" and execute the processing program.
(16) The platform provider has a program on the server of the platform provider to realize processing corresponding to the “illegal value master” (FIG. 9) and the “processing method master” (FIG. 10).
Hereinafter, the detailed Example is demonstrated.

  FIG. 1 is a diagram showing an overall configuration of a service providing system in which a customer-specific data cleansing processing system of the present invention is applied to a service platform operator device.

  The service providing system includes a data providing operator device 12 of a data providing operator and a service infrastructure operator device 3 of a service infrastructure operator.

  The data provider device 12 includes a plurality of service provider devices 2 including a plurality of service provider devices (A, B...) And a plurality of service receiving provider devices (a, b...). Including a service receiving service provider device 1.

The data provider company 12 includes a data provider server 121.
Each service receiving company device 1 is, for example, a device on the side of a plurality of stores (a1, a2) or a plurality of companies (b1, b2), and includes a server 11 for service receiving companies. And each server 11 for service enjoyment companies outputs original user information, ie, original data (Ia and Ib), to the service provider company device 2.

  The original user information Ia and Ib is sent from the user side corresponding to the first service S1 (business / reservation function, etc.) provided from the service infrastructure company apparatus 3 via the network 4. .

  Each service provider device 2 includes a service provider server 21. The service provider server 21 receives each original data I (Ia, Ib) and anonymizes the original data directly or by using a well-known anonymization technology to the extent that the source cannot be specified. Data (including provided data = original data I (Ia), field type attribute information F (FIa) of the data, and data cleansing processing policy C (Ca)) is transmitted to the service infrastructure provider apparatus 3 side via the network 4 To do.

  The service infrastructure provider device 3 includes a service infrastructure provider server 31. The service infrastructure provider server 31 receives data I (Ia), field type attribute information F (FIa), and data cleansing processing policy C (Ca) provided from the server 121 for the data provider (company A). .

In addition, the service infrastructure provider device 3 receives a data cleansing processing policy C (Cp) from a data beneficiary (P company) server (see 1216 in FIG. 2).
And according to the said data cleansing process policy, it has a function which performs the data cleansing process of the original data provided from the server for data providers, and outputs it as the data S after a data cleansing process.

  FIG. 2 is a block diagram illustrating the configuration of the service platform provider server 31, the data provider provider server 121, and the data beneficiary server 1216.

In this example, the service provider server 21 of FIG. 1 is used as the data provider server 121, the service receiver server 11 is used as the data receiver server 1216, and the service provider business is started from the data provider server 121. The data provided to the user server 31 will be described on the premise of original data (original data) I (Ia, Ib,...). Further, description will be made on the assumption that data cleansing processing is performed on data from the data provider server 121 and the data S after the data cleansing processing is transmitted to the data receiver server 1216.
Here, a desired analysis may be performed on the original data as necessary. Further, this analysis may be performed on the data receiver side based on the data after the data cleansing process, but since it is not the essence of the present invention, the description thereof is omitted.

  The server 121 for the data provider provides the service infrastructure provider server 31 with the original data I, the information F indicating the field type attribute of the data, the data cleansing processing policy C, and the like. You may use as a server for data receivers who receive data S after data cleansing processing from server 31 for users. The data receiver server 1216 provides the data cleansing process policy Cp on the data receiver side when receiving the data S after the data cleansing process. However, the data receiver server 1216 provides data to the service infrastructure provider server. You may utilize as a server for data providers to provide.

  In order for the data receiver server 1216 to receive the data S after the data cleansing process from the service infrastructure provider server 31, the service infrastructure provider server 31 is requested to that effect (data reception request signal). At the same time, the data cleansing processing policy C (Cp) of the data is notified to the service infrastructure provider server 31 side. The data cleansing processing policy C (Cp) may be set and registered in advance on the service infrastructure provider server 31 side.

  The service infrastructure provider server 31 includes an input / output device (input / output interface) 311, a storage device (storage means) 312, and an arithmetic device (calculation means) 313. These are connected by LAN.

  The storage device 312 includes a data provider / data receiver management master DB / data and field type attribute information management DB 3121, a data provider / data receiver / data cleansing process policy information management DB 3122, a field type attribute master DB 3123, and an illegal value master. A DB 3124 and a processing method master DB 3125;

  The data provider and data receiver management master DB / data and field type attribute information management DB 3121 manages data and field type attribute information of the data provider and data receiver, and from the data provider server 121. The provided data and the field type attribute information of the data are stored in association with each other and managed.

  Data provider / data beneficiary / data cleansing process policy information management DB 3122 is a data cleansing process policy (“data provider ID / data beneficiary ID”, “original data” set on the data provider or data beneficiary side). Source ID ”,“ Original Data Field ID ”,“ Original Data Field Name ”,“ Invalid Value ”,“ Processing Method ID ”, etc.) are stored.

  In other words, the data cleansing process policy master that performs the data cleansing process on the provided data I from the data provider server 121 is managed by the service infrastructure provider server 31.

  The field type attribute master DB 3123 stores “field type attribute ID” and “attribute”, that is, “data number”, “purchase date”, “store name”, “customer name” “product name”, “amount”, “number of purchases”. "," Date ", etc. are memorized.

  The illegal value master DB 3124 records “invalid value ID”, “type of illegal value”, that is, “blank”, “unmatched full-width / half-width”, “illegal character”, “non-formatted”, and the like.

The processing method master DB 3125 includes “processing ID”, “processing”, that is, “do nothing”, “delete all records”, “blank”, “replace with average value”, “data beneficiary data cleansing processing policy” According to the data receiver default data cleansing policy, according to the data provider data cleansing policy, according to the data provider default data cleansing policy, according to the data provider default data cleansing policy, Memorize etc.
The contents of these DBs will be described later.

The arithmetic device 313 includes control means 3131 that performs data cleansing processing on the original data I.
The control means 3131 includes a data provider and data receiver recognition unit 31131, a data and field type attribute information reception unit 31121, a data cleansing processing unit 31313, a data cleansing-processed data distribution unit 31314, a data cleansing processing policy reception unit 31315, data A writing / reading unit 31316.

  The data provider and data receiver recognition unit 31131 recognizes the data provider and the data receiver.

  The data and field type attribute information receiving unit 31212 receives data I and field type attribute information F from the data provider server 121.

  The data cleansing process policy receiving unit 31315 receives the data cleansing process policy information C (Ca, Cp) of the original data provided from the data provider server 121 and the data beneficiary server 1216.

  The data cleansing processing unit 31313 performs data cleansing processing on the data provided from the data provider server 121 according to the data cleansing processing policy of the data cleansing processing policy information management DB 3122.

This data cleansing process is performed so that, for example, the service provider server 31 can analyze the data of the data provider server 121.
As a result, the data receiver server 1216 can receive the data after the data cleansing process from the service infrastructure provider server 31 only by setting the data cleansing process policy that the data cleansing process is desired.

  The data writing / reading unit 31316 writes the data I (Ia, Ib,...) And the data cleansing processing policy C (Ca, Cp) of each data to the DB 3121 and reads from the DB 3121. To do.

  The data distribution unit 31315 after the data cleansing process transmits data subjected to the data cleansing process by the data cleansing process unit 31313 to the data receiver server 1216.

  FIG. 3 is a functional block diagram showing an example of the service infrastructure provider server 31. In the figure, the server 31 receives provided data I and field type attribute information F of the data at the data and field type attribute information receiving unit 31212 (see FIG. 2), and associates the data and field type attribute with the data. It is stored in the information management DB 3121.

  The provided data I is stored in the data and field type attribute information management DB 3121 in the data format provided from the data provider server 121, and then the data cleansing processing unit 31313 performs data cleansing processing on the data.

  In addition, the data cleansing processing unit 31313 may store the data cleansing-processed data S after the data cleansing processing in the data and field type attribute information management DB 3121.

  The field type attribute information of the data is stored in the data and field type attribute information management DB 3121 in association with the provided data I.

The table contents of each DB will be described below.
FIG. 4 is a diagram illustrating an example of data (original data I) provided from the data provider server, and the original data of the data provider name “Kakaka Shoten Co., Ltd.” in the original data master of FIG. It is a table indicating the name “sales journal table”.

  In the figure, the original data includes the original data primary key (“1”, “2”, “3”, “4”,...), The name (“○ Da”, “Ogi”). , “○ Tate * ro”, “J Ferry”,...), Postal code (“111-1111”, “111112”, “111122”, “111-2222”,...), Address (“Sapporo Asahi Ward, "1-1, Minatomachi, Hachinohe City," "Akita City, Akita Prefecture," "4-chome Kawabe, Morioka City, ...", product name ("xxxx", "yyyy", "zzz", " Abcde ”,..., And the amount of money (“ 1000 ”,“ 20,000 ”,“ 30,000 ”,“ 40000 ”,...), Etc.

  FIG. 5 is a table showing the original data master prepared in the service infrastructure provider server for the data (original data Ia) provided from the data provider server.

  In the drawing, the original data source master includes an original data ID (“1”, “2”, “3”, “4”,...), An original data name (“sales journal table”, “reserved journal table”). , “Sales table by time zone”, “Reservation journal table”,...), Data provider ID (“1”, “2”, “3”, “3”,...), Data provider name (“Kakaka Shoten Co., Ltd.”, “Kikiki Hotel Co., Ltd.”, “Keke Hair Salon Co., Ltd.”, “Keke Hair Salon Co., Ltd.”, etc.) It is out.

  FIG. 6 is a table showing a data provider (Company A) master in the service infrastructure provider server.

  In this figure, the data provider master has a data provider ID (“1”, “2”, “3”,..., Data provider name (“Sasa Co., Ltd.” Each information 601 to 602 such as “noodle making”, “Sususu Corporation”,.

  FIG. 7 is a table showing an original data field type attribute master prepared in the service infrastructure provider server for data provided from the data provider server.

  In the figure, the original data field type attribute master includes a field ID (“1”, “2”, “3”, “4”, “5”, “6”, “7”,...), Original data. ID (“1”, “1”, “1”, “1”, “1”, “1”, “1”,...) Field name (“data number”, “purchase date”, “store name” ”,“ Customer name ”,“ product name ”,“ number of purchases ”,“ money amount ”,...), Field type attribute IDs (“ 1 ”,“ 8 ”,“ 3 ”,“ 3 ”,“ 3 ”) , “7”, “6”,...) And the like.

  FIG. 8 is a table showing a field type attribute master of data provided from the data provider server.

  In the figure, the field type attribute master includes field type attribute IDs (“1”, “2”, “3”, “4”, “5”, “6”, “7”, “8”,... ), Attributes ("data number", "purchase date", "store name", "customer name", "product name", "money", "number of purchases", "date", etc.) 801-802 are included.

9 is a table showing an illegal value master that is comprehensively prepared in the server for service infrastructure providers.
In the figure, the illegal value master includes an illegal value ID (“1”, “2”, “3”, “4”,...), An illegal value type (“blank”, “unmatched one-byte / half-width”, Each information 901 to 902 such as “illegal characters”, “denormalized”,.

  10 is a table showing a processing method master that is comprehensively prepared in the server for service infrastructure providers.

  In this figure, the processing method master has processing method IDs (“1”, “2”, “3”, “4”, “5”, “6”, “7”, “8”, “9”,. ..), processing ("do nothing", "delete all records", "blank", "replace with average value", "follow data beneficiary data cleansing process policy ⇒ Company P data cleansing process policy CpIa" , “According to data receiver default data cleansing process policy ⇒ P company default data cleansing process policy Cp”, “According to data provider data cleansing process policy ⇒ A company data cleansing process policy CaIa for company A original data Ia”, “ According to the infrastructure provider data cleansing policy ⇒ Infrastructure provider default data cleansing policy Cq ”,“ Default default of data provider ⇒A company default data cleansing process policy Ca "according to the data cleansing process policy, ...) includes the information 1001-1002 and the like.

  FIG. 11 is a diagram showing a data cleansing process policy setting screen for setting a data cleansing process policy on the service beneficiary server.

In the figure, the data cleansing process policy setting screen has a “data number” 7031, “purchase date” 7032, “store name” 7033, “customer name” 7034, “field name” 703 (see FIG. 7) in the horizontal direction. “Product Name” 7035, “Purchase Date” 7036, “Amount” 7037, and “Primary Key” 7041 ′, “Date” 7042 ′, “Name” 7043 ′, “Name” 7044 ′ of the field type attribute of the field name, “Name” 7045 ′, “Quantity” 7046 ′, “Price” 7047 ′, and the like are displayed.
In the vertical direction, “blank”, “full-width half-width inconsistency”, “illegal character”, “non-format”, etc., are displayed.
Then, in the “field type attribute” corresponding to “illegal value”, the processing ID (1 to 9,...) Of the processing method <see FIG.

  FIG. 12 is a diagram illustrating an example of a data provider (company A) data (Ia) default data cleansing process policy master.

  In the figure, the default data cleansing process policy master includes a data provider ID (“1”, “1”, “1”, “1”,...), An original data source ID (“1”, “1”, "1", "1", ...), original data field ID ("1" ..., "2" ..., "3" ..., "4" ..., ...) , Original data field name ("original data key" ..., "name" ..., "zip code" ..., "address", ...), illegal value ID ("1", "2") , “3”, “4”, “1”, “2”, “3”, “4”, “1”, “2”, “3”, “4”...), Processing method ID (“ “2”, “2”, “2”, “2”, “1”, “1”, “3”, “3”, “1”, “5”, “3”, “3”, “1” , “5”, “5”, “3”), etc. Which comprise.

  FIG. 13 is a diagram illustrating an example of a data beneficiary (Company P) master in the service infrastructure provider server.

  In this figure, the data beneficiary master is the data beneficiary ID (“1”, “2”, “3”,..., The data beneficiary name (“Sasa Manufacturing Co., Ltd.” Each information 1301 to 1302 such as “noodle making”, “Sususu Corporation”,.

  14 is a diagram showing an example of a data cleansing processing policy master in the service infrastructure provider server.

  In the figure, the data cleansing processing policy master includes a data receiver ID (“1”, “1”, “1”, “1”,...), An original data source ID (“1”, “1”, “ 1), “1”,...), Original data field ID (“1”..., “2”..., “3”..., “4”...), Original data field name ("original data key" ..., "name" ..., "zip code" ..., "address", ...), illegal value ID ("1", "2", “3”, “4”, “1”, “2”, “3”, “4”, “1”, “2”, “3”, “4”...), Processing method ID (“2”) ”,“ 2 ”,“ 2 ”,“ 2 ”,“ 1 ”,“ 1 ”,“ 3 ”,“ 3 ”,“ 1 ”,“ 5 ”,“ 3 ”,“ 3 ”,“ 1 ”, “5”, “5”, “3”), etc. .

Next, the processing procedure of the customer-specific data cleansing processing method will be described.
FIG. 15 is a diagram illustrating a processing flow of the customer-specific data cleansing processing method of the present invention. In this example, the case where the number of data provider servers 121 is two is shown, and the case where the number of data receiver servers 1216 is one is shown. In addition, it is assumed that the default data cleansing processing policy of each person has been set, and the field type attribute master, the invalid value master, and the processing method master are shared by the data provider server and the data receiver server. In the figure, the following processing is executed.
(1) The data provider (Company A) creates the field type attribute information FIa for data Ia and the data cleansing processing policy CaIa for company I for data Ia based on the above-described three masters.
(2) Then, the data provider (Company A) transmits the data Ia & field type attribute information FIa and the data A data cleansing processing policy CaIa for the data Ia to the base provider.
(3) The data provider (Company B) also performs the same processing as (2) and (3) above.
(4) The data beneficiary (Company P) requests the field provider from the field type attribute information FIa of the data Ia and the company A data cleansing processing policy CaIa for the data Ia.
(5) In response to this request, the base business operator responds to the data beneficiary (Company P) with the field type attribute information FIa of the data provider <Company A> data Ia, and the data cleansing data for Company A for data Ia. The processing policy CaIa is transmitted.
(6) The data beneficiary (Company P) receives FIa and CaIa, creates a data cleansing processing policy CpIa for data Ia with reference to these, and transmits it to the base business operator.
(7) Moreover, the data beneficiary (Company P) requests the data after the data cleansing process from the base company.
(8) The infrastructure provider executes the data cleansing process based on this request, and transmits the data after the data cleansing process to the service user (Company P).
(9) The service receiving company (Company P) receives the data after the data cleansing process.

  In step S1211, the data provider (company A) server 121 transmits the data Ia and the field type attribute FIa of the data to the service infrastructure provider server 31.

  Further, another data provider (company B) server 121 similarly transmits the data Ib and the field type attribute FIb of the data to the service infrastructure provider server 31 in step S1212.

  On the other hand, in step S12161, the data beneficiary (P company) server 1216 requests the service infrastructure provider server 31 to receive data cleansed data (data after cleansing process) and performs data cleansing. The processing policy Cp is transmitted.

In step S311, the service infrastructure provider server 31 receives the data Ia and Ib and the field type attribute information FIa and FIb, and writes and stores them in the data and field type attribute information management DB 3121.
In step S315, in response to a request from the server 1216 for the data beneficiary (P company), the field type attribute information of the data Ia of the data provider <Company A> is sent to the data beneficiary (P company). FIa and A company data cleansing processing policy CaIa for data Ia are transmitted.

  In step S312, the data request and data cleansing process policy Cp is received, and written and stored in the data provider and data beneficiary data cleansing process policy information management DB 3122.

  Next, in step S313, data cleansing processing is performed based on a processing method according to the data cleansing processing policy Cp of the data cleansing processing policy information management DB 3122. The data cleansing process according to the data cleansing process policy Cp is performed for each original data.

  In addition, the data cleansing process is performed as necessary by the default data cleansing process policy Ca set by the data provider side, the default data cleansing process policy CaIa for data provider data, or the base set by the base provider side. In the provider data cleansing process policy CpIa partially changed on the data receiver side for the provider data cleansing process policy Cg or the data provider default data cleansing process policy CaIa set by the data provider You may go.

  Thereafter, in step S314, the data cleansing-processed data obtained by performing the data cleansing process on the provision data (original data) Ia from the data provider server 121 is transmitted to the data receiver server 121.

  At this time, although not illustrated, the data format of the data from the data provider server 121 is converted into a data format according to the data cleansing processing policy of the service infrastructure provider server 31 and unified as necessary. The data in the basic data format is subjected to data analysis processing by a desired analysis method (data mining). Alternatively, the basic data format or the data format of the data provider server 121 may be used. These are made in response to a data format request from the data receiver server 1216.

  Data receiver server 1216 receives data cleansing process result data in step S12162.

  Thereby, the data cleansing process policy on the data receiver side can be reflected and the data cleansing process data desired on the data receiver side can be received.

  FIG. 16 is a diagram schematically showing a flow of transmitting original data, field type attributes, and data cleansing processing policy provided from the data provider (Company A) side to the base provider side.

  In this figure, the data provider (Company A) first starts from the storage device on the base provider side with a field type attribute master (see FIG. 8), an illegal value master (see FIG. 9), and a processing method master (see FIG. 10). ). And A company default data cleansing process policy Ca is created based on these, and it transmits to the base | substrate provider side.

  Further, the company A original data Ia provided from the company A is transmitted to the platform provider side. At this time, A company original data based on A company original data Ia and the field type attribute master (see FIG. 8), illegal value master (see FIG. 9), and processing method master (see FIG. 10) taken in from the platform operator side. The field type attribute FIa of Ia and the default data cleansing processing policy CaIa for the company A data Ia are created and transmitted to the infrastructure provider side.

  The base company receives the A company default data cleansing processing policy Ca, the A company original data Ia, the field type attribute FIa of the A company original data Ia (see FIG. 8), and the default data cleansing processing policy CaIa for the A company data Ia. These are stored in the data cleansing process policy information management DB 31323 of the storage device (see FIG. 12).

  FIG. 17 transmits a data cleansing process policy from the data beneficiary (Company P) side to the base provider, and transmits data cleansed data from the base provider side to the data beneficiary side. It is the figure which showed the flow typically.

  In this figure, the data beneficiary (Company P) first starts from the storage device on the base provider side with a field type attribute master (see FIG. 8), an illegal value master (see FIG. 9), and a processing method master (see FIG. 10). ). And P company default data cleansing process policy Cp is created based on these, and it transmits to the base | substrate provider side.

  In addition, the data provider (company A) side creates the company A original data Ia, the field type attribute FIa of the company A original data Ia, and the company A data cleansing processing policy CaIa for the company A original data Ia. Import from storage.

  Then, when it is desired to change the default data cleansing processing policy CaIa for the A company data Ia with respect to the A company original data Ia, that is, when setting the processing policy according to the intention of the data receiver side, the data receiver ( A desired data cleansing process policy CpIa is set from the P company) side and transmitted to the platform provider side. For example, with reference to the field type attribute information FIa of the A company original data Ia and the A company data cleansing processing policy for the A company original data Ia, which is imported from the base company side, the P company data cleansing processing policy for the A company original data Ia is Create and reflect the created policy on the data cleansing process master on the base provider side so that the data cleansing process can be performed with the policy.

  This data cleansing process policy setting is performed, for example, using a data cleansing process setting screen as shown in FIG.

  The base company side receives the P company default data cleansing process policy Cp and the P company data cleansing process policy CpIa for the A company original data Ia from the data beneficiary (P company) side. And based on these processing policies, or A company data cleansing process policy CaIa for A company company Ia, base company default data cleansing process policy Cg, A company default data cleansing process policy Ca, and A company company data Ia On the other hand, data cleansing processing is performed.

  Thereafter, desired format conversion processing and desired analysis are performed, and the result data is transmitted to the data receiver (Company P).

  FIG. 18 is a flowchart for generating a display of a data cleansing process policy setting screen between the data beneficiary and the infrastructure provider. In the figure, the data beneficiary side obtains an irregular value list from the “illegal value master” on the platform operator side in step S1111 and in step S1112, the illegal value is displayed in the “data cleansing process policy setting screen” ( It is displayed on the vertical axis of FIG.

  In step S1113, a field type attribute list is acquired from the “field type attribute master” (see FIG. 8). In step S1114, the field type attribute value is displayed on the horizontal axis of the “data cleansing process policy setting screen”. indicate.

  In step S3115, a processing method list is acquired from the “processing method master”, and in step S1116, the value of the processing method is displayed so as to be selectable in the value column of the “data cleansing processing policy setting screen”.

In step S1117, the data receiver sets a “processing method” for each “illegal value” of each “field type attribute” on the “data cleansing processing policy setting screen”, and in step S3118, the set “ "Processing method" is sent to the platform provider (server) side.
The infrastructure provider side receives the “processing method” from the infrastructure provider side, creates a “data cleansing processing policy master” in step S3119 based on the “processing method”, and stores it in the storage device.

  FIG. 19 is a flowchart for displaying and generating a data cleansing process policy setting screen between the data provider side and the base provider side.

  In the figure, the data provider side (data provider company A) takes in the “field type attribute master”, “incorrect value master”, and “processing method” master from the base provider side in step S1121.

  In step S1122, the platform operator side uses the “data cleansing process policy setting screen” from the “field type attribute information master”, “illegal value master”, and “processing method master” to execute the “platform provider data cleansing process”. Policy Cg ”is set.

  In step S 1123, the data provider A uses the “data cleansing process policy setting screen” from the “field type attribute information master”, “illegal value master”, and “processing method master” to “A company default data cleansing process policy”. “Ca” is set.

  In step S1124, the data provider A transmits “A company original data Ia” to the platform provider.

  In step S1125, the data provider A transmits “field type attribute information type FIa” of “A company original data Ia” to the base company.

  In step S1126, the data provider A sets “default data cleansing processing policy CaIa for company A data Ia” with reference to “company A default data cleansing processing policy Ca”.

  In step S1127, the data provider A transmits “default data cleansing processing policy CaIa for company A data Ia” to the base company.

  20 to 26 are flowcharts showing an example of a procedure from creation of a data cleansing process policy to data cleansing process.

  In FIG. 20, the data beneficiary company P imports “field type attribute information master”, “illegal value master”, and “processing method master” from the base provider in step S1411, and “data cleansing processing policy setting screen”. Is used to create “P company default data cleansing policy Cp” and send it to the base operator.

  In step S1412, the data beneficiary company P fetches “field type attribute FIa for company A data Ia” and “data cleansing processing policy CaIa for company A data Ia” from the base company.

  In step S1413, the data receiver P uses the “data cleansing process policy setting screen” with reference to “field type attribute FIa for company A data Ia” and “data cleansing process policy CaIa for company A data Ia”. Then, “Company P data cleansing processing policy CpIa for company A original data Ia” is created and transmitted to the base operator.

  Thereafter, the process proceeds to * 1 in FIG. In the figure, in step S1421, the number of records i of data Ia is processed until i reaches the number of records (i ≦ number of records). In step S1422, the field j in the record i of data Ia is The processing from 1 to the number of fields (j ≦ number of fields) is executed, and the process proceeds to step S1423, where it is determined whether or not the field (i, j) is an illegal value.

As a result, in the case of an invalid value (Y), the process proceeds to step S1424, and in this step, the presence / absence of the P company data cleansing processing policy CpIa for the A company original data Ia is determined.
If it is not an illegal value (N), the process proceeds to step S1430, and a process described later is performed.

  If CpIa is present in step S1424, the process proceeds to * 0 in FIG. 25, and processing according to the processing method in FIG. If there is no CpIa, the process proceeds to step S1425, and in this step, the presence / absence of the data cleansing processing policy CaIa for company A data Ia is determined.

  If CaIa is present in step S1425, the process proceeds to * 7out in FIG. 23, and processing according to the processing method in FIG. If there is no CaIa, the process proceeds to step S1426, and the presence or absence of the P company default data cleansing processing policy Cp is determined in this step.

  If Cp is present in this step, the process proceeds to 6out in FIG. 24, and processing according to the processing method in FIG. If there is no Cp, the process proceeds to step S1427, and the presence / absence of the company A default data cleansing processing policy Ca is determined in this step.

  If Ca is present in this step, the process proceeds to * 9out in FIG. 25, and processing according to the processing method in FIG. If there is no Ca, the process proceeds to step 1428, where the presence / absence of the base provider default data cleansing processing policy Cg is determined.

  If Cg is present in this step, the process proceeds to * 8out in FIG. 26, and processing according to the processing method in FIG. If there is no Cg, the process returns to * 100, 1 is added to j (j = j + 1) in step S1430, 1 is added to i (i + 1) in step S1431, and the process ends.

Next, processing according to the processing method of FIG. 10 will be described with reference to FIGS.
If CpIa is present in step S1424 in FIG. 21, in step S14241 in FIG. 22, the processing policy in FIG. 14 is referred to and a processing method ID is acquired.

  Next, in step S14242, it is determined whether the processing method ID is 1, 2, 3, 4, or 5. Here, if ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 (Y), the process returns to * 4 in FIG. 21, and the process according to the process method ID is performed in step S1429. Execute. In addition, the process ends through the above-described steps S1430 and S1431.

  When ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 are not satisfied in step S14243 in FIG. 22 (N), the process proceeds to step S14243, and in this step, the processing method ID is ID. It is determined whether or not = 7.

  If the processing method ID is 7 (Y), the process returns to * 7 in FIG. 21 and the above-described processing is executed. If the processing method ID is not 7 (N), the process proceeds to step S14244, and in this step, it is determined whether the processing method ID is ID = 6.

  If the processing method ID = 6 (Y), the process returns to * 6 in FIG. 21 and the above-described processing is executed. If the processing method ID is not 6 (N), the process proceeds to step S14245, and in this step, it is determined whether the processing method ID is ID = 9.

  If the processing method ID is 9 (Y), the process returns to * 9 in FIG. 21 and the above-described processing is executed. When the processing method ID is not 9 (N), the process proceeds to step S14246, where it is determined whether the processing method ID is 8 or not.

  If the processing method ID is 8 (Y), the process returns to * 8 in in FIG. 21 and the above-described processing is executed. When the processing method ID is not 8 (N), the processing is terminated.

  In step S1425 of FIG. 21, if CaIa is present, in FIG. 23, the processing policy ID is acquired with reference to the processing policy of FIG. 14 in step S14251.

  Next, in step S14252, it is determined whether the processing method ID is 1, 2, 3, 4, or 5. Here, if ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 (Y), the process returns to * 4 in FIG. 21, and the process according to the process method ID is performed in step S1429. Execute. In addition, the process ends through the above-described steps S1430 and S1431.

  If ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 are not satisfied (N), the process proceeds to step S14254, and whether or not the processing method ID is ID = 6 in this step. judge.

  If the processing method ID = 6 (Y), the process returns to * 6 in FIG. 21 and the above-described processing is executed. If the processing method ID is not 6 (N), the process advances to step S14255 to determine whether or not the processing method ID is ID = 9.

  If the processing method ID is 9 (Y), the process returns to * 9 in FIG. 21 and the above-described processing is executed. If the processing method ID is not 9 (N), the process advances to step S14256 to determine whether or not the processing method ID is 8 in this step.

  If the processing method ID is 8 (Y), the process returns to * 8 in in FIG. 21 and the above-described processing is executed. When the processing method ID is not 8 (N), the processing is terminated.

  If Cp is present in step S1426 of FIG. 21, the processing method ID is acquired in step S14261 in FIG. 24 with reference to the policy in FIG.

  Next, in step S14262, it is determined whether the processing method ID is 1, 2, 3, 4, or 5. Here, if ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 (Y), the process returns to * 4 in FIG. 21, and the process according to the process method ID is performed in step S1429. Execute. In addition, the process ends through the above-described steps S1430 and S1431.

  If ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 are not satisfied (N), the process proceeds to step S14265, and whether or not the processing method ID is ID = 9 in this step. judge.

  If the processing method ID is 9 (Y), the process returns to * 9 in FIG. 21 and the above-described processing is executed. If the processing method ID is not 9 (N), the process proceeds to step S14266, and it is determined whether or not the processing method ID is 8 in this step.

  If the processing method ID is 8 (Y), the process returns to * 8 in in FIG. 21 and the above-described processing is executed. When the processing method ID is not 8 (N), the processing is terminated.

  In step S1427 in FIG. 21, if Ca is present, in FIG. 25, in step S14271, the policy in FIG. 14 is referred to and the processing method ID is acquired.

  Next, in step S14272, it is determined whether the processing method ID is 1, 2, 3, 4, or 5. Here, if ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 (Y), the process returns to * 4 in FIG. 21, and the process according to the process method ID is performed in step S1429. Execute. In addition, the process ends through the above-described steps S1430 and S1431.

  If ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 are not satisfied (N), the process proceeds to step S14275, and whether or not the processing method ID is ID = 8 is determined in this step. judge.

  If the processing method ID is 8 (Y), the process returns to * 8 in in FIG. 21 and the above-described processing is executed. When the processing method ID is not 8 (N), the processing is terminated.

  If Cg is present in step S1428 of FIG. 21, the processing method ID is acquired in step S14281 with reference to the policy of FIG. 14 in FIG.

  Next, in step S14282, it is determined whether the processing method ID is 1, 2, 3, 4, or 5. Here, if ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 (Y), the process returns to * 4 in FIG. 21, and the process according to the process method ID is performed in step S1429. Execute. In addition, the process ends through the above-described steps S1430 and S1431.

  When ID = 1, ID = 2, ID = 3, ID = 4, and ID = 5 are not satisfied (N), the process returns to * 100 in FIG.

  In addition, this invention is not limited to the Example mentioned above, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor. Information such as programs, tables, and files for realizing each function can be stored in a recording device such as a memory or a hard disk.

  Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

DESCRIPTION OF SYMBOLS 1 Service enjoyment provider apparatus 2 Service provision provider apparatus 3 Service infrastructure provider apparatus 31 Server 311 for infrastructure providers Input / output device (input / output interface means)
312 Storage device (storage means)
313 Arithmetic unit (control means)
121 service provider server 31 service infrastructure provider server 121 data provider server 1216 data beneficiary server 3121 data provider and data beneficiary master DB / data and field type attribute information management DB
3122 Data Provider & Data Receiver Data Cleansing Process Policy Information Management Master DB
3123 Field type attribute master DB
3124 Invalid value master DB
3125 Processing Method Master DB
31311 Data provider & data receiver recognition unit 3112 Data & field type attribute information receiving unit 31313 Data cleansing processing unit 31314 Data cleansing processed post-data distribution unit 31315 Data cleansing processing policy receiving unit 31316 Data writing / reading unit

Claims (9)

A data providing system having at least one service infrastructure company device, a plurality of data provider devices, and a plurality of data receiver devices,
The service platform operator device is:
Input / output means, storage means, control means,
The input / output means includes
Send and receive data between the data provider device and the data beneficiary device,
The storage means includes a data cleansing process policy master for data provided from the data provider device,
The control means has a function of performing a data cleansing process of the data with a data cleansing process policy of the data cleansing process policy master,
A receiving unit that receives data provided from the data provider device and field type attribute information of the data;
A data cleansing process policy receiving unit for receiving the data cleansing process policy;
Data cleansing processing means for performing data cleansing processing on data provided from the data provider device according to the data cleansing processing policy;
Means for transmitting the data cleansing data after the data cleansing processing by the data cleansing processing means to the data receiver device;
A customer-specific data cleansing processing system. In the customer-specific data cleansing processing system according to claim 1,
The control means further includes
Default data cleansing processing policy of the data receiver device, data receiver data cleansing processing policy for data provided from the data provider device, default data cleansing processing policy of the data provider device, and the data provider device Means for receiving a data provider data cleansing processing policy for data provided from and reflecting the data on the data cleansing processing policy master;
A customer-specific data cleansing processing system. In the customer-specific data cleansing processing system according to claim 2,
The control means further includes
A field type attribute master, an illegal value master, a processing method master, and a means for creating a data cleansing processing policy for the service infrastructure provider device from the field type attribute master, the illegal value master, and the processing method master. The means for reflecting in the data cleansing process policy master reflects the data cleansing process policy of the service infrastructure provider apparatus in the data cleansing process policy master. In the customer-specific data cleansing processing system according to claim 1,
The data cleansing process policy master includes an original data field name of data provided from the data provider device, a processing method corresponding to each illegal value that may occur in the field type attribute of the field name,
The illegal values are “blank (NULL)”, “full-width half-width inconsistency”, “illegal character”, “abnormal value”, “hiragana character inconsistency”, “kana character inconsistency”, “English letter inconsistency”, “ 1 of "Numeric Ununiformity", "Unidentified Symbol", "Unidentified Kanji Character", "Denormalized Phone Number", "Denormalized Postal Code", "Denormalized Address", "Denormalized Amount" Two or more
The type attribute of the field is one or more of “primary key”, “name”, “zip code”, “address”, “quantity”, “amount”, “date”,
The processing methods are “do nothing”, “delete all records”, “fit to data provider format”, “fit to data provider normalization”, “blank”, “replace with average value”, A customer-specific data cleansing processing system characterized by being one or more of the following. Receives the original data provided from the provider's device for data providers, and includes a base provider device that provides the original data to the data receiver device, and data sharing data among multiple providers In customer-specific data cleansing processing system for cleansing processing,
The infrastructure provider device is
A data cleansing processing control device that performs data cleansing processing on original data from the data provider device, an illegal value master that defines an illegal value that can occur in each field of the original data, and an illegal value that can occur in each field of the original data A data cleansing processing policy master having a processing method for
The data provider device comprises:
When the value of the data is an illegal value with respect to the original data, there is means for specifying which item of the illegal value master the illegal value corresponds to,
The data receiver device is:
Means for setting a processing policy for illegal values for field type attribute types prepared comprehensively for the original data;
The data cleansing process control device includes:
Reading means for reading the processing policy set on the data receiver device side from the data cleansing processing policy master for the field type attribute information of the original data set on the data provider device side, and the reading A customer-specific data cleansing processing system, comprising: control means for performing data cleansing processing on the original data in accordance with a data cleansing processing policy. A data cleansing processing method for each customer in a data providing system having at least one service infrastructure provider device, a plurality of data provider devices, and a plurality of data receiver devices,
The service platform operator device is:
Input / output means, storage means, control means,
The input / output means includes
Send and receive data between the data provider device and the data beneficiary device,
The storage means includes a data cleansing process policy master for data provided from the data provider device,
The control means has a function of performing a data cleansing process of the data with a data cleansing process policy of the data cleansing process policy master,
Receiving data provided from the data provider device and field type attribute information of the data;
Receiving a data cleansing process policy of the data cleansing process policy master;
Performing data cleansing processing on data provided from the data provider device according to the data cleansing processing policy;
Transmitting data cleansing-processed data after data cleansing processing by the data cleansing processing means to the data receiver device;
A customer-specific data cleansing processing method. In the customer-specific data cleansing processing method according to claim 6,
The control means further includes
Default data cleansing processing policy of the data receiver device, data receiver data cleansing processing policy for data provided from the data provider device, default data cleansing processing policy of the data provider device, and the data provider device Receiving a data cleansing process policy for data provider for data provided from and reflecting it on the data cleansing process policy master;
A customer-specific data cleansing processing method. In the customer-specific data cleansing processing system according to claim 7,
The control means further includes
A field type attribute master, an illegal value master, a processing method master, and a step of creating a data cleansing processing policy for the service infrastructure provider device from the field type attribute master, the illegal value master, and the processing method master. The means for reflecting in the data cleansing process policy master has a step of reflecting the data cleansing process policy of the service infrastructure provider device in the data cleansing process policy master. The customer-specific data cleansing process system. In the customer-specific data cleansing processing method according to claim 6,
The data cleansing process policy master includes: an original data field name of data provided from the data provider device; a field type attribute corresponding to each illegal value that may occur in the field type attribute of the field name; and a process for the incorrect value Including methods,
The illegal values are “blank (NULL)”, “full-width half-width inconsistency”, “illegal character”, “abnormal value”, “hiragana character inconsistency”, “kana character inconsistency”, “English letter inconsistency”, “ 1 of "Numeric Ununiformity", "Unidentified Symbol", "Unidentified Kanji Character", "Denormalized Phone Number", "Denormalized Postal Code", "Denormalized Address", "Denormalized Amount" Two or more
The type attribute of the field is one or more of “primary key”, “name”, “zip code”, “address”, “quantity”, “amount”, “date”,
The processing methods are “do nothing”, “delete all records”, “fit to data provider format”, “fit to data provider normalization”, “blank”, “replace with average value”, A customer-specific data cleansing processing method, characterized by being one or more of the following.

Images