JP2020149451A - Data processor and data processing method - Google Patents

Abstract

To use data having no representative property as data having representative property.SOLUTION: A data processor comprises: a first storage part for storing pieces of first data which are collected for first object people being selected randomly, as objects, according to the number of the first object people; a second storage part for storing pieces of second data which are collected for second object people whose number is larger than that of the first object people and which satisfy a predetermined collection condition, according to the number of the second object people; and a data extraction part for extracting second data which is used as data for adding, from pieces of the second data stored in the second storage part. Both of the pieces of first and second data comprise data indicating a content of a common item which is common to both pieces of data. The data extraction part calculates a similarity of the content of the common item between each of pieces of first data and each of pieces of second data, and extracts second data of the second object people identified on the basis of the calculated similarity as data for adding.SELECTED DRAWING: Figure 4

Description

The present invention relates to a data processing apparatus and a data processing method, and more particularly to a data processing apparatus and a data processing method for processing non-representative data.

When aggregating the collected data from various places, the representativeness of the data is important. Here, "representativeness" means whether or not the survey results of some of the survey subjects extracted from all the survey subjects can accurately reflect the results of the entire survey subjects without bias. , If it can be reflected accurately, it is said to be "representative".

Further, conventionally, a technique for ensuring the representativeness of collected data has been developed, and an example thereof is the technique described in Patent Document 1. Patent Document 1 discloses a search support device capable of efficiently selecting a representative search subject from the entire search target at low cost.

JP-A-2015-185008

On the other hand, with the development of communication technology in recent years, it has become possible to collect large-scale data (for example, log data showing a specific behavior history of a person) as represented by so-called big data. Since the randomness of the data collection target is not guaranteed, there is a risk that the collected data will lack representativeness.

Specifically, if data is collected by targeting a person who satisfies the conditions for collecting log data, the representativeness of the log data is not guaranteed. If log data that is not representative is aggregated as it is, the aggregation result may be biased. Therefore, it is necessary to take measures (correction) to eliminate the above bias for data that is not representative. However, it may be difficult to identify the cause of the bias only from the non-representative data, and in such a case, it is difficult to correct the non-representative data alone.

Therefore, the present invention has been made in view of the above circumstances, and an object of the present invention is to solve the following object.
Specifically, an object of the present invention is to solve the above-mentioned problems of the prior art and to provide a data processing apparatus and a data processing method for using non-representative data as representative data. ..

In order to achieve the above object, the data processing apparatus of the present invention stores the first data collected for the first target person randomly selected, in an amount corresponding to the number of the first target person. The first storage unit and the second data acquired for the second target person who meets the predetermined collection conditions and is larger than the first target person are obtained according to the number of the second target person. It has a second storage unit that stores minutes and a data extraction unit that extracts the second data to be used as aggregation data from the second data stored in the second storage unit. Both the first data and the second data include data indicating the contents of common items common to both, and the data extraction unit includes each of the first data and the second data. It is characterized in that the degree of similarity of the contents of the common items is calculated with each of them, and the second data of the second target person specified based on the calculated degree of similarity is extracted as the aggregation data. And.

The data processing apparatus of the present invention configured as described above extracts the data specified based on the representative first data from the non-representative second data as the aggregation data.
More specifically, the second data of the second target person specified based on the degree of similarity of the contents of the common items calculated between each of the first data and each of the second data is the data for aggregation. Extract as. By such a procedure, data similar to the representative first data (specifically, data having similar contents of common items) can be extracted from the non-representative second data. Then, the extracted second data can be treated as representative data.
As a result of the above, even data that is not originally representative can be treated as if it is representative data.

Further, in the above data processing device, the second data may be data collected when the second target person who satisfies the collection condition performs a specific action.
In the above configuration, when the second target person performs a specific action, the second data is collected by using this as a trigger. When the second data is collected in this way, the representativeness of the second data is likely to be lost. Therefore, the "non-representative data" of the present invention can be treated as representative data. The effect of "to do so" will be more prominent.

Further, in the above data processing device, the second target person is a person who satisfies the condition that the device used by the second target person for contacting the broadcasting media is connected to the Internet as the collection condition. You may.
With the above configuration, the second data is collected from the target person (second target person) whose device used for contacting the broadcasting media is connected to the Internet. In this case, the second data may be non-representative data, and the effect of the present invention will be effectively exhibited.

Further, in the above data processing device, the second data may be log data transmitted by the device when the second target person comes into contact with the broadcast media using the device.
With the above configuration, since the second data is log data transmitted by the device when it comes into contact with the broadcast media, it can be collected relatively easily, but it can be non-representative data. Further, in general, since detailed attribute information is not added to the above log data, it is difficult to correct the log data alone (implementation of bias elimination measures). Therefore, when the above log data is collected as the second data, the effect of the present invention will be more prominently exhibited.

In the above data processing apparatus, the common item may be the contact status with the broadcast media. Further, the contact status with the broadcast media may be a television viewing status.
With the above configuration, data for aggregation is obtained so that the second data, which has no representativeness, becomes representative data based on the degree of similarity of the contact status with the broadcasting media (for example, the viewing status of television). Can be extracted.

In the above data processing apparatus, the data extraction unit changes the combination of the first target person and the second target person, calculates the degree of similarity for each combination, and for each of the first target persons, the said The second target person belonging to the combination is specified in order from the combination having the maximum degree of similarity, and when the number of the specified second target persons reaches the set number of people, the number of the specified set number of people is reached. It is preferable to extract the second data of the second subject as the aggregation data.
With the above configuration, each first target person is specified in order from the second target person who has the most similar contents of common items. Then, when the number of the specified second target person reaches the set number of people, the second data of the second target person specified so far is extracted as aggregation data. This makes it possible to extract more appropriate data for ensuring representativeness when extracting aggregation data from the second data.

Further, in the above data processing device, when the data extraction unit identifies the second target person for the set number of people, if a certain second target person is specified more than once, the data extraction unit Is preferable to aggregate the second data of the second subject as the same number of aggregation data as the number of times the second subject is specified.
In the above configuration, when the second target person for the set number of people is specified, a certain second target person may be specified more than once. In this case, if the second data of a certain second target person is extracted as the same number of aggregation data as the number of times the person is specified, the specific number of times is calculated when the aggregation work using the aggregation data is subsequently performed. It can be used as an aggregation weight, and more appropriate aggregation becomes possible.

Further, in order to solve the above-mentioned problems, in the data processing method of the present invention, the first storage unit collects the first data for the first target person randomly selected, and the first data is collected by the first target person. The second data collected by the second storage unit for the second target person, which meets the predetermined collection conditions and is larger than the first target person, is stored as much as the number of people. The amount corresponding to the number of the second target person is stored, and the computer extracts the second data to be used as the aggregation data from the second data stored in the second storage unit, and the said Both the first data and the second data include data indicating the contents of common items common to both, and the computer uses each of the first data and each of the second data. It is characterized in that the degree of similarity of the contents of the common items is calculated among the two, and the second data of the second target person specified based on the calculated degree of similarity is extracted as the aggregation data.
According to the above data processing method, the first data which is originally non-representative can be extracted as representative data for aggregation.

According to the present invention, a data processing apparatus and a data processing method for using non-representative data as representative data are realized.

It is explanatory drawing of each collected data. It is a figure which shows the structure of the data processing apparatus which concerns on one Embodiment of this invention. It is an image diagram which shows the extraction procedure of the 2nd data. It is a figure which shows the flow of the data processing method which concerns on one Embodiment of this invention. It is a table which shows the correspondence relation between the 1st data of each 1st subject and the 2nd data of each 2nd subject. It is explanatory drawing of the procedure for identifying the 2nd subject similar to the 1st subject. It is a figure which shows the number of times of extraction about the 2nd data of the 2nd subject specified by duplication. It is a figure when the second data of each second subject is clustered according to the degree of similarity with the first data of the first subject.

A data processing apparatus and a data processing method according to an embodiment of the present invention (the present embodiment) will be described in detail below with reference to the accompanying drawings.
It should be noted that the embodiments described below are merely examples for facilitating the understanding of the present invention, and do not limit the present invention. That is, the present invention may be modified or improved from the embodiments described below without departing from the spirit of the present invention. Moreover, as a matter of course, the present invention includes an equivalent thereof.

Further, in the present specification, the term "device" refers to one device that independently exerts a specific function, or a plurality of devices that exist in a dispersed manner but cooperate to exert a specific function. Also includes.

Further, in the following description, "person" or "person" is a concept including not only an individual but also a group (for example, a household) to which the individual belongs.

Further, in the following description, "broadcast media" is an information transmission medium (mass media) that distributes programs and advertisements by radio wave broadcasting or data broadcasting, and specifically, television (including Internet television) and radio. (Including IP simul radio) etc. are applicable.
In the following, television will be described as an example of broadcasting media. However, the contents described below can, of course, be applied to broadcasting media other than television.

Further, in the following description, the "device used for contacting the broadcasting media" is a device that receives video signals and audio signals distributed from the broadcasting media, and specifically, a television receiver and a radio receiver. When using a TV or radio via the machine and the Internet, terminal devices (computers, tablet terminals, smartphones and mobile phones) connected to the Internet are applicable.
In the following, a television receiver (receiver) will be described as an example of a device used for contacting broadcast media. However, the contents described below may, of course, be applied to devices other than television receivers used for contacting broadcast media.

Further, in the following description, "viewing" means not only viewing the broadcasted program and advertisement in real time, but also recording or recording the program and advertisement and reproducing and viewing the program and advertisement within a certain period after the broadcast. This includes so-called time-shifted viewing, in which users watch what is distributed on the Web.
Further, in the following explanation, "attribute" is a demographic such as a person's gender and age, a psychographic such as a person's interests and lifestyle, and a classification set according to a behavioral tendency and a behavioral history. is there.

<< About the first data and the second data >>
Prior to the description of the data processing apparatus and the data processing method of the present embodiment, the first data and the second data to be processed will be described with reference to FIG. FIG. 1 is an explanatory diagram of each collected data and shows a distribution route of each collected data.

(First data)
The first data is data for a survey collected by conducting a survey (strictly speaking, a sample survey) on the first subject T. Here, the first subject T is a person randomly (randomly) selected from the set population by a statistical method. Further, in the present embodiment, when randomly selecting the first target person T from the population, a survey point / area is set, and the first target person T is selected from those who live in the survey point / area. .. However, the selection method of the first target person T is not limited to the above selection method, and any method can be freely adopted as long as it is a random selection method.

Further, the number of selected first target persons T is preferably set to an appropriate number according to the purpose of the survey, but in the present embodiment, for example, it is set to a scale of several hundred to tens of thousands. And. By the way, in FIG. 1, the number of the first subject T is smaller than the actual number for the convenience of illustration.

Explaining the survey targeting the first target person T, in the present embodiment, the first target person T is surveyed regarding the contact situation with the broadcasting media, specifically, the television viewing situation. More specifically, at the home of the first subject T, a known measuring device (not shown) for measuring the viewing time and viewing channel of television is installed. With this measuring device, the TV viewing situation of the first subject T is investigated every day during the investigation period.

The above-mentioned measuring device generates data (hereinafter, also referred to as measurement data) indicating a measurement result on a regular basis (for example, every minute) and transmits the data to the first collection center C1. The first collection center C1 receives measurement data from the measuring device of each first subject T through a dedicated communication line. As a result, the first collection center C1 acquires measurement data indicating the viewing status of television in an amount corresponding to the number of first target persons T. In addition, the first collection center C1 stores and stores the measurement data acquired from each first subject T in a database.

Here, the measurement data acquired by the first collection center C1 from each first subject T corresponds to the first data, and shows the survey result regarding the television viewing situation of each first subject T. More specifically, the measurement data acquired from each first target person T includes the identification information of each first target person T, and a TV station that broadcasts a TV program or TV commercial viewed by each first target person T ( It contains data indicating the viewing channel), viewing date, viewing time, and the like.

The transmission of the first data, that is, the transmission of the measurement data from the measuring device may be performed in minutes, or the data for one hour or one day may be transmitted together. Further, in the present embodiment, as the first data indicating the viewing status of the television, the measurement data sent from the above-mentioned measuring device is acquired through the communication line, but the present invention is not limited to this. For example, each first target person T fills in the TV viewing status (specifically, the viewing time for each TV station in each time zone, etc.) on a predetermined entry form, and the first collection center C1 has completed the form. Is collected from each first target person T, and by inputting the contents of the collected form on the first collection center C1 side, the data (first data) indicating the TV viewing status of the first target person T is first. It may be acquired for each target person T.

By the way, since the measurement data (first data) corresponding to the number of the first target person T is the data acquired for the randomly selected first target person T as described above, it is representative. It can be said that the data has sex. That is, the TV viewing status of each first target person T shown in the first data accurately reflects the TV viewing status of the population (all the people living in the survey point / area) without bias. Become.

(Second data)
The second data is data collected for the second target person U. Here, the second target person U is a restricted target person who satisfies a preset selection condition. Specifically, the second target person U of the present embodiment has the condition that the TV viewing device (that is, the TV receiver) used by the person is connected to the Internet as the above collection condition. A person who meets. More strictly speaking, the second target person U is a person who has consented to provide log data (hereinafter, also referred to as device log data) indicating a television viewing history from a television receiver via the Internet.

The collection conditions of the second target person U are not limited to the above contents, and may be conditions other than the above contents, for example, on the condition that the Internet is used on a daily basis. There may be.

Further, the number of the second target person U is larger than the number of the first target person T selected, and in the present embodiment, for example, it is assumed that the number of the second target person U is several hundred thousand to several million. By the way, in FIG. 1, the number of the second subject U is smaller than the actual number for the convenience of illustration.

Explaining the data collection for the second target person U, in the present embodiment, as with the first target person T, data collection regarding the contact status with the broadcasting media, specifically, the viewing status of television is performed. More specifically, at the home of each second subject U, a television receiver (not shown) capable of transmitting device logs is installed, and the television receiver is connected to the Internet. (Connected).

Then, while each second target person U is watching TV on the TV receiver, the TV receiver periodically generates device log data (for example, at intervals of 1 minute to several minutes). , Stored in the storage device in the TV receiver. Further, the television receiver transmits the device log data accumulated up to that point to the manufacturer M of the television receiver via the Internet at a fixed cycle (for example, a cycle of 1 hour to 1 day). The manufacturer M provides the received device log data to the second collection center C2.

The second collection center C2 receives the device log data of each second target person U from the manufacturer M through a communication line such as the Internet. As a result, the second collection center C2 acquires the device log data corresponding to the number of the second target person U. Further, the second collection center C2 stores and stores the device log data of each second target person U in a database. After that, in the second collection center C2, the accumulated device log data is aggregated and used for a predetermined analysis (for example, audience rating calculation).

In the case shown in FIG. 1, the first collection center C1 and the second collection center C2 exist separately, but the present invention is not limited to this, and the first collection center C1 and the second collection center C2 It may be the same institution. Further, in the case shown in FIG. 1, there is only one manufacturer M of the television receiver, but of course, there may be a plurality of manufacturers M, and in that case, the second collection center C2. Will be provided with device log data from each manufacturer M.

Further, regarding the device log data, the cycle of transmission from the television receiver of each second target U to the manufacturer M and the cycle provided by the manufacturer M to the second collection center C2 can be arbitrarily set. For example, the data may be set in minutes, or the data for one hour to one day may be collectively transmitted.

Further, in the present embodiment, device log data (second data) indicating a television viewing log is transmitted from the television receiver of each second target U to the second collection center C2 via the manufacturer M. However, it is not limited to this. For example, data indicating a viewing log may be sent from the television receiver of each second target person U to the television station side via the Internet, and then transmitted from the television station to the second collection center C2. Alternatively, the second collection center C2 may receive the device log data directly from the television receiver of each second subject U.

Here, the device log data of each second target person U acquired by the second collection center C2 corresponds to the second data, and indicates the television viewing status of each second target person U. More specifically, the device log data is log data transmitted by the TV receiver when the second target U watches TV using the TV receiver. Specifically, each second target person It includes U identification information, a TV station (viewing channel) that broadcasts a TV program or TV CM watched by each second target person U, viewing date, viewing time, and the like. The identification information of the second target person U corresponds to ID information (device ID) or the like incorporated in the device log data.

By the way, in the present embodiment, as the second data indicating the TV viewing status of each second target U, the device log data transmitted from the TV receiver of each second target U is acquired through a communication line such as the Internet. I decided to do it, but it is not limited to this. For example, each second target person U fills in the TV viewing status (specifically, the viewing time for each TV station in each time zone, etc.) on a predetermined entry form, and the second collection center C2 has completed the form. Is collected from each second target person U, and the above data (second data) can be obtained for each second target person U by inputting the contents of the collected form on the second collection center C2 side. Good.

As described above, both the measurement data, which is the first data, and the device log data, which is the second data, include data indicating the TV viewing status (specifically, the viewing time and the viewing channel). There is. In other words, both the first data and the second data include data indicating the contents of common items common to both, and in the present embodiment, the contents of the common items are the television viewing status. There is.

To add to the second data (specifically, device log data), the second data is collected when the second target person who meets the collection conditions performs a specific action (specifically, TV viewing behavior). Data may lack representativeness. In this case, when the second data having no representativeness is aggregated and the analysis process (for example, the audience rating calculation process) is performed, the analysis result may be biased.

To explain with a specific example, it is assumed that the person selected as the second target person U tends to correspond to a unique attribute (for example, an attribute that the Internet is frequently used). In that case, since a bias occurs due to the above attributes, when the device log data of each second target person U is aggregated to calculate the TV audience rating, the calculation result is other than the device log data. It may not accurately reflect the results of all the subjects (that is, the population), such as being smaller than the audience rating calculated from the data of.

On the other hand, the device log data as the second data generally does not include detailed attribute information regarding the second target person U of the data provider. Therefore, it is difficult to identify the cause of the bias only from the device log data, and it is difficult to correct the device log data alone (measures for eliminating the bias).

Therefore, in the present invention, the device log data whose representativeness is not guaranteed is processed based on the representative measurement data so that it can be handled as the representative data. Specifically, by using the data processing device according to the present embodiment, a part of the device log data corresponding to the number of the second target person U is extracted as aggregation data. At that time, the data processing device according to the present embodiment includes the measurement data of each first target person T and the device log data of each second target person U so that the representativeness of the extracted device log data is guaranteed. Extract device log data based on the relationship of. As a result, if a predetermined analysis is performed using the extracted device log data (that is, aggregation data), the analysis result is less likely to be biased.

The functions of the data processing device according to this embodiment will be described in detail in the following sections.

<< About the configuration of the data processing device according to this embodiment >>
The configuration of the data processing device (hereinafter, data processing device 10) according to the present embodiment will be described with reference to FIG. FIG. 2 is a diagram showing the configuration of the data processing device 10.

The data processing device 10 is a device that processes measurement data corresponding to the number of the first target person T and device log data corresponding to the number of the second target person U. In the present embodiment, the data processing device 10 includes a server computer managed and used by the first collection center C1 (hereinafter, processing side server 11) and a server computer managed and used by the second collection center C2 (hereinafter, hereinafter). It is configured by a data provider server 12). That is, in the present embodiment, the processing side server 11 and the data providing side server 12 cooperate in order to exert the function as the data processing device 10. However, the present invention is not limited to this, and even if one of the servers of the first collection center C1 and the second collection center C2 has the functions of the other server and the data processing device 10 is configured by one unit. Good. Alternatively, a third server different from either the first collection center C1 or the second collection center C2 may function as the data processing device 10. For example, the ASP (Application Service Provider) server may serve as the data processing device 10. The function may be provided as an ASP service.

The processing side server 11 and the data providing side server 12 have the same hardware configuration as a normal server computer, and as shown in FIG. 2, communicate with the CPUs 11a and 12a and the memories 11b and 12b composed of ROM and RAM. It has interfaces 11c and 12c, hard disk drives 11d and 12d as auxiliary storage devices, input devices 11e and 12e including a keyboard and a mouse, and output devices 11f and 12f including a display and a printer. In addition, a program (data processing program) for exerting a function corresponding to each server among the functions of the data processing device 10 is installed in each of the processing side server 11 and the data providing side server 12. ..

The processing side server 11 receives the measurement data from the measuring device of each first target person T, and stores and stores the measurement data corresponding to the number of the first target person T in the hard disk drive 11d. That is, the hard disk drive 11d of the processing side server 11 functions as the "first storage unit" of the present invention. However, the present invention is not limited to this, and the auxiliary storage device connected to the processing side server 11 in an external format or another computer (data providing side server 12) communicably connected to the processing side server 11 is used. Includes) may function as the first storage unit.

The data provider server 12 receives the device log data of each second target U from the manufacturer M, and stores and stores the device log data corresponding to the number of the second target U in the hard disk drive 12d. There is. That is, the hard disk drive 12d of the data providing side server 12 functions as the "second storage unit" of the present invention. However, the present invention is not limited to this, and the auxiliary storage device connected to the data providing side server 12 in an external format or another computer (processing side server 11) connected to the data providing side server 12 in a communicable manner. (Including) may function as a second storage unit.

Further, in the present embodiment, the processing side server 11 accesses the hard disk drive 12d of the data providing side server 12 by communicating with the data providing side server 12, and each second target person stored in the hard disk drive 12d. The device log data (second data) of U can be read.

Further, the processing side server 11 functions as a computer that forms a main part of the data processing device 10. More specifically, the processing side server 11 selects the device log data used as the aggregation data from the device log data of each second target person U stored in the hard disk drive 12d of the data providing side server 12. Extract. That is, the processing side server 11 functions as the "data extraction unit" of the present invention. Strictly speaking, the "data extraction unit" of the present invention is realized by the cooperation between the CPU 11a of the processing side server 11 and the data processing program installed in the processing side server 11.

The extraction result of the device log data (that is, which second target U's device log data was extracted as aggregation data) is first collected by the processing side server 11 communicating with the data providing side server 12. The center C1 notifies the second collection center C2. The device log data extracted as the aggregation data is used for the analysis process performed on the second collection center C2 side.

Further, when the processing side server 11 extracts the device log data, the processing side server 11 refers to the measurement data of each first target person T stored in the hard disk drive 11d of the processing side server 11, and the measurement data of each first target person T. And the device log data of each second target person U are obtained.

Specifically, the processing side server 11 associates the device log data of each second target person U with the measurement data of each first target person T. By this association, the device log data of the second subject U, which is similar to the measurement data of each first subject T, can be calculated. Here, "similarity" means that the contents of common items in the first data and the second data, specifically, the viewing situation of television is similar. The association of device log data with each measurement data will be described in detail later.

After that, as shown in FIG. 3, the processing side server 11 is similar to the measurement data of each first target person T among the device log data of each second target person U stored in the data providing side server 12 side. Extract the device log data that is being used as aggregation data. As shown in FIG. 3, the device log data extracted in this way is similar to the representative measurement data, and thus can be treated as pseudo “representative data”.
FIG. 3 is an image diagram showing a procedure for extracting device log data.

After extracting the data, the processing side server 11 transmits information indicating the extraction result of the device log data to the data providing side server 12. As a result, it becomes possible for the second collection center C2 side to grasp which device log data of the second target person U has been extracted by the processing side server 11. Then, the second collection center C2 uses the extracted device log data as aggregation data to perform a predetermined analysis. At this time, as described above, the extracted device log data can be used as representative data, so that an analysis result with suppressed bias can be obtained.

<< About the data processing method according to this embodiment >>
Next, as an operation example of the above-mentioned data processing device 10, a flow in which the data processing device 10 processes the collected data (hereinafter, data processing flow) will be described.
In the data processing flow, the data processing method of the present invention is adopted. That is, the following description includes a description of the data processing method of the present invention, and each step in the data processing flow described below corresponds to a step constituting the data processing method of the present invention.

In the data processing flow, the data processing apparatus 10 carries out each step shown in FIG. FIG. 4 is a diagram showing a flow of a data processing method according to the present embodiment, and is an explanatory diagram of the data processing flow.

When executing the data processing flow, the first target person T is randomly selected, and the second target person U to which the television receiver is connected to the Internet consents to the provision of device log data. After that, data acquisition targeting the first target person T and data acquisition targeting the second target person U are carried out, respectively. In the present embodiment, both of the above two data acquisitions are data acquisitions related to the viewing status of television. Further, the above two data acquisitions may be carried out at the same time, or may be carried out at different times.

During the implementation period of the survey targeting the first target person T, the processing side server 11 acquires the measurement data from each first target person T on the first collection center C1 side (S001). Specifically, for the first subject T who is watching TV, the measuring device installed at that person's home periodically generates measurement data during the TV viewing time and transmits the generated data. To do. The processing side server 11 acquires (receives) the measurement data transmitted from the above-mentioned measuring device through the communication line. As a result, the processing side server 11 acquires the number of measurement data corresponding to the number of the first target person T.

Further, in step S001, the processing side server 11 stores and stores the acquired measurement data of each first target person T in the hard disk drive 11d as the first storage unit. At this time, the processing side server 11 stores the measurement data of each first target person T for each first target person T based on the identification information of the first target person T indicated by the measurement data.

In addition, step S001 is repeated, for example, until the implementation period of the survey targeting the first subject T expires.

On the other hand, during the data acquisition period for the second target U, on the second collection center C2 side, the data provider server 12 receives the device log data transmitted from the television receiver of each second target U. Acquire (S002). Specifically, when the second target person U watches TV through the TV receiver, the viewing log is stored in the TV receiver as device log data, and the TV receiver stores the device log data at a predetermined timing. Send to manufacturer M. The manufacturer M automatically provides the device log data of each second target person U in response to the request of the data providing side server 12 or at a predetermined timing. The data providing side server 12 acquires (receives) the device log data of each second target person U from the manufacturer M through a communication line such as the Internet. As a result, the data providing side server 12 acquires the number of device log data corresponding to the number of the second target person U.

Further, in step S002, the data providing side server 12 stores and stores the acquired device log data of each second target person U in the hard disk drive 12d as the second storage unit. At this time, the data providing side server 12 stores the device log data of each second target person U for each second target person U based on the identification information of the second target person U indicated by the device log data.

In addition, step S002 is repeated, for example, until the execution period of data acquisition for the second target person U expires. Further, in FIG. 4, step S002 is supposed to be performed after step S001, but the present invention is not limited to such a case, and may be performed before step S001, for example, at the same time. It may be performed, or the other step may be performed while one step is being performed.

Subsequent steps (specifically, S003 to S007 in FIG. 4) are the main flow of the data processing flow, and are mainly performed by the processing side server 11.

In steps S003 to S007, the processing side server 11, which is a computer, functions as a data extraction unit. First, the processing side server 11 communicates with the data providing side server 12 and reads out the device log data of each second target person U stored in the data providing side server 12 side (S003). The device log data may be read out for all the device log data stored in the data provider server 12, and only for the device log data corresponding to the TV viewing time or time indicated by the device log data at a predetermined time. May be good.

Next, the processing side server 11 has a degree of similarity between the measurement data of each first target person T stored by itself and the device log data of each second target person U read from the data providing side server 12. Is calculated (S004). Here, the degree of similarity is the degree of similarity regarding the contents of common items common to both the measurement data and the device log data, specifically, the television viewing situation. The processing side server 11 changes the combination of the first target person T and the second target person U, and calculates the above degree of similarity for each combination. That is, if the number of the first subject T is X and the number of the second subject U is Y (both X and Y are natural numbers), the degree of similarity is calculated for each of the XY combinations. Become.

As a method for calculating the degree of similarity between data, a known method can be used. For example, a method of obtaining a correlation coefficient as an index value of the degree of similarity may be adopted, or an absolute error (Absolute Error). The method of calculating the distance (Euclidean distance, Mahalanobis distance, cosine distance, etc.) may be adopted.

In step S004, as described above, the degree of similarity between the measurement data of each first subject T and the device log data of the second subject U for Y is calculated. As a result, for the measurement data of each first subject T, the device log data of all the second subject U can be ranked based on the degree of similarity. That is, it is possible to associate the device log data of the second subject U for Y with each of the measurement data of the first subject T for X people in the order of similarity as shown in FIG. become able to. FIG. 5 is a table showing the correspondence (association) between the measurement data of each first subject T and the device log data of each second subject U. In the figure, the notation "T _i (i is 1 to X)" represents each first target person T, and the notation "U _j (j is 1 to Y)" is each second target. Represents person U.

After that, the processing side server 11 specifies the second target person U belonging to the combination in order from the combination having the maximum degree of similarity for each first target person T (S005). That is, in this step S005, for each first target person T, the second target person U whose TV viewing status (that is, the contents of common items) is similar to that of each first target person T has a large degree of similarity. Specify in order from the side. The specific procedure of this step S005 will be described in detail below with reference to FIG. FIG. 6 is an explanatory diagram of a procedure for identifying a second subject U similar to the first subject T.

In the following explanation, in order to make the explanation easier to understand, the number of each of the first subject T and the second subject U is set to be smaller than the actual number, and specifically, the number of the first subject T The number of people X is 10, and the number Y of the second target person U is 100.

In step S004 of the previous stage, as shown in FIG. 6, the device log data of 100 second subjects U are associated with each of the measurement data of 10 first subjects T in order of similarity. ing. For example, there is the first subject T ₁ of the measurement data, the device logs the data of the second subject U ₂ are most similar, and device log data of the second subject U ₅₉ is similar to the second Since then, the device log data of the remaining 98 second subjects U are ranked according to the degree of similarity.

In step S005, the processing-side server 11 selects the combination of the first target person T and the second target person U in order from the combination having the maximum degree of similarity, referring to the relationship shown in FIG. The second target person U to which the person belongs is specified for each first target person T. More specifically, the processing side server 11 specifies, for each first target person T, the second target person U having similar TV viewing conditions by the set number of people from the one with the largest degree of similarity.

Here, the set number of people is a value set in advance when executing the data processing flow, and specifically, is the number of device log data required as aggregation data (that is, the number of required extraction data). This set number of people can be set to any number, and can be changed after setting. In the following, it is assumed that the set number of people is 20.

In step S005, the processing server 11 identifies the most similar second target U for each of the ten first target Ts. As a result, the second target U for 10 people is first identified. After that, the processing side server 11 specifies in order from the second target person U having the highest degree of similarity. By the above procedure, the second target person U for the set number of people (20 people) is specified.

Then, when the number of the specified second target U reaches the set number, the processing side server 11 has the device log data of the second target U corresponding to the specified set number (in the case of FIG. 6, in the case of FIG. Device log data for 20 people surrounded by a frame) is extracted as aggregation data (S006).

At this time, the device log data of the same second target U may be duplicated in the extracted device log data. That is, in step S005, when the processing side server 11 specifies the second target person U for the set number of people, a certain second target person may be specified more than once. For example, in the case of FIG. 6, the second subject U ₂ is identified twice and the second subject U ₇ is identified three times.

In the above case, the processing side server 11 extracts the device log data of the second target person U, which is duplicately specified, as the same number of aggregation data as the number of times that person is specified. That is, as shown in FIG. 7, the device log data of the second target U specified in duplicate is extracted in duplicate as many times as the number of times specified. FIG. 7 is a diagram showing the number of extractions of the device log data of the second target U, which is duplicately identified.

The number of times the device log data extracted in duplicate is extracted is used as a weight in the subsequent aggregation work. That is, the device log data extracted n times (n is a natural number of 2 or more) is treated as the device log data of the second target person U for n people at the time of aggregation.

In the present embodiment, as described above, the device log data of the second target U specified in duplicate is extracted as the same number of aggregation data as the number of duplicates, but the present invention is limited to this. It's not a thing. Specifically, when the second target person U for the set number of people is specified, if a certain second target person is specified more than once, one device log data of that person is not duplicated. It may be extracted as device log data. In that case, the number of extracted device log data will be less than the set number of people, so the missing data will be the device log data of the second target U with a higher ranking based on the ranking of the degree of similarity (Fig. In the case of 6, the device log data of the second subject U, which is similar to the third, may be supplemented in order.

After the end of step S006, the processing side server 11 communicates with the data providing side server 12 and extracts the extraction result in step S006 (that is, which second target U's device log data is extracted as aggregation data). The indicated information is transmitted to the second collection center C2 side (S007). When the steps up to the above are completed, the data processing flow ends.

After the end of the data processing flow, the second collection center C2 side receives the information transmitted from the processing side server 11 via the Internet, aggregates the device log data extracted by the processing side server 11, and performs a predetermined analysis. carry out.

<< About the effectiveness of this embodiment >>
As described above, in the present embodiment, between the measurement data (first data) of each first subject T and the device log data (second data) of each second subject U, a television is used. Calculate the degree of similarity regarding the viewing status (contents of common items). Then, the device log data of the second target person U specified based on the calculated degree of similarity is extracted as aggregation data.

As described above, from the device log data of the second subject U, which is originally non-representative, data similar to the measurement data of the first subject T, which has representativeness, can be extracted as aggregation data. Then, if the extracted device log data is aggregated and a predetermined analysis is performed, an analysis result with suppressed bias can be obtained.

<< Other Embodiments >>
The data processing apparatus and data processing method of the present invention have been described above with reference to one specific embodiment, but the embodiment is merely an example, and other embodiments are also conceivable.

For example, in the above-described embodiment, in the data processing flow, the device log data of all the second target persons U are associated with the measurement data of each first target person T according to the degree of similarity (strictly speaking, (Ranking) was decided. Then, for each of the first subject T, the device log data of the second subject U having a higher degree of similarity (that is, a higher rank) is extracted in order as aggregation data. However, the present invention is not limited to this, and a form of extracting device log data by another method (hereinafter referred to as a modified example) can be considered.

Specifically, a modification will be described. For example, after calculating the degree of similarity between the measurement data of each first subject T and the device log data of each second subject U, each second object is subjected to the degree of similarity. The device log data of the person U is associated with the measurement data of the first subject T, which is most similar to each of the second subject U. As a result, the device log data of each of the Y second subjects U is clustered as shown in FIG. 8, and belongs to any one of the same number of groups as the X first subjects T. Will be. FIG. 8 is a diagram when the device log data of each second subject U is clustered according to the degree of similarity with the measurement data of each first subject T. In FIG. 8, the device log data of the second subject U is indicated by a black dot, and the measurement data of the first subject T is indicated by a cross. Further, for convenience of illustration, in FIG. 8, the number of the first subject T and the second subject U is smaller than the actual number, which is 5 and 30, respectively.

After the completion of clustering, in each class (group), the device log data of the second subject U, which has the greatest degree of similarity with the measurement data of the first subject T corresponding to each class, is in order from each class. Extract device log data as aggregation data. Then, the extraction of the device log data is repeated until the device log data for the set number of people is obtained. At this time, since different device log data is extracted from each class each time, the device log data of the same second target U is not extracted a plurality of times as in the above-described embodiment. That is, in the modified example, it is possible to extract the device log data of the second target person U without duplication.

Further, in the above-described embodiment, the first data is measurement data indicating the result of measuring the television viewing status of the first target person T with a dedicated measuring device, and the second data is the second target person U. Is the viewing log (device log data) transmitted from the TV receiver when watching TV. However, the present invention is not limited to this, and the first data may be data indicating the answer contents of each question in the questionnaire survey conducted for the first subject T. Similarly, the second data may be data indicating the answer contents of each question in the questionnaire survey conducted for the second target person U. The second target person U in this case is a person who satisfies the condition (collection condition) of responding to the request of the questionnaire survey.
Other second data include the following examples.
[1] TV viewing status measurement data collected from the Internet survey panel [2] Operation log data of home appliances connected to the network (for example, hard disk recorder) [3] Present the membership card at the store to purchase products POS (Point of Sales) data with ID generated when the data is collected Here, in the example of [1], the second target is a person who satisfies the condition (collection condition) of responding to the request of the Internet survey, that is, the survey panel. Corresponds to person U. Further, in the example of [2], a person who owns the above-mentioned home electric appliances and satisfies the condition (collection condition) of responding to data collection corresponds to the second target person U. In the example of [3], a person who satisfies the condition (collection condition) of performing a purchasing action that triggers the acquisition of POS data corresponds to the second target person U.

Further, in the above-described embodiment, the common item common to both the first data (specifically, measurement data) and the second data (specifically, device log data) is the contact status with the broadcasting media. More specifically, it was decided that the situation was TV viewing. However, the common items are not particularly limited, and may be contents other than the contact status with the broadcasting media, and may be demographic attributes such as gender and age, or may be. It may be a psychographic attribute such as interests and lifestyle.

10 Data processing device 11 Processing side server 11a CPU
11b Memory 11c Communication interface 11d Hard disk drive 11e Input device 11f Output device 12 Data provider server 12a CPU
12b Memory 12c Communication interface 12d Hard disk drive 12e Input device 12f Output device C1 First collection center C2 Second collection center M Manufacturer T First target person U Second target person

In order to achieve the above object, the data processing apparatus of the present invention stores the first data collected for the first target person randomly selected, in an amount corresponding to the number of the first target person. The first storage unit and the second data acquired for the second target person who meets the predetermined collection conditions and is larger than the first target person are obtained according to the number of the second target person. It has a second storage unit that stores minutes and a data extraction unit that extracts the second data to be used as aggregation data from the second data stored in the second storage unit. Both the first data and the second data include data indicating the contents of common items common to both, and the data extraction unit includes each of the first data and the second data. The degree of similarity of the contents of the common items with each is calculated, and the second data of the second target person for the set number of people specified based on the calculated degree of similarity is extracted as the aggregation data. It is characterized by doing.

Further, in order to solve the above-mentioned problems, in the data processing method of the present invention, the first storage unit collects the first data for the first target person randomly selected, and the first data is collected by the first target person. The second data collected by the second storage unit for the second target person, which meets the predetermined collection conditions and is larger than the first target person, is stored as much as the number of people. The amount corresponding to the number of the second target person is stored, and the computer extracts the second data to be used as the aggregation data from the second data stored in the second storage unit, and the said Both the first data and the second data include data indicating the contents of common items common to both, and the computer uses each of the first data and each of the second data. The degree of similarity of the contents of the common items is calculated between the two, and the second data of the second target person for the set number of people specified based on the calculated degree of similarity is extracted as the aggregation data. It is a feature.
According to the above data processing method, the first data which is originally non-representative can be extracted as representative data for aggregation.

Claims (9)

The first storage unit that stores the first data collected for the first randomly selected target person according to the number of the first target person, and
The second data collected for the second target person who meets the predetermined collection conditions and is larger than the first target person is stored in the amount corresponding to the number of the second target person. With the second memory
It has a data extraction unit that extracts the second data to be used as aggregation data from the second data stored in the second storage unit.
Both the first data and the second data include data indicating the contents of common items common to both.
The data extraction unit calculates the degree of similarity of the contents of the common items between each of the first data and each of the second data, and the second object specified based on the calculated degree of similarity. A data processing apparatus characterized in that the second data of a person is extracted as the aggregation data. The data processing device according to claim 1, wherein the second data is data collected when the second target person who satisfies the collection condition performs a specific action. The data according to claim 1 or 2, wherein the second target person is a person who satisfies the condition that the device used by the second target person for contacting the broadcasting media is connected to the Internet as the collection condition. Processing equipment. The data processing device according to claim 3, wherein the second data is log data transmitted by the device when the second target person contacts the broadcast media using the device. The data processing device according to claim 4, wherein the content of the common item is a contact situation with the broadcasting media. The data processing device according to claim 5, wherein the contact status with the broadcast media is a television viewing status. The data extraction unit calculates the degree of similarity for each of the combinations by changing the combination of the first subject and the second subject, and for each of the first subjects, the degree of similarity is maximized. The second target person belonging to the combination is specified in order from the combination, and when the number of the specified second target person reaches the set number of people, the second target person of the specified number of people is said to be said. The data processing apparatus according to any one of claims 1 to 6, which extracts the second data as the aggregation data. When the data extraction unit identifies the second target person for the set number of people, if a second target person is specified more than once, the data extraction unit will be used for the second target person. The data processing apparatus according to claim 7, wherein the second data is extracted as the same number of aggregation data as the number of times the second target person is specified. The first storage unit stores the first data collected for the first randomly selected target persons according to the number of the first target persons.
The second storage unit meets the predetermined collection conditions and collects the second data for the second target person, which is larger than the first target person, according to the number of the second target person. I remember
The computer extracts the second data to be used as aggregation data from the second data stored in the second storage unit.
Both the first data and the second data include data indicating the contents of common items common to both.
The computer calculates the degree of similarity of the contents of the common items between each of the first data and each of the second data, and the second subject specified based on the calculated degree of similarity. A data processing method characterized by extracting the second data as the aggregation data.