JP2023179031A - Determination device and determination method - Google Patents

Abstract

To provide a determination device and determination method capable of appropriately determining a message containing an inappropriate expression.SOLUTION: A determination device includes a management part for managing messages between requesters and providers, and a control part for determining whether or not the messages are inappropriate. The control part determines whether or not a message is inappropriate by using a prediction model constructed by a message variable relating to one or more words contained in the message and an attribute variable relating to an attribute of the message.SELECTED DRAWING: Figure 2

Description

The present invention relates to a determination device and a determination method.

In recent years, with the development of information and communication technology, various services have been provided. Technologies for detecting inappropriate expressions included in communications have also been proposed in providing various services. For example, in a service that provides advertisements, a technology has been proposed that detects inappropriate expressions included in advertisements and limits the display of advertisements containing inappropriate expressions (for example, Patent Document 1).

JP2021-26690A

By the way, one of the various services known is a service that supports requesters in receiving various information from providers by matching requesters and providers (hereinafter referred to as matching services). . For example, in a case where the requester is a job seeker (for example, a student) who wishes to work at a specified company, and the provider is a current employee or retiree of the specified company, the requester wishes to find employment at the specified company or change jobs. A case can be considered in which the provider is an agent who mediates employment or job change at a predetermined company.

In the matching service, it is assumed that two-way communication is carried out between the requester and the provider, and it is assumed that messages are sent and received several times.

Against this background, it is desirable to identify messages that contain inappropriate expressions, but if inappropriate expressions are set to a wide range, there is a possibility that the number of messages containing inappropriate expressions will be enormous. be. Therefore, restricting the transmission of messages containing inappropriate expressions impedes smooth communication, and visually inspecting messages containing inappropriate expressions increases the workload associated with visual inspection.

Therefore, the present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a determination device and a determination method that can appropriately determine messages containing inappropriate expressions. .

One aspect of the disclosure includes a management unit that manages messages between a requester and a provider, and a control unit that determines whether the message is inappropriate, and the control unit is configured to manage messages between the requester and the provider. This is a determination device that determines whether or not the message is inappropriate using a prediction model constructed from message variables related to one or more words included in the message and attribute variables related to the attributes of the message.

One aspect of the disclosure includes a step A of managing messages between a requester and a provider, and a step B of determining whether the message is inappropriate. A determination method, comprising: determining whether the message is inappropriate using a predictive model constructed by a message variable regarding one or more words included in the message and an attribute variable regarding an attribute of the message. be.

According to the present invention, it is possible to provide a determination device and a determination method that can appropriately determine messages containing inappropriate expressions.

FIG. 1 is a diagram showing a determination system 100 according to an embodiment. FIG. 2 is a diagram showing the determination device 30 according to the embodiment. FIG. 3 is a diagram showing a determination method according to the embodiment. FIG. 4 is a diagram showing a determination method according to the embodiment. FIG. 5 is a diagram for explaining learning according to modification example 1. FIG. 6 is a diagram for explaining determination according to modification example 1.

Embodiments will be described below with reference to the drawings. In addition, in the description of the following drawings, the same or similar parts are given the same or similar symbols.

However, it should be noted that the drawings are schematic and the ratio of each dimension may differ from the actual one. Therefore, specific dimensions etc. should be determined with reference to the following explanation. Furthermore, it goes without saying that the drawings may include portions with different dimensional relationships or ratios.

[Summary of disclosure]
A determination device according to an overview of the disclosure includes a management unit that manages messages between a requester and a provider, and a control unit that determines whether the message is inappropriate, and the control unit includes: , it is determined whether the message is inappropriate using a prediction model constructed by a message variable related to one or more words included in the message and an attribute variable related to an attribute of the message.

One aspect of the determination method according to the outline of disclosure includes a step A of managing a message between a requester and a provider, and a step B of determining whether the message is inappropriate. Preparation: Step B determines whether the message is inappropriate using a prediction model constructed by a message variable related to one or more words included in the message and an attribute variable related to an attribute of the message. and a step.

In the summary of the disclosure, a determination device determines whether a message is inappropriate using a prediction model constructed by a message variable related to one or more words included in the message and an attribute variable related to an attribute of the message. According to such a configuration, when assuming a matching service between requesters and providers, it is possible to identify inappropriate messages by considering message attributes, rather than simply determining based on the words contained in the message. Messages containing expressions can be appropriately judged.

Note that the requester may be a job seeker who wishes to belong to the predetermined organization, and the provider may be a current employee or retiree of the predetermined organization. The requester may be a job seeker who wishes to belong to a predetermined organization, and the provider may be an agent who mediates membership to the predetermined organization.

[Embodiment]
(judgment system)
A determination system according to an embodiment will be described below. FIG. 1 is a diagram showing a determination system 100 according to an embodiment.

As shown in FIG. 1, the determination system 100 includes a first terminal 10, a second terminal 20, and a determination device 30. The first terminal 10, the second terminal 20, and the determination device 30 are connected by a network 200. Although not particularly limited, the network 200 may be constituted by the Internet network. Network 200 may include a local area network, a mobile communication network, or a VPN (Virtual Private Network).

The determination system 100 (determination device 30) is a system (device) used in a service (hereinafter referred to as matching service) that supports a requester to receive various information from a provider by matching the requester and the provider. ).

For example, the requester may be a job seeker (eg, a student, a member of society) who wishes to belong to a predetermined organization (eg, a company, etc.) (eg, get a job or change jobs). The provider may be a member of or retired from the predetermined organization, or may be an agent who mediates membership in the predetermined organization (for example, employment or job change). In such cases, the matching service may be considered a recruitment support service.

For example, the requester may be a person who desires to acquire the predetermined information, or may be a person who provides the predetermined information. In such cases, the matching service may be considered an information support service.

In the following, a case where the matching service is a recruitment support service will be exemplified. Although not particularly limited, recruitment support services may be provided to requesters (e.g., current or retired) who have graduated from the same school (high school, university, graduate school, etc.) as the requester (e.g., student). It may also include services to be introduced. The recruitment support service may include a service that introduces the requester (for example, a working adult) to a provider (for example, an agent) who mediates employment or job change for the requester (for example, a working adult).

The first terminal 10 is a terminal used by the requester. For example, the first terminal 10 may be a personal computer, a smartphone, or a tablet terminal. The first terminal 10 transmits a message to the second terminal 20. Although not particularly limited, the message may be in the form of text data such as email or chat. However, the message may be in the form of data obtained by converting audio into text. In the following, the first terminal 10 may also be referred to as a requester.

The second terminal 20 is a terminal used by the provider. The second terminal 20 may be a personal computer, a smartphone, or a tablet terminal. The second terminal 20 transmits a message to the first terminal 10. Although not particularly limited, the message may be in the form of text data such as email or chat. However, the message may be in the form of data obtained by converting audio into text. In the following, the second terminal 20 may be referred to as a provider.

The determination device 30 determines whether the communication (message in the embodiment) between the first terminal 10 and the second terminal 20 is inappropriate. The determination device 30 may obtain the message in real time by relaying the message, or may obtain the message in real time or after the fact from a server that relays the message. Details of the determination device 30 will be described later.

(Judgment device)
A determination device according to an embodiment will be described below. FIG. 2 is a diagram showing the determination device 30 according to the embodiment. As shown in FIG. 2, the determination device 30 includes a communication section 31, a management section 32, and a control section 33.

The communication unit 31 is configured by a communication module. The communication module may be a wireless communication module that complies with standards such as IEEE802.11a/b/g/n/ac/ax, LTE, 5G, 6G, etc., and may be a wired communication module that complies with standards such as IEEE802.3. It may be.

In the embodiment, the communication unit 31 receives messages between the first terminal 10 and the second terminal 20. The communication unit 31 may receive messages in real time by relaying the messages. The communication unit 31 may receive messages in real time or after the fact from a server that relays messages.

The management unit 32 is configured with a storage medium such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive), and stores various information.

In the embodiment, the management unit 32 constitutes a management unit that manages messages between the first terminal 10 (requester) and the second terminal 20 (provider). Messages may be managed in association with combinations of requesters and providers. Messages may be managed in association with message transmission times.

The management unit 32 may manage attributes of the requester. If the requester is a student, the requester's attributes include information that identifies the requester (name, ID, etc.), the requester's occupation (for example, student, university student, graduate student, part-time worker, etc.), and the requester's It may also include affiliation (for example, affiliated school, affiliated university, department, etc.). If the requester is a member of society, the requester's attributes include information that identifies the requester (name, ID, etc.), the requester's industry or occupation (e.g., current or past industry or occupation), and the requester. affiliation (eg, current or past company).

The management unit 32 may manage attributes of the provider. If the donor is a current employee or retired member of a given organization, the donor's attributes include information that identifies the donor (name, ID, etc.), and the donor's origin (e.g., alma mater, alma mater, (Faculty, etc.) may also be included. The provider's attributes may include the provider's affiliation (for example, affiliated company, etc.). The provider's attributes may include the provider's age, the type of job the provider engages in, and the like. If the provider is an agent, information that identifies the provider (name, ID, etc.), the provider's affiliation (for example, the agent company to which the agent belongs), the industry that the provider specializes in (for example, (manufacturing industry, IT industry, etc.) or job type (for example, software engineer, legal personnel, etc.). An agent may also be referred to as a third party.

Control unit 33 may include at least one processor. At least one processor may be configured by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), one or more Integrated Circuits, one or more Discrete Circuits, or a combination thereof. .

In the embodiment, the control unit 33 constitutes a control unit that determines whether a message is inappropriate. In other words, the control unit 33 determines messages that include inappropriate expressions. The inappropriate expression may be a combination of one or more inappropriate words as described below.

Specifically, the control unit 33 determines whether the message is inappropriate using a predictive model constructed from message variables related to one or more words included in the message and attribute variables related to attributes of the message. . Specifically, the control unit 33 uses a prediction model to calculate the probability that a message is inappropriate, and determines that a message for which the calculated probability exceeds a threshold is inappropriate. Note that details of the prediction model will be described later.

First, the message variables include whether or not inappropriate words appear in messages managed (stored) in the management unit 32, the number of times inappropriate words appear in messages, and the number of times inappropriate words appear in messages. may be defined by one or more data selected from the order.

Here, the inappropriate word may be a word that is determined to be inappropriate in light of the purpose of the matching service. Inappropriate words include, but are not limited to, illegal words, anti-social words, words intended to slander or slander, religious words, words intended for inappropriate encounters, and matching words. It may also include words intended for communication by methods other than services. Inappropriate encounters may include attempting to contact the requester or provider without using the matching service, attempting to contact the requester or provider in violation of the terms of the matching service, and the like. For example, an inappropriate encounter may include an agent mediating membership in a predetermined organization attempting to contact the requester without using a matching service.

Inappropriate words may be defined by regular expressions (patterns of strings). For example, an inappropriate word may be defined as a word alone or in combination with other words.

Inappropriate words may be manually registered by a matching service provider (hereinafter referred to as an operator, etc.). Inappropriate words may be managed by the management unit 32.

Second, the message variable may be a variable that is set for n (n is an integer of 2 or more) messages between the requester and the provider. Specifically, a case is assumed in which message communication is executed n times between the same requester and the same provider. Assuming such a case, a message variable may be set for each of the n messages.

Here, a different weighting value may be applied to the message variable for each of the n messages. For example, the weighted value of the message variable for the n-th message may be higher than the weighted value of the message variable for the (n-1)th message. The weight value may be a value for adjusting the importance of the message variable.

The attribute variable may be defined by one or more data selected from requester attributes, provider attributes, message sending time, message sending frequency, and message sending frequency.

As described above, the attributes of the requester may include information for identifying the requester, the requester's occupation, the requester's affiliation, and the like. The requester's attributes can be specified by information managed by the management unit 32.

As described above, the attributes of the provider may include information that identifies the provider, and the attributes of the provider may include the origin of the provider, the affiliation of the provider, the age of the provider, the type of job the provider engages in, etc. . The attributes of the provider can be specified by information managed by the management unit 32.

The sending time of the message may be the time when the message was sent from the requester to the provider, or the time when the message was sent from the provider to the requester. The message transmission time can be specified by information managed by the management unit 32.

The number of times a message has been sent may be the number of times a message has been sent from a requester to a provider, or the number of times a message has been sent from a provider to a requester. The number of messages sent is the number of times messages are sent between the requester and the provider (i.e., the total number of times the requester sends messages to the provider and the number of times the provider sends messages to the requester). You can. The number of message transmissions can be specified by information managed by the management unit 32.

The number of times a message is sent may be the frequency at which a requester sends a message to a provider in a certain period of time, or may be the number of times a message is sent from a provider to a requester in a certain period of time. The number of messages sent may be the number of times messages are sent between the requester and the provider (i.e., the frequency at which messages are sent from the requester to the provider and from the provider to the requester in a certain period of time). . For example, the fixed period may be arbitrarily set, such as one day or one week. The message transmission frequency can be specified by information managed by the management unit 32.

(prediction model)
A prediction model according to an embodiment will be described below. When the message variable is represented by m and the attribute variable is represented by x, the probability that a message is inappropriate may be expressed as p=F(m,x). p may be a probability that takes a value between 0 and 1. In such a case, F is an example of a prediction model (prediction function).

For example, consider a case where it is determined whether or not the nth message is inappropriate. In such a case, the message variable m and the attribute variable x may be expressed as follows.

First, the message variable m may include the following variables:

Message variables for the nth message: m _n _1, m _n _2, m _n _3,…
Message variables for the n-1st message: m _n-1 _1, m _n-1 _2, m _n-1 _3,…
Message variables for the n-2nd message: m _n-2 _1, m _n-2 _2, m _n-2 _3,…
…

“m _n _1, m _n _2, m _n _3,…”, “m _n-1 _1, m _n-1 _2, m _n-1 _3,…”, “m _n-2 _1, m _n-2 _2 , m _n-2 _3,…", as described above, determines whether inappropriate words appear in a message, the number of times inappropriate words appear in a message, and the order in which inappropriate words appear in a message. It may also include. m _n _1, m _n _2, m _n _3,…, m _n-1 _1, m _n-1 _2, m _n-1 _3,…, m _n-2 _1, m _n-2 _2, m _n-2 Different weighting values may be applied to each of _3,.... As mentioned above, message variables may be given different weighting values for each of the n messages.

Second, the attribute variable x may include the following variables.

Requester attribute variables: xs_1, xs_2, …
Provider attribute variables: xo_1, xo_2, …

As described above, "xs_1, xs_2, ..." may include information identifying the requester, the requester's occupation, the requester's affiliation, and the like. As mentioned above, "xo_1, xo_2, ..." is the information that identifies the donor, and the attributes of the donor include the donor's origin, the donor's affiliation, the donor's age, the type of job the donor engages in, etc. May include. Different weighting values may be applied to each of xs_1, xs_2, .... Different weighting values may be applied to each of xo_1, xo_2, ....

Under such a premise, the determination device 30 determines that the n-th message is inappropriate when p=F(m,x) exceeds the threshold p_0 (=0 to 1). On the other hand, the determination device 30 determines that the n-th message is not inappropriate if p=F(m,x) does not exceed the threshold p_0 (=0 to 1). When the determination device 30 determines that the n-th message is inappropriate, it may notify an operator or the like that the message is inappropriate.

Here, the threshold p_0 may be set to allow a message to be incorrectly determined to be inappropriate. That is, the threshold p_0 may be set so that an erroneous determination that the message is not inappropriate is unlikely to occur, and an erroneous determination that the message is determined to be inappropriate is allowed. The threshold p_0 may be set based on the history regarding whether messages (past messages) managed (accumulated) by the management unit 32 are inappropriate. The threshold p_0 may be set by an operator or the like.

(Judgment method)
The determination method according to the embodiment will be described below. 3 and 4 are diagrams illustrating the determination method according to the embodiment.

First, construction of a prediction model will be explained with reference to FIG.

As shown in FIG. 3, messages are exchanged between the first terminal 10 and the second terminal 20 in steps S10 and S11. Step S10 and step S11 may be repeated.

In step S12, the determination device 30 manages (accumulates) messages between the first terminal 10 and the second terminal 20. The determining device 30 may receive messages in real time by relaying the messages. The determination device 30 may receive messages in real time or after the fact from a server that relays messages.

In step S13, the determination device 30 constructs a predictive model and stores the constructed predictive model. The prediction model may be constructed based on the messages (past messages) managed (accumulated) in step S12.

Here, the operator or the like may refer to the messages (past messages) managed (accumulated) in step S12 and manually specify the contents shown below. An operator or the like may manually identify whether each message is inappropriate. An operator or the like may manually identify inappropriate words. An operator or the like may specify the types of message variables that define the predictive model. An operator or the like may specify the type of attribute variable that defines the prediction model. An operator or the like may specify weighting values to apply to message variables. An operator or the like may specify weighting values to be applied to attribute variables. An operator or the like may specify a threshold for determining whether each message is inappropriate.

As described above, it should be noted that an operator or the like can construct a prediction model on the premise that the management unit 32 can manage sufficient messages between the requester and the provider.

Second, determination of whether a message is inappropriate will be explained with reference to FIG.

As shown in FIG. 4, messages are exchanged between the first terminal 10 and the second terminal 20 in steps S20 to S23. Message transmission and reception may be repeated between the first terminal 10 and the second terminal 20.

In step S30, the determination device 30 determines whether the message is inappropriate using the prediction model constructed in step S13. Specifically, the determination device 30 calculates the probability that a message is inappropriate, and determines that a message for which the calculated probability exceeds a threshold value is inappropriate.

Note that the target message for determining whether or not it is inappropriate may be a message sent from the first terminal 10 to the second terminal 20 in step S22, and a message sent from the second terminal 20 to the first terminal in step S23. It may be a message sent to 10.

(action and effect)
In the embodiment, the determination device 30 determines whether a message is inappropriate using a prediction model constructed by message variables related to one or more words included in the message and attribute variables related to attributes of the message. According to such a configuration, when assuming a matching service between requesters and providers, inappropriate judgments can be made by considering the attributes of the message, rather than simply making a judgment based on the words contained in the message. Messages containing expressions can be appropriately judged.

[Change example 1]
Modification example 1 of the embodiment will be described below. In the following, differences from the embodiment will be mainly explained.

In the embodiment, a case where the prediction model is a prediction function F constructed by an operator or the like is exemplified. In contrast, in modification example 1, the prediction model is constructed by learning about messages (past messages) managed by the management unit 32.

Here, it is assumed that whether or not the message managed by the management unit 32 is inappropriate is specified by an operator or the like. That is, it is assumed that the message has already been labeled as inappropriate or not.

In modification example 1, the predictive model is constructed by learning the learning data, using the message variables, attribute variables, and whether or not the message is inappropriate as learning data. Since the messages have already been labeled as inappropriate or not, the learning data may be considered to be training data.

First, as shown in FIG. 5, the learning unit 35A learns the correlation between message variables, attribute variables, and whether or not a message is inappropriate. The learning section 35A may be provided in the control section 33 described above. The message variables and attribute variables are specified by information managed by the management unit 32. Whether or not a message is inappropriate is determined by an operator or the like. The learning unit 35A constructs a prediction model for calculating the probability that a message is inappropriate as a learning result.

As a learning method, deep learning represented by AI (Artificial Intelligence) may be used. For example, as a learning method, logistic regression may be used, or Gradient Boosting Decision Tree may be used.

Second, as shown in FIG. 6, the determining unit 35B inputs the message variables and attribute variables of the target message to be determined whether or not it is inappropriate into the prediction model, and calculates the probability that the target message is inappropriate. It may be calculated. The determining unit 35B may determine that a message whose calculated probability exceeds a threshold is inappropriate. The determination unit 35B may be provided in the control unit 33 described above.

Modification Example 1 illustrates a case where the prediction model is a model for calculating the probability that a message is inappropriate. However, modification example 1 is not limited to this. The predictive model may be a model for identifying whether a message is inappropriate.

[Change example 2]
Modification example 2 of the embodiment will be described below. In the following, differences from the embodiment will be mainly explained.

In modification example 2, a case will be mainly described in which the determination device 30 determines whether or not a message regarding an agent who mediates membership in a predetermined organization is inappropriate.

For example, the message variables may include a history of scout emails sent to the requester (number of times, frequency, time, etc.), and a history of reply emails to scout emails from the requester (number of times, frequency, time, etc.).

For example, the attribute variable may include membership type, such as whether the requester is a paid member who pays a fee for the matching service or whether the requester is a free member who does not pay a fee for the matching service.

Furthermore, in modification example 2, other variables may be used in addition to message variables and attribute variables as variables for constructing a predictive model. Other variables may include variables related to matching services (hereinafter referred to as service variables).

For example, service variables include the history in which the requester applied for membership in a given organization (application history), the history in which the requester had an interview or interview with a person in charge of the given organization (interview/interview history), and the requestor's attention. Registration history of the organization (favorite registration history), update history of the requester's work history (resume) (resume update history), history of the requester viewing job postings of the specified organization (browsing history), job postings of the specified organization It may also include a history of searches by the requester (search history).

For example, the service variables may include whether or not the requester has been selected in the predetermined organization, the progress history of the requester's selection in the predetermined organization, the history of changes to job postings in the predetermined organization, the hiring history of the requester in the predetermined organization, and the like.

In such a case, the determination device 30 may have a database that manages information for identifying service variables, and may be able to communicate with the database that manages information for identifying service variables. good. The matching service may include services such as management of job postings of employers, management of work history of job seekers, and management of selection of job seekers. The above-mentioned requester may also be read as a job applicant. The above-mentioned predetermined organization or provider may be replaced with the adopter.

[Other embodiments]
Although the present invention has been described with reference to the embodiments described above, the statements and drawings that form part of this disclosure should not be understood as limiting the present invention. Various alternative embodiments, implementations, and operational techniques will be apparent to those skilled in the art from this disclosure.

In the above disclosure, inappropriate words are manually registered by an operator or the like. However, the above disclosure is not limited thereto. On the premise that the operator etc. has identified whether or not the message managed by the management unit 32 is inappropriate, inappropriate words are automatically registered by analyzing the message identified as inappropriate. may be done. For example, the analysis may be performed using a technique such as TF-IDF (Term Frequency-Inverse Document Frequency).

Although not specifically mentioned in the above disclosure, the messages between the requester and the provider may be one-to-one messages, one-to-many messages, or many-to-many messages. There may be.

In the above-mentioned disclosure, the case where the management unit 32 is provided in the determination device 30 has been exemplified. However, the above disclosure is not limited thereto. The management unit 32 may be included separately from the determination device 30 in the same external server capable of communicating with the determination device 30, or may be included in separate external servers capable of communicating with the determination device 30.

Although not specifically mentioned in the above disclosure, a program that causes a computer to execute each process performed by the determination device 30 may be provided. Moreover, the program may be recorded on a computer-readable medium. Computer-readable media allow programs to be installed on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or a DVD-ROM, for example.

Alternatively, a chip may be provided that includes a memory that stores programs for executing each process performed by the determination device 30 and a processor that executes the programs stored in the memory.

10...first terminal, 20...second terminal, 30...judgment device, 31...communication department, 32...management department, 33...control unit, 40...third party, 100...judgment system, 200...network

For example, the attribute variables may include a history of scout emails sent to the requester (number of times, frequency, time, etc.), and a history of reply emails to scout emails from the requester (number of times, frequency, time, etc.).

Claims (6)

a management unit that manages messages between the requester and the provider;
a control unit that determines whether the message is inappropriate;
The control unit determines whether or not the message is inappropriate using a prediction model constructed by a message variable related to one or more words included in the message and an attribute variable related to an attribute of the message. Device. The message variable is one or more selected from the following: whether an inappropriate word appears in the message, the number of times the inappropriate word appears in the message, and the order in which the inappropriate word appears in the message. The determination device according to claim 1, defined by data of. The determination device according to claim 1, wherein the message variable is a variable set for n messages between the requester and the provider. The attribute variable is defined by one or more data selected from the attributes of the requester, the attributes of the provider, the time of sending the message, the number of times the message is sent, and the frequency of sending the message. The determination device according to item 1. The determination device according to claim 1, wherein the prediction model is constructed by learning the learning data, using the message variable, the attribute variable, and whether or not the message is inappropriate as learning data. step A of managing messages between the requester and the provider;
Step B of determining whether the message is inappropriate;
The step B includes:
determining whether or not the message is inappropriate using a predictive model constructed by a message variable regarding one or more words included in the message and an attribute variable regarding an attribute of the message. Method.