JP2024037111A - Information processing program, information processing method, and information processing device - Google Patents

Abstract

[Problem] To accurately evaluate the occurrence of criminal acts against a target person. An information processing device (100) specifies, in a target period, sections that match each of one or more voice patterns, based on audio data (101) indicating a conversation of a target person during the target period. For each of the identified sections, the information processing device 100 acquires biometric data 102 of the target person in the section. For each identified section, the information processing device 100 generates noise corresponding to one of the voice patterns that matches the section among the one or more voice patterns, based on the acquired biometric data 102 of the target person in the section. Perform the removal process. The information processing device 100 detects a criminal act against the target person based on the target person's biometric data 102 after performing noise removal processing corresponding to one of the voice patterns that matches the specified interval in each of the identified intervals. Assess the situation of occurrence. [Selection diagram] Figure 1

Description

The present invention relates to an information processing program, an information processing method, and an information processing apparatus.

Traditionally, fraud can occur over the telephone. For example, fraud damage may occur due to a fraudulent act called "Ore Ore Fraud" in which a fraudster pretends to be a close relative of a targeted person over the phone and attempts to defraud the targeted person of money. On the other hand, a possible technique is to analyze the voice data of the target person during a phone call and attempt to detect the occurrence of fraud against the target person.

Prior art includes, for example, determining the stress level of a caller based on the caller's biometric data, which indicates the degree to which the caller's mental state differs from his or her normal mental state, such as the caller's level of tension. There is something to do.

Japanese Patent Application Publication No. 2018-107769

However, the conventional technology has a problem in that it is difficult to accurately detect the occurrence of a criminal act against a target person. For example, if a target person becomes the target of fraudulent activity over the telephone, the target person's speaking time or number of utterances tends to decrease. However, it is difficult to accurately detect the occurrence of fraud against a target person.

In one aspect, the present invention aims to accurately evaluate the occurrence of a criminal act against a target person.

According to one embodiment, audio data indicating a conversation of a target person in a certain period is acquired, and based on the acquired audio data, one or more set audio patterns are matched in the period. Identifying sections, and for each of the identified sections, removing noise corresponding to any one of the one or more voice patterns that matches the section, with respect to the biometric data of the target person in the section. A crime against the target person based on the biometric data of the target person after performing processing and performing noise removal processing corresponding to one of the voice patterns that matched the interval in each of the identified intervals. An information processing program, an information processing method, and an information processing device for evaluating the occurrence of an act are proposed.

According to one aspect, it becomes possible to accurately evaluate the occurrence of a criminal act against a target person.

FIG. 1 is an explanatory diagram showing an example of an information processing method according to an embodiment. FIG. 2 is an explanatory diagram showing an example of the information processing system 200. FIG. 3 is an explanatory diagram showing a first application example of the information processing system 200. FIG. 4 is an explanatory diagram showing a second application example of the information processing system 200. FIG. 5 is a block diagram showing an example of the hardware configuration of the information processing device 100. FIG. 6 is an explanatory diagram showing an example of the stored contents of the filter information management table 600. FIG. 7 is an explanatory diagram showing an example of the stored contents of the emotional pattern information management table 700. FIG. 8 is an explanatory diagram showing an example of the stored contents of the biometric pattern information management table 800. FIG. 9 is a block diagram showing an example of the hardware configuration of the audio data acquisition device 201. FIG. 10 is a block diagram showing an example of the functional configuration of the information processing device 100. FIG. 11 is an explanatory diagram (part 1) showing an example of the operation of the information processing device 100. FIG. 12 is an explanatory diagram (part 2) showing an example of the operation of the information processing device 100. FIG. 13 is an explanatory diagram (part 3) showing an example of the operation of the information processing device 100. FIG. 14 is a flowchart illustrating an example of the overall processing procedure.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of an information processing program, an information processing method, and an information processing apparatus according to the present invention will be described in detail below with reference to the drawings.

(An example of an information processing method according to an embodiment)
FIG. 1 is an explanatory diagram showing an example of an information processing method according to an embodiment. The information processing device 100 is a computer for evaluating the occurrence of criminal acts.

The information processing device 100 is, for example, a server or a PC (Personal Computer). The information processing device 100 may be, for example, a smartphone or a smart speaker.

The criminal act is, for example, a fraudulent act called special fraud. Specifically, the criminal act is an act of fraud over the telephone. For example, there is a fraudulent act called "Ore Ore Fraud" in which a fraudster pretends to be a close relative of a targeted person over the telephone and attempts to defraud the targeted person of money.

In addition, for example, a scammer pretends to be an employee of a local government or tax office, suggests to a targeted person that he/she will receive overpayments for medical expenses, insurance premiums, etc., and attempts to defraud the targeted person of money. There is a fraudulent act called fraud. For example, there is a fraudulent act called theatrical fraud in which multiple fraudsters pretend to be a close family member, a police officer, a lawyer, etc. of the targeted person and attempt to defraud the targeted person of money. .

Therefore, there is a need for technology that prevents fraud damage caused by fraudulent acts. For example, the first step is to detect the occurrence of fraud against a target person and prevent damage caused by fraud by analyzing the voice data of the target person's phone call and evaluating the occurrence of fraud. possible technologies. Specifically, it is conceivable to analyze the voice data of the target person's phone call during the call period from the start of the call to the end of the call to evaluate the occurrence of fraud.

However, this first technique has a problem in that it is difficult to accurately detect the occurrence of fraud against a target person. For example, if a target person becomes the target of fraudulent activity over the telephone, the target person's speaking time or number of times he or she speaks tends to decrease. For this reason, it is difficult to accurately detect the occurrence of fraud against a target person even if the voice data of the target person's phone call is analyzed. For example, of the voice data of a target person's phone call, partial data related to sections with relatively short utterance times, such as sections where the target person chimed in, may be used as noise when evaluating the occurrence of fraud against the target person. It is possible that

In addition, for example, by analyzing the target person's biometric data during a call and evaluating the occurrence of fraud, the system can detect the occurrence of fraud against the target person and try to prevent fraud damage due to fraud. A second technique is possible. Specifically, it is conceivable to analyze the biometric data of the target person during the call during the call period from the start of the call to the end of the call to evaluate the occurrence of fraud.

Even with this second technique, it is difficult to accurately detect the occurrence of fraud against a target person. For example, if the target person's biometric data during a call contains noise, it is difficult to accurately detect the occurrence of fraud against the target person even if the target person's biometric data during the call is analyzed. There is.

Specifically, due to the influence of the breathing cycle, the biometric data of a target person during a call may include noise whose size is not constant but fluctuates. Further, specifically, while the target person is speaking, noise included in the biometric data of the target person during the call tends to increase due to body movements of the target person. Specifically, depending on the flow of the conversation of the target person, the target person may perform various actions or the size of the target person's body movements may change, causing the target person's conversation to Biometric data may contain noise whose size is not constant but fluctuates. For this reason, even if the biometric data of the target person during a call is analyzed, it is difficult to accurately detect the occurrence of fraud against the target person. For example, even if an attempt is made to remove noise from biometric data of a target person during a phone call using a uniform standard, it may not be possible to appropriately remove the noise.

Therefore, in this embodiment, an information processing method that can accurately evaluate the occurrence of a criminal act against a target person will be described.

In FIG. 1, (1-1) the information processing device 100 acquires audio data 101 indicating a conversation of a target person during a target period. The target person is a person who is desired to be protected from criminal activity. The criminal act is, for example, fraud. Audio data 101 indicates a conversation between a target person and another person via telephone. The other person may be a criminal, for example. A criminal is, for example, a fraudster. The target period is, for example, a period during which the target person is having a conversation. Specifically, the target period is the call period from the start of the call to the end of the call.

The information processing device 100 has a microphone, for example, and acquires audio data 101 by recording a conversation of a target person using the microphone. The information processing device 100 may obtain the audio data 101 by receiving it from another computer, for example. The other computer is, for example, a smartphone or a smart speaker.

Specifically, the information processing device 100 continues to record the conversation of the target person using a microphone while the target person is talking, and records the conversation of the target person during the period from the start of the call to the end of the call. Acquire audio data 101 indicating. Specifically, the information processing device 100 continues to record the conversation of the target person using a microphone during the conversation of the target person, and records the conversation of the target person a certain period of time before the current time at each predetermined timing during the conversation of the target person. Audio data 101 indicating the conversation of the target person during the period up to this point may be acquired. The predetermined timing is, for example, timing at regular intervals.

(1-2) The information processing device 100 stores one or more voice patterns. The voice pattern indicates, for example, characteristics of conversational voice indicated by voice data corresponding to a certain utterance attribute. The utterance attributes include, for example, "silence" indicating that the target person has been silent, and "shuzuchi" indicating that the target person has agreed. The utterance attribute may be, for example, "listening" indicating that the target person has listened to another person's utterance. The utterance attribute may be, for example, "utterance" indicating that the target person has uttered the utterance.

The voice pattern indicates a reference pattern that makes it possible to identify the utterance attribute corresponding to a certain section by comparing it with the voice or utterance content of a certain section, for example. Specifically, the voice pattern indicates a keyword corresponding to a certain utterance attribute. Specifically, the voice pattern may indicate the tone, volume, or wavelength of voice corresponding to a certain speech attribute.

Specifically, the information processing device 100 stores a voice pattern corresponding to "silence", a voice pattern corresponding to "agoushu", and the like. Specifically, the information processing device 100 may store a voice pattern corresponding to "utterance". The voice characteristic may indicate that the voice pattern corresponding to "speech" does not match the voice pattern corresponding to "silence" and the voice pattern corresponding to "speech".

Based on the acquired audio data 101, the information processing device 100 identifies sections of the target period that match each of one or more audio patterns. For example, based on the audio data 101, the information processing device 100 classifies and specifies a section in which a voice pattern corresponding to "silence" appears in the target period as a silence section. For example, based on the audio data 101, the information processing device 100 classifies and specifies a section in which a voice pattern corresponding to "Aitsuchi" appears in the target period as an Aitsuchi section. The silent section and the intermittent section are part of the listening section.

For example, based on the audio data 101, the information processing device 100 classifies and specifies a section in which a voice pattern corresponding to "utterance" appears in the target period as a speech section. Specifically, based on the audio data 101, the information processing device 100 may classify and specify the remaining sections other than the silent section and the intermittent section as speech sections in the target period.

(1-3) For each specified section, the information processing device 100 acquires the biometric data 102 of the target person in the section. The information processing device 100 acquires the biometric data 102 of the target person for, for example, the silent section, the cooperative section, and the utterance section.

Specifically, the information processing device 100 has a biosensor, and acquires biometric data 102 by measuring characteristic values such as heartbeat, pulse, body temperature, or perspiration of the target person using the biosensor. do. Specifically, the information processing device 100 may acquire the biometric data 102 by receiving it from another computer. The other computer is, for example, a smartphone or a wearable terminal.

More specifically, the information processing device 100 continues to measure the characteristic values of the target person using a biosensor during the target person's call, and measures the target person's characteristic values during the period from the start of the call to the end of the call. Obtain overall biometric data that shows changes in a person's characteristic values over time. More specifically, the information processing device 100 extracts the biometric data 102 of the target person in the silent section, the intermittent section, and the utterance section from among the entire acquired biometric data. Data 102 is acquired.

More specifically, the information processing device 100 continues to measure the characteristic values of the target person using the biosensor while the target person is on the phone. More specifically, the information processing device 100 calculates, at each predetermined timing during the call of the target person, the overall Biometric data may also be obtained. The predetermined timing is, for example, timing at regular intervals. More specifically, the information processing device 100 extracts the biometric data 102 of the target person in the silent section, the intermittent section, and the utterance section from among the entire acquired biometric data. Data 102 is acquired.

(1-4) For each of the identified sections, the information processing device 100 selects one of the one or more voice patterns that matches the section, based on the acquired biometric data 102 of the target person in the section. Perform noise removal processing corresponding to the pattern. For example, the information processing device 100 responds to the speech tendency or movement tendency based on the biometric data 102 of the target person in the specified section, based on the speech tendency or movement tendency that corresponds to the voice pattern that matches the specified section. Remove noise of any size and type.

The movement tendency indicates, for example, that when a person is silent, the movement is relatively small. The movement tendency indicates, for example, that the movement when a person is in agreement tends to be larger than when the person is silent. The movement tendency indicates, for example, that the movement when a person speaks is relatively large, and that the movement when a person speaks is likely to be large compared to when a person chimes in.

(1-5) The information processing device 100 performs the target person's biometric data 102 after performing noise removal processing corresponding to one of the voice patterns that matches the specified interval in each of the identified intervals. Assess the occurrence of criminal acts against persons.

For example, the information processing device 100 performs a process on the target person based on the biometric data 102 of the target person after performing noise removal processing corresponding to any voice pattern that matches the specified interval in each of the identified intervals. Generate features.

The feature amount is, for example, a biological feature amount. Specifically, the feature amount is one or more biometric feature values, a statistical value of the biometric feature value, an amount of change in the biometric feature value, or the like. The biological characteristic value is, for example, a characteristic value related to heartbeat, pulse, body temperature, sweating, or the like. Specifically, the biological characteristic value is heart rate. The statistical value is, for example, an average value, median value, mode, maximum value, or minimum value. Thereby, the information processing device 100 can obtain a feature value that serves as an index value for evaluating the occurrence of a criminal act against the target person.

For example, the information processing device 100 evaluates the occurrence of a criminal act against a target person based on the generated feature amount. Specifically, the information processing apparatus 100 compares the generated feature amount with a threshold value if the generated feature amount is the amount of change in the biological feature value for each specified section. For example, the threshold value is set in advance by the user. Specifically, the information processing device 100 evaluates that a criminal act against the target person has occurred if the generated feature amount is equal to or greater than the threshold value. Specifically, the information processing device 100 evaluates that a criminal act against the target person has not occurred if the generated feature amount is less than the threshold value.

Specifically, if the generated feature amount for each identified section is one or more biometric feature values, the information processing device 100 identifies the target person based on the change pattern of the feature amount in the section. The occurrence of criminal acts may be evaluated. Specifically, if the generated feature amount is a statistical value of a biological feature value for each specified section, the information processing device 100 calculates the statistical value of the feature amount based on the change pattern of the feature amount across the plurality of sections. The occurrence of criminal acts against the target person may be evaluated.

Thereby, the information processing device 100 can accurately evaluate the occurrence of fraud against the target person. Therefore, the information processing apparatus 100 can accurately detect the occurrence of fraud against a target person. The information processing device 100 can easily prevent fraud damage caused by fraud against a target person.

For example, for each identified section, the information processing device 100 determines appropriate information from the biometric data 102 of the target person in the section, depending on the speech tendency or movement tendency that corresponds to one of the voice patterns that matches the section. noise can be removed. Therefore, the information processing device 100 can accurately evaluate the occurrence of fraud against the target person based on the biometric data 102 from which noise has been appropriately removed.

Although the case where the information processing device 100 operates independently has been described here, the present invention is not limited to this. For example, the information processing device 100 may cooperate with another computer. For example, a plurality of computers may implement the functions of the information processing device 100. Specifically, the functions of the information processing device 100 may be implemented on the cloud.

(Example of information processing system 200)
Next, an example of an information processing system 200 to which the information processing apparatus 100 shown in FIG. 1 is applied will be described using FIG. 2.

FIG. 2 is an explanatory diagram showing an example of the information processing system 200. In FIG. 2, an information processing system 200 includes an information processing device 100, an audio data acquisition device 201, a biological data acquisition device 202, and an alert output device 203.

In the information processing system 200, the information processing device 100 and the audio data acquisition device 201 are connected via a wired or wireless network 210. The network 210 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or the like.

Further, in the information processing system 200, the information processing device 100 and the biometric data acquisition device 202 are connected via a wired or wireless network 210. Further, in the information processing system 200, the information processing device 100 and the alert output device 203 are connected via a wired or wireless network 210.

The information processing device 100 is a computer for evaluating the occurrence of criminal acts against a target person. The target person is a person who is desired to be protected from criminal activity. The criminal act is, for example, fraud. The occurrence status is, for example, whether or not an occurrence occurs.

The information processing device 100 acquires voice data of a target person during a target period. The audio data represents a conversation between the person of interest and another person via telephone. The other person may be a criminal, for example. A criminal is, for example, a fraudster. The information processing device 100 acquires, for example, voice data of a target person during a target period by receiving from the voice data acquisition device 201.

Specifically, the information processing device 100 acquires voice data of a target person during a target period of a certain amount of time in the past by receiving from the voice data acquisition device 201 at regular intervals. Specifically, the information processing device 100 acquires the voice data of the target person during the target period from the start of the call to the end of the call by receiving from the voice data acquisition device 201 when the target person's call ends. It's okay.

The information processing device 100 acquires biometric data of a target person during a target period. Biometric data indicates changes over time in biometric characteristic values of a target person. The biological characteristic values include, for example, heart rate, pulse rate, body temperature, or amount of perspiration. For example, the information processing device 100 acquires biometric data of a target person during a target period by receiving the biometric data from the biometric data acquisition device 202 .

Specifically, the information processing device 100 acquires biometric data of a target person during a target period for a predetermined amount of time in the past by receiving from the biometric data acquisition device 202 at regular intervals. Specifically, the information processing device 100 acquires the biometric data of the target person during the target period from the start of the call to the end of the call by receiving the biometric data from the biometric data acquisition device 202 at the end of the call with the target person. It's okay.

The information processing device 100 evaluates the occurrence of fraud against the target person based on the acquired voice data and the acquired biometric data. For example, the information processing device 100 classifies the target period into a speaking period and a listening period based on the acquired audio data. Specifically, the information processing device 100 identifies multiple speakers in the target period based on the acquired audio data. Specifically, based on the acquired audio data, the information processing device 100 calculates, for each speaker, the section in which the speaker speaks, the speaker's voice in the section, and the speaker in the section. Identify the content of the utterance.

Here, the information processing device 100 specifically stores voice patterns and utterance attributes in association with each other. The voice pattern is, for example, a reference pattern that makes it possible to identify the utterance attribute corresponding to the section by comparing it with the target person's voice or utterance content. The utterance attributes include, for example, "silence" indicating that the target person has been silent, and "shuzuchi" indicating that the target person has agreed. The utterance attribute may be, for example, "utterance" indicating that the target person has uttered the utterance. More specifically, the information processing device 100 stores a filter information management table 600, which will be described later in FIG.

Specifically, based on the identified speaker, the information processing device 100 compares the voice or utterance content of the identified target person with the stored voice pattern, thereby determining whether or not there is "silence" within the target period. ” or a section having a speech attribute such as “Aitsuchi”. Specifically, based on the identified speaker, the information processing device 100 compares the voice or utterance content of the identified target person with the stored voice pattern, thereby determining whether or not the ``utterance'' is detected during the target period. ” may be specified.

Specifically, the information processing device 100 classifies, as a listening section, a section in the target period that has the speech attribute of "silence" and in which the only speakers are people different from the target person. . For example, the information processing device 100 classifies a section having the utterance attribute of "Aitsuchi" as a listening section. Specifically, the information processing device 100 classifies, as speech sections, sections in which the speaker includes at least the target person and does not have the speech attribute of "Aitsuchi" in the target period. More specifically, the information processing device 100 may classify, in the target period, a section having the speech attribute of "utterance" into a speech section. Specifically, the information processing device 100 may combine two or more consecutive listening sections. Specifically, the information processing device 100 may combine two or more consecutive speech sections.

The information processing device 100 extracts and processes the biometric data of the target person in the listening section from among the biometric data of the target person in the target period. Here, the information processing device 100 stores, for example, utterance attributes and filters for processing biometric data in association with each other. Specifically, the information processing apparatus 100 stores a filter information management table 600, which will be described later in FIG.

The information processing device 100 removes noise from the biometric data of the target person in the segment having the silent speech attribute among the classified listening sections by applying a filter corresponding to the silent speech attribute. remove. The information processing device 100 removes noise from the biometric data of the target person in the segment having the utterance attribute of ``agistetsu'' among the classified listening sections by applying a filter corresponding to the utterance attribute of ``agistetsu''. remove.

The information processing device 100 calculates a first feature amount for each listening section based on the biological data of the target person after removing noise in the listening section. The first feature amount is, for example, a statistical value of heart rate in the listening section. The first feature amount may be the amount of change in heart rate during the listening section.

The information processing device 100 calculates a second feature amount for each speech section based on the voice data of the target person in the speech section. The second feature amount is, for example, a feature value of the emotion of the target person in the utterance section. The emotion feature value indicates, for example, whether the emotion of the target person is negative or positive.

The emotion characteristic value may indicate, for example, that the emotion of the target person is joy, anger, sadness, regret, surprise, motivation, or reluctance. The characteristic value of emotion may be, for example, the degree of stress. The second feature amount may be, for example, the amount of change in the feature value of the emotion of the target person during the utterance period. Specifically, the second feature amount may be the amount of change in the stress level of the target person during the utterance period.

The information processing device 100 evaluates the occurrence of fraud against the target person based on the calculated first feature amount and the calculated second feature amount. Here, the information processing device 100 stores, for example, the change pattern of the first feature amount and the solution of the occurrence situation of the fraudulent act in association with each other. Specifically, the information processing device 100 stores an emotion pattern information management table 700, which will be described later in FIG. The information processing device 100 evaluates the occurrence of fraud against the target person, for example, by specifying a solution to the occurrence of fraud that corresponds to the change pattern of the calculated first feature amount.

For example, if the calculated first feature amount is the amount of change in heart rate, the information processing device 100 evaluates the occurrence of fraud against the target person by comparing the first feature amount and the threshold value. You may. For example, the threshold value is set in advance by the user. Specifically, the information processing device 100 evaluates that a fraudulent act against the target person has occurred if the first feature amount is equal to or greater than the threshold value. Specifically, the information processing device 100 evaluates that fraud against the target person has not occurred if the first feature amount is less than the threshold value.

Further, the information processing device 100 stores, for example, the change pattern of the second feature amount and the solution of the occurrence situation of fraud in association with each other. Specifically, the information processing apparatus 100 stores a biometric pattern information management table 800, which will be described later in FIG. The information processing device 100 evaluates the occurrence situation of fraud against the target person, for example, by specifying a solution to the situation of occurrence of fraud corresponding to the calculated change pattern of the second feature amount.

For example, if the calculated second feature amount is the amount of change in stress level, the information processing device 100 evaluates the occurrence of fraud against the target person by comparing the second feature amount with a threshold value. You may. For example, the threshold value is set in advance by the user. Specifically, the information processing device 100 evaluates that a fraudulent act against the target person has occurred if the second feature amount is equal to or greater than the threshold value. Specifically, if the second feature amount is less than the threshold value, the information processing device 100 evaluates that no fraud has occurred against the target person.

If it is determined that a fraudulent act has occurred against the target person as a result of evaluating the occurrence status of the fraudulent act against the target person, the information processing device 100 issues a notification indicating that a fraudulent act has occurred against the target person as an alert. It is transmitted to the output device 203. The information processing device 100 is, for example, a server or a PC.

The audio data acquisition device 201 is a computer that acquires audio data of a target person during a target period. For example, the audio data acquisition device 201 continues to record the conversation of the target person when the target person is on the phone. For example, the audio data acquisition device 201 generates audio data representing a recorded conversation of a target person during a target period of a certain amount of time in the past at regular intervals, and transmits it to the information processing device 100 . The audio data acquisition device 201 is, for example, a PC, a tablet terminal, a smartphone, a wearable terminal, or a smart speaker.

The biometric data acquisition device 202 is a computer that acquires biometric data of a target person during a target period. The biometric data acquisition device 202 continues to measure the biometric characteristic values of the target person, for example, when the target person is on the phone. For example, the biometric data acquisition device 202 generates biometric data that indicates the time change in the measured biological characteristic values of the target person during the target period for a certain past time at regular intervals, and generates biometric data that indicates the temporal change in the measured biological characteristic values of the target person. Send to. The biometric data acquisition device 202 is, for example, a PC, a tablet terminal, a smartphone, or a wearable terminal.

The alert output device 203 is a computer that outputs an alert indicating that a fraudulent act has occurred against a target person. The alert output device 203 receives, for example, a notification from the information processing device 100 indicating that a fraudulent act against a target person has occurred. The alert output device 203 outputs an alert indicating that a fraudulent act against the target person has occurred, for example, in response to receiving a notification indicating that a fraudulent act against the target person has occurred.

Specifically, the alert output device 203 outputs, to a specific person or a specific organization, an alert indicating that a fraudulent act has occurred against the target person. The specific person is, for example, the target person himself or a close relative of the target person. The specific organization is, for example, the police, a financial institution, a security company, or a nursing home. The alert output device 203 is, for example, a PC, a tablet terminal, a smartphone, a wearable terminal, or a smart speaker.

Although the case where the information processing device 100 and the audio data acquisition device 201 are different devices has been described here, the present invention is not limited to this. For example, the information processing device 100 may have a function as the audio data acquisition device 201 and also operate as the audio data acquisition device 201.

Although the case where the information processing device 100 and the biometric data acquisition device 202 are different devices has been described here, the present invention is not limited to this. For example, the information processing device 100 may have a function as the biometric data acquisition device 202 and may also operate as the biometric data acquisition device 202.

Although the case where the information processing device 100 and the alert output device 203 are different devices has been described here, the present invention is not limited to this. For example, the information processing device 100 may have a function as the alert output device 203 and may also operate as the alert output device 203.

Although the case where the audio data acquisition device 201 and the biometric data acquisition device 202 are different devices has been described here, the present invention is not limited to this. For example, the audio data acquisition device 201 may have a function as the biometric data acquisition device 202 and also operate as the biometric data acquisition device 202.

Although the case where the audio data acquisition device 201 and the alert output device 203 are different devices has been described here, the present invention is not limited to this. For example, the audio data acquisition device 201 may have a function as the alert output device 203 and may also operate as the alert output device 203.

Although the case where the biometric data acquisition device 202 and the alert output device 203 are different devices has been described here, the present invention is not limited to this. For example, the biological data acquisition device 202 may have a function as the alert output device 203 and may also operate as the alert output device 203.

(Application example of information processing system 200)
Next, an application example of the information processing system 200 will be described using FIGS. 3 and 4.

FIG. 3 is an explanatory diagram showing a first application example of the information processing system 200. In FIG. 3, a user 301 who is a person to be protected from fraud has a smartphone 310. The user 301 is, for example, at his or her home. The user 301 may be away from home, for example. The information processing system 200 can be applied to a case where the user 301 has a conversation with another person via the smartphone 310. The other person is, for example, a fraudster 302.

In this case, for example, the smartphone 310 realizes the functions of the information processing device 100, the voice data acquisition device 201, the biological data acquisition device 202, and the alert output device 203. .

The smartphone 310 has, for example, a microphone, and when the user 301 makes a call, uses the microphone to acquire voice data indicating the conversation of the user 301, thereby realizing the function as the voice data acquisition device 201. For example, the smartphone 310 has a biosensor 311 on the side, and when the user 301 is on the phone, the biometric sensor 311 that comes into contact with the user's 301 hand is used to acquire the biometric data of the user 301, thereby acting as a biometric data acquisition device. 202.

For example, the smartphone 310 performs information processing by evaluating the occurrence of a criminal act against a target person based on voice data and biometric data, and determining whether a fraudulent act against the target person has occurred. The functions of the device 100 are realized. For example, when it is determined that a fraudulent act against the target person has occurred, the smartphone 310 realizes the function as the alert output device 203 by outputting an alert indicating that a fraudulent act against the target person has occurred. Thereby, the information processing system 200 can easily prevent damage caused by fraud when the user 301 converses with a fraudster via the smartphone 310. Next, the description will move on to FIG. 4.

FIG. 4 is an explanatory diagram showing a second application example of the information processing system 200. In FIG. 4, a user 401 who is a person to be protected from fraud wears a wearable terminal 411. Further, it is assumed that a smart speaker 412 exists near the user 401. The user 401 is, for example, at his or her home.

Further, it is assumed that there is a server 413 that can communicate with the wearable terminal 411 and the smart speaker 412. Further, it is assumed that a person 403 related to the user 401 has a smartphone 414. The related person 403 is, for example, a close relative of the user 401. The person concerned 403 is, for example, at the person's house, which is the home of the person concerned 403 . The information processing system 200 can be applied to a case where the user 401 has a conversation with another person via the landline telephone 415. The other person is, for example, a fraudster 402.

In this case, for example, the server 413 will realize the function of the information processing device 100. The server 413 is provided, for example, at a management company that manages the information processing system 200. Furthermore, the smart speaker 412 realizes the function of the audio data acquisition device 201. Furthermore, the wearable terminal 411 realizes the function of the biometric data acquisition device 202. Furthermore, the smartphone 414 will realize the function of the alert output device 203.

The smart speaker 412 has, for example, a microphone, and when the user 401 is talking, uses the microphone to acquire voice data indicating the conversation of the user 401, thereby realizing the function as the voice data acquisition device 201. For example, the smart speaker 412 transmits the acquired audio data to the server 413.

The wearable terminal 411 has a biosensor, for example, and functions as the biometric data acquisition device 202 by acquiring the biometric data of the user 401 using the biosensor that comes into contact with the user's 401 hand during the user's 401 call. Achieve functionality. Wearable terminal 411 transmits the acquired biometric data to server 413, for example.

The server 413 performs information processing by, for example, evaluating the occurrence of a criminal act against a target person based on voice data and biometric data, and determining whether a fraudulent act against the target person has occurred. The functions of the device 100 are realized. For example, if the server 413 determines that a fraudulent act against the target person has occurred, it transmits a notification to the smartphone 414 indicating that a fraudulent act against the target person has occurred.

For example, in response to receiving a notification indicating that a fraudulent act against the target person has occurred, the smartphone 414 outputs an alert indicating that a fraudulent act against the target person has occurred, thereby controlling the alert output device 203. Realize the function as Thereby, the information processing system 200 can easily prevent damage caused by fraud when the user 401 converses with the fraudster 402 via the landline telephone 415.

For example, when the information processing system 200 determines that a fraudulent act against the user 401 has occurred, the information processing system 200 can enable the person concerned 403 to understand that a fraudulent act against the user 401 has occurred. Therefore, the information processing system 200 makes it easy for the person concerned 403 to quickly take measures such as contacting the user 401, contacting the police, or the bank in order to prevent damage caused by fraudulent acts. can do.

Here, a case has been described in which the smartphone 414 owned by the person 403 associated with the user 401 realizes the function as the alert output device 203, but the present invention is not limited to this. For example, a computer such as a server or a PC installed at a police station or a bank may function as the alert output device 203.

Here, a case has been described in which the information processing system 200 is applied to a situation where the fraudster 402 attempts to commit fraud via the telephone, but the present invention is not limited to this. For example, the information processing system 200 may be applied to a situation in which a fraudster 402 attempts to commit fraud while facing a user 401.

(Example of hardware configuration of information processing device 100)
Next, an example of the hardware configuration of the information processing device 100 will be described using FIG. 5.

FIG. 5 is a block diagram showing an example of the hardware configuration of the information processing device 100. In FIG. 5, the information processing device 100 includes a CPU (Central Processing Unit) 501, a memory 502, a network I/F (Interface) 503, a recording medium I/F 504, and a recording medium 505. Further, each component is connected to each other by a bus 500.

Here, the CPU 501 is in charge of overall control of the information processing apparatus 100. The memory 502 includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and a flash ROM. Specifically, for example, a flash ROM or ROM stores various programs, and a RAM is used as a work area for the CPU 501. The program stored in the memory 502 is loaded into the CPU 501 and causes the CPU 501 to execute the coded processing. The memory 502 stores, for example, a filter information management table 600 that will be described later in FIG. 6, an emotional pattern information management table 700 that will be described later in FIG. 7, and a biometric pattern information management table 800 that will be described later in FIG.

Network I/F 503 is connected to network 210 through a communication line, and connected to other computers via network 210. The network I/F 503 serves as an internal interface with the network 210, and controls data input/output from other computers. The network I/F 503 is, for example, a modem or a LAN adapter.

The recording medium I/F 504 controls reading/writing of data to/from the recording medium 505 under the control of the CPU 501 . The recording medium I/F 504 is, for example, a disk drive, an SSD (Solid State Drive), a USB (Universal Serial Bus) port, or the like. The recording medium 505 is a nonvolatile memory that stores data written under the control of the recording medium I/F 504. The recording medium 505 is, for example, a disk, a semiconductor memory, a USB memory, or the like. The recording medium 505 may be removable from the information processing apparatus 100.

In addition to the components described above, the information processing device 100 may include, for example, a keyboard, a mouse, a printer, a scanner, or the like. Furthermore, the information processing apparatus 100 may include a plurality of recording medium I/Fs 504 and recording media 505. Furthermore, the information processing apparatus 100 does not need to have the recording medium I/F 504 or the recording medium 505.

In addition to the above-described components, the information processing device 100 may include a component corresponding to an audio sensor 906, which will be described later in FIG. Further, the information processing device 100 may include a component corresponding to a biosensor 907, which will be described later in FIG. 9, in addition to the components described above. Further, the information processing apparatus 100 may include a component corresponding to an output device 908, which will be described later in FIG. 9, in addition to the component described above.

(Stored contents of filter information management table 600)
Next, an example of the stored contents of the filter information management table 600 will be explained using FIG. 6. The filter information management table 600 is realized, for example, by a storage area such as the memory 502 or the recording medium 505 of the information processing apparatus 100 shown in FIG.

FIG. 6 is an explanatory diagram showing an example of the stored contents of the filter information management table 600. As shown in FIG. 6, the filter information management table 600 has fields for voice pattern, type, and filter. In the filter information management table 600, filter information is stored as a record 600-a by setting information in each field for each voice pattern. a is any integer.

In the voice pattern field, a voice pattern indicating conditions for classifying the period during which the target person is having a conversation into a plurality of sections and assigning speech attributes to the sections is set. The voice pattern makes it possible to specify the utterance attributes to be assigned to the sections. The voice pattern indicates, for example, the characteristics of the conversational voice indicated by the voice data corresponding to the utterance attribute. Specifically, the voice pattern indicates a keyword corresponding to an utterance attribute. Specifically, the voice pattern may indicate the tone, volume, wavelength, etc. of the voice corresponding to the speech attribute.

Specifically, in the voice pattern field, a voice pattern corresponding to "silence" or a voice pattern corresponding to "agoushu" is set. Specifically, a voice pattern corresponding to "utterance" may be set in the voice pattern field. The speech pattern corresponding to "speech" is not a specific keyword, nor is it a specific tone, volume, or wavelength of speech, but rather a speech pattern corresponding to "silence" and a speech pattern corresponding to "agoushu". It may also indicate that it does not match.

The type of the utterance attribute is set in the type field. In the type field, for example, the type of speech attribute to be assigned to the section corresponding to the voice pattern during the conversation of the target person is set. A filter to be applied to the biometric data in the section corresponding to the voice pattern is set in the filter field. The filter has, for example, a function of removing noise from biometric data.

(Memory contents of emotion pattern information management table 700)
Next, an example of the contents stored in the emotional pattern information management table 700 will be explained using FIG. 7. The emotional pattern information management table 700 is realized, for example, by a storage area such as the memory 502 or the recording medium 505 of the information processing apparatus 100 shown in FIG.

FIG. 7 is an explanatory diagram showing an example of the stored contents of the emotional pattern information management table 700. As shown in FIG. 7, the emotion pattern information management table 700 has fields for emotion patterns and determination results. In the emotional pattern information management table 700, emotional pattern information is stored as a record 700-b by setting information in each field for each emotional pattern. b is an arbitrary integer.

In the field of emotion pattern, an emotion pattern indicating a standard for evaluating the occurrence of fraud is set. The emotion pattern is, for example, a pattern of combinations of characteristic values of emotions of a target person in each of a plurality of intervals. In the judgment result field, for each of the multiple intervals, the occurrence situation of fraud corresponding to the case where the pattern of the combination of characteristic values of the emotions of the actually identified target person corresponds to the above emotion pattern is displayed. The determination result that is the solution is set.

For example, record 700-b specifies that the target person's emotion characteristic values in each of the plurality of speech sections are a combination of positive, negative, and positive from the beginning. Indicates rules such as evaluating that an act has occurred.

(Stored contents of biometric pattern information management table 800)
Next, an example of the stored contents of the biometric pattern information management table 800 will be described using FIG. 8. The biometric pattern information management table 800 is realized, for example, by a storage area such as the memory 502 or the recording medium 505 of the information processing apparatus 100 shown in FIG.

FIG. 8 is an explanatory diagram showing an example of the stored contents of the biometric pattern information management table 800. As shown in FIG. 8, the biometric pattern information management table 800 has fields for biometric patterns and determination results. In the biometric pattern information management table 800, biometric pattern information is stored as a record 800-c by setting information in each field for each biometric pattern. c is any integer.

In the biometric pattern field, a biometric pattern indicating a standard for evaluating the occurrence of fraud is set. The biological pattern is, for example, a pattern of combinations of biological characteristic values of a target person in each of a plurality of sections. The biological pattern may be, for example, a pattern of a combination of statistical values of biological characteristic values of the target person in each of a plurality of sections. The biological pattern may be, for example, a pattern of temporal changes in biological characteristic values of a target person in a certain section.

In the judgment result field, for each of the multiple sections, the occurrence of fraud corresponding to the case where the pattern of combination of biometric feature values of the target person actually identified corresponds to the above biometric pattern. A determination result that is a solution to the situation is set. In the judgment result field, for each of the multiple sections, if the pattern of the combination of the statistical values of the biometric feature values of the target person actually identified corresponds to the above biometric pattern, the fraud A determination result that is a solution to the situation in which the act occurs is set. In the judgment result field, the solution to the occurrence of fraud corresponding to the case where the pattern of time change of the biological characteristic values of the actually identified target person corresponds to the above biological pattern for a certain interval is displayed. A determination result may be set.

For example, record 800-c specifies that if the pattern of time change in the biometric feature values of the actually identified target person matches the biometric pattern showing a monotonous increase in a certain interval, Indicates rules such as evaluating that an act has occurred.

(Example of hardware configuration of audio data acquisition device 201)
Next, an example of the hardware configuration of the audio data acquisition device 201 will be described using FIG. 9.

FIG. 9 is a block diagram showing an example of the hardware configuration of the audio data acquisition device 201. In FIG. 9, the audio data acquisition device 201 includes a CPU 901, a memory 902, a network I/F 903, a recording medium I/F 904, a recording medium 905, an audio sensor 906, a biological sensor 907, and an output device 908. has. Further, each component is connected to each other by a bus 900.

Here, the CPU 901 is in charge of overall control of the audio data acquisition device 201. Memory 902 includes, for example, ROM, RAM, flash ROM, and the like. Specifically, for example, a flash ROM or ROM stores various programs, and a RAM is used as a work area for the CPU 901. The program stored in the memory 902 is loaded into the CPU 901 and causes the CPU 901 to execute the coded processing.

Network I/F 903 is connected to network 210 through a communication line, and connected to other computers via network 210. The network I/F 903 serves as an internal interface with the network 210, and controls data input/output from other computers. The network I/F 903 is, for example, a modem or a LAN adapter.

The recording medium I/F 904 controls reading/writing of data to/from the recording medium 905 under the control of the CPU 901 . The recording medium I/F 904 is, for example, a disk drive, an SSD, a USB port, or the like. The recording medium 905 is a nonvolatile memory that stores data written under the control of the recording medium I/F 904. The recording medium 905 is, for example, a disk, a semiconductor memory, a USB memory, or the like. The recording medium 905 may be removable from the audio data acquisition device 201.

The audio sensor 906 has a microphone that detects audio, and acquires audio data indicating the detected audio by detecting audio related to the target person using the microphone. The audio sensor 906 obtains audio data by generating audio data indicating conversational audio between the target person and another person, detected using a microphone, for example.

A biosensor 907 acquires biometric data of a target person. The biosensor 907 measures biological characteristic values of a target person, such as heart rate, pulse rate, amount of perspiration, or body temperature, and generates biological data indicating changes over time in the measured biological characteristic values. Obtain by doing. The biosensor 907 includes, for example, an imaging device, and determines biological characteristic values of the target person regarding heart rate, pulse rate, amount of perspiration, body temperature, etc. based on image information of the target person captured by the imaging device. may be measured. The biosensor 907 includes, for example, a millimeter wave sensor, and the millimeter wave sensor may be used to measure biological characteristic values of the target person, such as heart rate, pulse rate, amount of perspiration, or body temperature. .

Output device 908 includes a speaker or a display. The output device 908 outputs data such as audio, documents, or images using a speaker or a display. The output device 908 outputs, for example, an alert indicating that a fraudulent act has occurred. Examples of the display include a CRT (Cathode Ray Tube), a liquid crystal display, and an organic EL (Electroluminescence) display.

The audio data acquisition device 201 may include, for example, a keyboard, a mouse, a printer, a scanner, etc. in addition to the components described above. Further, the audio data acquisition device 201 may have a plurality of recording medium I/Fs 904 and recording media 905. Furthermore, the audio data acquisition device 201 does not need to have the recording medium I/F 904 or the recording medium 905.

(Example of hardware configuration of biological data acquisition device 202)
The hardware configuration example of the biometric data acquisition device 202 is specifically the same as the hardware configuration example of the audio data acquisition device 201 shown in FIG. 9, so the description thereof will be omitted.

(Example of hardware configuration of alert output device 203)
The hardware configuration example of the alert output device 203 is specifically the same as the hardware configuration example of the audio data acquisition device 201 shown in FIG. 9, so the description thereof will be omitted.

(Functional configuration example of information processing device 100)
Next, an example of the functional configuration of the information processing device 100 will be described using FIG. 10.

FIG. 10 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a storage unit 1000, an acquisition unit 1001, a classification unit 1002, a processing unit 1003, a calculation unit 1004, a determination unit 1005, and an output unit 1006.

The storage unit 1000 is realized, for example, by a storage area such as the memory 502 or the recording medium 505 shown in FIG. 5. Although a case will be described below in which the storage unit 1000 is included in the information processing device 100, the present invention is not limited to this. For example, there may be a case where the storage unit 1000 is included in a device different from the information processing device 100, and the storage contents of the storage unit 1000 can be referenced from the information processing device 100.

The acquisition unit 1001 to output unit 1006 function as an example of a control unit. Specifically, the acquisition unit 1001 to the output unit 1006 execute the program by causing the CPU 501 to execute a program stored in a storage area such as the memory 502 or the recording medium 505 shown in FIG. This function is realized by The processing results of each functional unit are stored in a storage area such as the memory 502 or the recording medium 505 shown in FIG. 5, for example.

The storage unit 1000 stores various information that is referenced or updated in the processing of each functional unit. The storage unit 1000 stores, for example, audio data. Specifically, the storage unit 1000 acquires audio data indicating a conversation of a target person during a target period. The target person is a person who is desired to be protected from criminal activity.

The criminal act is, for example, fraud. Specifically, the criminal act is an act of fraud over the telephone. Specifically, the criminal act may be a fraudulent act performed face-to-face with the target person. The audio data indicates a conversation between the target person and another person, for example via telephone. The other person may be a criminal, for example. A criminal is, for example, a fraudster.

The target period is, for example, a period during which the target person is having a conversation. The period during a conversation corresponds to, for example, a period during a telephone call. Specifically, the target period is a period from the start of a conversation with the target person to the end of the conversation with the target person. The conversation start corresponds to, for example, the start of a phone call. The end of the conversation corresponds to the end of a phone call, for example. Specifically, the target period may be a part of the period included in the entire period from the start of the conversation of the target person to the end of the conversation of the target person. The audio data is acquired by the acquisition unit 1001, for example.

The storage unit 1000 stores, for example, biometric data. Specifically, the storage unit 1000 acquires biometric data of the target person during the target period. The biometric data indicates, for example, changes over time in biometric feature values of a target person. The biological characteristic value is, for example, a characteristic value related to heartbeat, pulse, body temperature, sweating, or the like. Specifically, the biological characteristic value is heart rate, pulse rate, body temperature, sweat amount, or the like. The biometric data is acquired by the acquisition unit 1001, for example.

The storage unit 1000 stores, for example, a voice pattern that allows a target period to be classified into a plurality of sections. The plurality of sections include, for example, a speech section in which the target person speaks and a listening section in which the target person listens to the speech of another person. Specifically, the utterance section is a section in which the target person uttered utterance contents other than ``Aitsu''.

The plurality of sections may include, for example, a silence section in which the target person is silent and a mutual agreement section in which the target person is in agreement. The silence section and the interlude section are included in the listening section, for example. Specifically, the storage unit 1000 stores voice patterns for each utterance attribute. The utterance attributes include, for example, "silence" indicating that the target person has been silent, and "shuzuchi" indicating that the target person has agreed. The utterance attribute may be, for example, "utterance" indicating that the target person has uttered the utterance. The utterance attribute may be, for example, "listening" indicating that the target person has listened to another person's utterance.

More specifically, the storage unit 1000 stores a voice pattern corresponding to "silence", a voice pattern corresponding to "agoushu", and the like. More specifically, the storage unit 1000 may store voice patterns corresponding to "utterances". More specifically, the storage unit 1000 may store a voice pattern corresponding to "listening". Specifically, the voice pattern indicates a keyword corresponding to an utterance attribute. More specifically, the voice pattern indicates a keyword corresponding to "Aitsuchi". Specifically, the voice pattern may indicate the tone, volume, or wavelength of the voice corresponding to the speech attribute. More specifically, the voice pattern may indicate the upper limit threshold of the target person's voice volume in association with "silence." The storage unit 1000 may store the voice pattern corresponding to "silence" and the voice pattern corresponding to "speech" that do not match the voice pattern corresponding to "speech". More specifically, the storage unit 1000 stores the filter information management table 600 shown in FIG. 6.

The storage unit 1000 stores, for example, each of one or more voice patterns in association with the content of noise removal processing for removing noise from biometric data. The noise removal process is performed using a filter, for example. Specifically, the storage unit 1000 stores each of one or more voice patterns in association with a filter for removing noise from biometric data. For example, the filter removes noise of a size and type corresponding to the speech tendency or the movement tendency of a person corresponding to the voice pattern, for example. More specifically, the storage unit 1000 stores the filter information management table 600 shown in FIG. 6.

The storage unit 1000 stores, for example, information that allows the occurrence of criminal acts to be evaluated. Specifically, the storage unit 1000 stores first correspondence information that associates a change pattern of feature amounts based on audio data with a solution of the occurrence situation of a criminal act corresponding to the change pattern. The feature amount based on the audio data is, for example, one or more feature values of a person's emotion, a statistical value of a feature value of a person's emotion in a certain section, or an amount of change in a characteristic value of a person's emotion in a certain section. . The emotion feature value is, for example, a feature value indicating a psychological state.

The emotion feature value indicates, for example, whether the emotion of the target person is negative or positive. The emotion characteristic value may indicate, for example, that the emotion of the target person is joy, anger, sadness, regret, surprise, motivation, or reluctance. The characteristic value of emotion may be, for example, the degree of stress. The change pattern corresponds to the emotion pattern shown in FIG. 7, for example. The solution is, for example, whether a criminal act has occurred. More specifically, the storage unit 1000 stores the emotional pattern information management table 700 shown in FIG. 7. The first correspondence information is acquired by the acquisition unit 1001, for example. The first correspondence information may be stored in the storage unit 1000 in advance, for example.

Specifically, the storage unit 1000 may store a first model that outputs a solution to the occurrence situation of a criminal act corresponding to the change pattern in response to an input of a change pattern of feature amounts based on voice data. good. The first model is acquired by the acquisition unit 1001, for example. The first model may be stored in the storage unit 1000 in advance, for example. The first model may be trained based on teacher data, for example.

Specifically, the storage unit 1000 stores second correspondence information that associates a change pattern of a feature amount based on biometric data with a solution of the occurrence situation of a criminal act corresponding to the change pattern. The feature amount based on biometric data is, for example, one or more biometric feature values of a person, a statistical value of a person's biometric feature values in a certain section, or the amount of change in a person's biometric feature values in a certain section. etc. The change pattern corresponds to the biological pattern shown in FIG. 8, for example. The solution is, for example, whether a criminal act has occurred. More specifically, the storage unit 1000 stores the biometric pattern information management table 800 shown in FIG. 8. The second correspondence information is acquired by the acquisition unit 1001, for example. The second correspondence information may be stored in the storage unit 1000 in advance, for example.

Specifically, the storage unit 1000 may store a second model that outputs a solution to the occurrence situation of a criminal act corresponding to the change pattern in response to an input of a change pattern of the feature amount based on biometric data. good. The second model is acquired by the acquisition unit 1001, for example. The second model may be stored in the storage unit 1000 in advance, for example. The second model may be trained based on teacher data, for example.

Specifically, the storage unit 1000 associates a combination of a change pattern of feature amounts based on voice data and a change pattern of feature amounts based on biometric data, and a solution of the occurrence situation of a criminal act corresponding to the combination. The attached third correspondence information may be stored. The solution is, for example, whether a criminal act has occurred. The third correspondence information is acquired by the acquisition unit 1001, for example. The third correspondence information may be stored in the storage unit 1000 in advance, for example.

Specifically, in response to an input of a combination of a change pattern of feature amounts based on voice data and a change pattern of feature amounts based on biometric data, the storage unit 1000 stores the occurrence status of a criminal act corresponding to the combination. A third model that outputs a solution may be stored. The third model is acquired by the acquisition unit 1001, for example. The third model may be stored in the storage unit 1000 in advance, for example. The third model may be trained based on teacher data, for example.

The acquisition unit 1001 acquires various information used in processing of each functional unit. The acquisition unit 1001 stores the acquired various information in the storage unit 1000 or outputs it to each functional unit. Further, the acquisition unit 1001 may output various information stored in the storage unit 1000 to each functional unit. The acquisition unit 1001 acquires various information based on, for example, a user's operation input. The acquisition unit 1001 may receive various information from a device different from the information processing device 100, for example.

The acquisition unit 1001 acquires, for example, audio data indicating a conversation of a target person during a target period. Specifically, the acquisition unit 1001 continues to record the conversation of the target person using a microphone during the conversation of the target person, and records the conversation of the target person during the target period from the start of the conversation to the end of the conversation. Acquired by generating audio data indicating the conversation.

Specifically, the acquisition unit 1001 continues to record the conversation of the target person using a microphone while the target person is talking, and records the conversation from the current time until a certain time ago at each predetermined timing while the target person is talking. Audio data indicating a conversation of the target person during the target period may be generated. The predetermined timing is, for example, at regular intervals from the start of the conversation.

Specifically, the acquisition unit 1001 may acquire audio data indicating a conversation of a target person during a target period by receiving from another computer. More specifically, the acquisition unit 1001 receives from another computer audio data indicating the conversation of the target person during the target period from the start of the conversation to the end of the conversation at the end of the conversation of the target person. Obtain by doing. More specifically, the acquisition unit 1001 acquires audio data indicating the conversation of the target person during the target period from the current moment to a certain time ago from another computer at each predetermined timing during the conversation of the target person. Obtained by receiving. The predetermined timing is, for example, at regular intervals from the start of the conversation.

The acquisition unit 1001 acquires, for example, biometric data of a target person during a target period. Specifically, the acquisition unit 1001 continues to measure the biological characteristic value of the target person using a biosensor while the target person is talking. Specifically, the acquisition unit 1001 acquires the target person's biological characteristics during the target period based on the biological characteristic values of the target person measured during the target period from the start of the conversation to the end of the conversation. Acquired by generating biometric data that shows changes in biometric feature values over time.

Specifically, the acquisition unit 1001 continues to measure the biological characteristic value of the target person using a biosensor while the target person is talking. Specifically, the acquisition unit 1001 generates biometric data indicating a temporal change in the biological characteristic value of the target person during the target period at each predetermined timing during the target person's call. You may obtain it. The predetermined timing is, for example, at regular intervals from the start of the conversation. More specifically, the acquisition unit 1001 acquires the target person during the target period based on the biological characteristic values of the target person measured during the target period from the current moment to a certain time ago at each predetermined timing. Generate biometric data that shows changes in biometric feature values over time.

Specifically, the acquisition unit 1001 may acquire biometric data indicating a temporal change in a biometric characteristic value of a target person during a target period from another computer. More specifically, the acquisition unit 1001 acquires biometric data indicating temporal changes in biological characteristic values of the target person during a target period from the start of the conversation to the end of the conversation at the end of the conversation with the target person. is obtained by receiving it from another computer. More specifically, the acquisition unit 1001 acquires biological information indicating temporal changes in the biological characteristic values of the target person during the target period from the current moment to a certain time ago at each predetermined timing during the conversation of the target person. Obtain data by receiving it from another computer. The predetermined timing is, for example, at regular intervals from the start of the conversation.

The acquisition unit 1001 acquires, for example, information that makes it possible to evaluate the occurrence of a criminal act. Specifically, the acquisition unit 1001 acquires first correspondence information that associates a change pattern of the feature amount based on the audio data with a solution of the occurrence situation of a criminal act corresponding to the change pattern. More specifically, the acquisition unit 1001 acquires the first correspondence information by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the first correspondence information by accepting an input of the first correspondence information based on the user's operation input.

Specifically, the acquisition unit 1001 may acquire, in response to an input of a change pattern of feature amounts based on audio data, a first model that outputs a solution to the occurrence situation of a criminal act corresponding to the change pattern. . More specifically, the acquisition unit 1001 acquires the first model by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the first model by accepting input of the first model based on the user's operation input. More specifically, the acquisition unit 1001 acquires the first data based on teacher data that associates a sample of a change pattern of feature amounts based on voice data with a correct answer of the occurrence situation of a criminal act corresponding to the change pattern. It may also be acquired by learning a model.

Specifically, the acquisition unit 1001 acquires second correspondence information that associates a change pattern of the feature amount based on biometric data with a solution of the occurrence situation of a criminal act corresponding to the change pattern. More specifically, the acquisition unit 1001 acquires the second correspondence information by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the second correspondence information by accepting an input of the second correspondence information based on the user's operation input.

Specifically, the acquisition unit 1001 may acquire, in response to input of a change pattern of feature amounts based on biometric data, a second model that outputs a solution to the occurrence situation of a criminal act corresponding to the change pattern. . More specifically, the acquisition unit 1001 acquires the second model by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the second model by accepting input of the second model based on the user's operation input. More specifically, the acquisition unit 1001 acquires second data based on teacher data that associates a sample of a change pattern of feature values based on biometric data with a correct answer of the occurrence situation of a criminal act corresponding to the change pattern. It may also be acquired by learning a model.

Specifically, the acquisition unit 1001 associates a combination of a change pattern of feature amounts based on voice data and a change pattern of feature amounts based on biometric data with a solution of the occurrence situation of a criminal act corresponding to the combination. The attached third correspondence information is acquired. More specifically, the acquisition unit 1001 acquires the third correspondence information by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the third correspondence information by accepting input of the third correspondence information based on the user's operation input.

Specifically, in response to input of a combination of a change pattern of feature amounts based on voice data and a change pattern of feature amounts based on biometric data, the acquisition unit 1001 obtains the occurrence status of a criminal act corresponding to the combination. A third model may be obtained that outputs a solution. More specifically, the acquisition unit 1001 acquires the third model by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the third model by accepting input of the third model based on the user's operation input. More specifically, the acquisition unit 1001 may acquire the third model by learning the third model based on the teacher data. The training data is, for example, a sample of a combination of a change pattern of feature values based on voice data and a change pattern of feature values based on biometric data, and a correct answer of the occurrence situation of a criminal act corresponding to the combination. It is data.

The acquisition unit 1001 may receive a start trigger that starts processing of any functional unit. The start trigger is, for example, a predetermined operation input by the user. The start trigger may be, for example, receiving predetermined information from another computer. The start trigger may be, for example, that any functional unit outputs predetermined information.

For example, the acquisition unit 1001 may receive detection of the end of the conversation of the target person as a start trigger for starting the processing of the classification unit 1002, the processing unit 1003, the calculation unit 1004, and the determination unit 1005. For example, the acquisition unit 1001 may receive the arrival of a predetermined timing as a start trigger for starting the processing of the classification unit 1002, the processing unit 1003, the calculation unit 1004, and the determination unit 1005. The acquisition unit 1001 may receive the acquisition of audio data and biometric data as a start trigger for starting the processing of the classification unit 1002, the processing unit 1003, the calculation unit 1004, and the determination unit 1005.

The classification unit 1002 classifies the target period into a plurality of sections based on the acquired audio data. The plurality of sections are, for example, a speaking section and a listening section. The plurality of sections may be, for example, a silence section and a mutual agreement section. The plurality of sections may be a speaking section, a listening section, a silence section, and a chiding section. The silent section and the interlude section may be included in the listening section. The target period may include, for example, an unclassified time period. The classification unit 1002 may not classify any time period included in the target period as a speaking period, a listening period, a silent period, or a comment period.

The classification unit 1002 classifies the target period into a speaking period and a listening period, for example, based on the acquired audio data. For example, the classification unit 1002 may classify the target period into a silent period and a silent period based on the acquired audio data. For example, the classification unit 1002 may classify the target period into a silent period and a chiding period, as well as a speaking period and a listening period, based on the acquired audio data.

Specifically, the classification unit 1002 specifies a plurality of speakers in the target period by analyzing the audio data. Specifically, for each speaker of the plurality of speakers identified, the classification unit 1002 classifies the section in which the speaker speaks, the voice of the speaker in the section, and the utterance of the speaker in the section. Specify the content etc. The speaker is, for example, the target person or another person. The speaker's voice indicates, for example, the speaker's tone, voice volume, or wavelength.

Specifically, the classification unit 1002 classifies sections in which the target person has uttered utterance contents other than ``Aitsu'' into utterance sections based on the identified speaker, section, voice, and utterance content. . More specifically, based on the identified speaker, section, voice, and utterance content, the classification unit 1002 determines whether the target person's voice or utterance content is " A section that matches the voice pattern corresponding to "utterance" is classified as a speech section. Thereby, the classification unit 1002 can obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process audio data or biometric data in order to evaluate the occurrence of a criminal act against a target person.

Specifically, the classification unit 1002 classifies the section in which the target person uttered the utterance content corresponding to the ``agisho'' into the ``agisho'' section based on the identified speaker, the section, the voice, the utterance content, and the like. Specifically, the classification unit 1002 may classify a section in which another person is speaking and the target person has uttered utterance content corresponding to an agreement into an agreement section. More specifically, based on the identified speaker, section, voice, and utterance content, the classification unit 1002 determines whether the target person's voice or utterance content is " The section that matches the voice pattern corresponding to "Aitsuchi" is classified as an Aitsuchi section. Thereby, the classification unit 1002 can obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process audio data or biometric data in order to evaluate the occurrence of a criminal act against a target person.

Specifically, the classification unit 1002 classifies an interval in which the target person is silent without speaking, as a silent interval, based on the identified speaker, interval, voice, utterance content, and the like. Specifically, the classification unit 1002 may classify a period in which another person is speaking and the target person is silent without speaking as a silent period. More specifically, based on the identified speaker, section, voice, and utterance content, the classification unit 1002 determines whether the target person's voice or utterance content is " A section that matches the voice pattern corresponding to "silence" is classified as a silence section. Thereby, the classification unit 1002 can obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process audio data or biometric data in order to evaluate the occurrence of a criminal act against a target person.

Specifically, the classification unit 1002 classifies sections in which the target person listens to another person's speech into listening sections based on the identified speaker, section, voice, utterance content, and the like. More specifically, the classification unit 1002 classifies the interlude section and the silence section into listening sections. More specifically, based on the identified speaker, section, voice, and utterance content, the classification unit 1002 determines whether the target person's voice or utterance content is " A section that matches the audio pattern corresponding to "listening" may be classified as a listening section. Thereby, the classification unit 1002 can obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process audio data or biometric data in order to evaluate the occurrence of a criminal act against a target person.

For example, based on the acquired audio data, the classification unit 1002 identifies a section of the target period that matches each of the one or more audio patterns. Specifically, as described above, the classification unit 1002 identifies a listening section that matches the audio pattern corresponding to "utterance". Specifically, as described above, the classification unit 1002 identifies a listening section that matches the audio pattern corresponding to "silence". Specifically, as described above, the classification unit 1002 identifies the listening section that matches the voice pattern corresponding to "Aitsuchi". Specifically, as described above, the classification unit 1002 identifies a listening section that matches the audio pattern corresponding to "listening". Thereby, the classification unit 1002 can obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process biometric data in order to evaluate the occurrence of a criminal act against a target person.

The processing unit 1003 performs noise removal processing corresponding to the voice pattern on the biometric data of the target person in the zone corresponding to each voice pattern. Specifically, the biometric data of the target person in the section is a part of the biometric data of the target person in the target period.

For example, the processing unit 1003 refers to the storage unit 1000 and applies a filter associated with the voice pattern to the biometric data of the target person in the interval for the interval that matches each voice pattern. , performs noise removal processing. Thereby, the processing unit 1003 can process the biometric data in order to improve the accuracy of evaluating the occurrence of criminal acts against the target person.

The calculation unit 1004 generates by calculating the first feature amount regarding the target person based on the biometric data. The calculation unit 1004 generates a first feature amount regarding the target person, for example, based on partial biometric data related to the classified listening section among the acquired biometric data in the target period. The partial biometric data may be subjected to noise removal processing by the processing unit 1003, for example. For example, the partial biometric data may not undergo noise removal processing in the processing unit 1003.

Specifically, for each classified listening section, the calculation unit 1004 generates a first feature amount regarding the target person based on partial biometric data related to the listening section out of the acquired biometric data in the target period. do. The first feature amount may be, for example, one or more biological characteristic values of the target person, statistical values of the biological characteristic values of the target person in the listening interval, or biological characteristic values of the target person in the listening interval. This includes the amount of change in feature values. Specifically, the first feature amount is a statistical value of a feature value related to the heartbeat or pulse of the target person in the listening interval, or a change amount of a feature value related to the heartbeat or pulse of the target person in the listening interval, etc. . Thereby, the calculation unit 1004 can enable the determination unit 1005 to evaluate the occurrence of a criminal act against the target person.

The calculation unit 1004 calculates, for example, partial audio data that is difficult to utilize for evaluating the occurrence of criminal acts because noise such as body movement noise tends to be relatively large, such as partial biological data related to utterance sections among the biological data. The first feature quantity can be calculated except for . The calculation unit 1004 calculates, for example, partial biometric data related to a mutual agreement section and a silent section, in which noise such as body movement noise tends to be relatively small, and is easy to use for evaluating the occurrence of criminal acts. Partial biometric data can be used selectively. The calculation unit 1004 can calculate the first feature amount by selectively using partial biometric data that is easy to use for evaluating the occurrence of criminal acts among the biometric data. Therefore, the calculation unit 1004 allows the determination unit 1005 to accurately evaluate the occurrence of criminal acts against the target person.

The calculation unit 1004 generates by calculating a second feature amount regarding the target person based on the audio data. The calculation unit 1004 generates a second feature amount regarding the target person, for example, based on partial voice data related to the classified utterance section among the acquired voice data in the target period.

Specifically, the calculation unit 1004 generates a second feature amount regarding the target person for each classified utterance interval based on partial audio data regarding the utterance interval out of the acquired voice data in the target period. do. The second feature value is, for example, one or more feature values of the target person's emotion, a statistical value of the target person's emotion characteristic value in the utterance interval, or a characteristic value of the target person's emotion characteristic value in the utterance interval. This includes the amount of change. Specifically, the second feature amount may be a statistical value or a change amount of the stress level of the target person in the utterance period. Thereby, the calculation unit 1004 can enable the determination unit 1005 to evaluate the occurrence of a criminal act against the target person.

The calculation unit 1004 calculates, for example, partial audio data related to the intermittent section of the audio data, which is uttered by a target person, but the characteristics of the audio are difficult to read, and it is difficult to utilize for evaluating the occurrence of criminal acts. The second feature amount can be calculated by excluding the partial audio data. For example, the calculation unit 1004 selectively uses partial audio data that is easy to read the characteristics of the audio and is easy to use for evaluating the occurrence situation of criminal acts, such as partial audio data related to utterance sections among the audio data. , the second feature quantity can be calculated. Therefore, the calculation unit 1004 allows the determination unit 1005 to accurately evaluate the occurrence of criminal acts against the target person.

The determination unit 1005 evaluates the occurrence of criminal acts against the target person. The criminal act is a criminal act that may be carried out by another person who converses with the person in question, for example via telephone. Specifically, a criminal act is an act of fraud. The fraudulent acts include, for example, ``I'm me'' fraud, refund fraud, and theatrical fraud.

The determination unit 1005 evaluates the occurrence of a criminal act against the target person, for example, based on the generated first feature amount. Specifically, the determination unit 1005 evaluates the occurrence of a criminal act against the target person based on the generated change pattern of the first feature amount.

More specifically, the determination unit 1005 calculates the amount of change in the generated first feature amount. The amount of change is, for example, the difference between the first feature amounts generated for each of two different listening sections. Specifically, the amount of change is the difference between the first feature amount generated for the first listening section and the first feature amount generated for the last listening section in the target period. More specifically, the determination unit 1005 evaluates that a criminal act against the target person has occurred when the amount of change in the generated first feature amount is equal to or greater than the threshold value. For example, the threshold value is set in advance by the user. More specifically, when the amount of change in the generated first feature amount is less than the threshold value, the determination unit 1005 evaluates that no criminal act has occurred against the target person.

More specifically, the determination unit 1005 evaluates the occurrence of a criminal act against the target person by comparing the above-mentioned second correspondence information with the generated change pattern of the first feature amount. good. More specifically, the determination unit 1005 determines whether the target It is assessed that a criminal act against a person has occurred.

More specifically, the determination unit 1005 determines that if the generated change pattern of the first feature value does not match a specific change pattern set in the second correspondence information that takes the occurrence of a criminal act as a solution, the determination unit 1005 determines whether the target It is assessed that no criminal acts have occurred against these persons. More specifically, the determination unit 1005 determines whether the target It may be assessed that no criminal act has occurred against the person.

More specifically, the determination unit 1005 inputs the generated change pattern of the first feature amount into the second model described above, and uses the second model to determine the occurrence situation of the criminal act against the target person. May be evaluated. More specifically, the determination unit 1005 evaluates the occurrence situation of the criminal act against the target person based on the solution of the occurrence situation of the criminal act outputted by the second model in response to the input.

Specifically, the determination unit 1005 may compare the first feature amount with a threshold value if the generated first feature amount is a change amount of a biological feature value. For example, the threshold value is set in advance by the user. Specifically, the determination unit 1005 evaluates that a criminal act against the target person has occurred when the generated first feature amount is equal to or greater than the threshold value. Specifically, when the generated first feature amount is less than the threshold value, the determination unit 1005 evaluates that no criminal act has occurred against the target person.

Thereby, the determination unit 1005 can accurately evaluate the occurrence of a criminal act. The determination unit 1005 can, for example, accurately evaluate whether a criminal act has occurred. Specifically, the determination unit 1005 uses the first feature amount based on partial biometric data in which noise superposition tends to be relatively small among the biometric data, so the determination unit 1005 accurately determines the occurrence of a criminal act against the target person. can be evaluated well.

The determination unit 1005 evaluates the occurrence of a criminal act against the target person, for example, based on the generated second feature amount. Specifically, the determination unit 1005 evaluates the occurrence of a criminal act against the target person based on the change pattern of the generated second feature amount.

More specifically, the determination unit 1005 calculates the amount of change in the generated second feature amount. The amount of change is, for example, the difference between the second feature amounts generated for each of two different speech sections. Specifically, the amount of change is the difference between the second feature amount generated for the first utterance section and the second feature amount generated for the last utterance section in the target period. More specifically, the determination unit 1005 evaluates that a criminal act against the target person has occurred when the amount of change in the generated second feature amount is equal to or greater than the threshold value. For example, the threshold value is set in advance by the user. More specifically, when the amount of change in the generated second feature amount is less than the threshold value, the determination unit 1005 evaluates that no criminal act has occurred against the target person.

More specifically, the determination unit 1005 evaluates the occurrence of a criminal act against the target person by comparing the first correspondence information described above with the generated change pattern of the second feature amount. good. More specifically, the determination unit 1005 determines whether the target It is assessed that a criminal act against a person has occurred.

More specifically, the determination unit 1005 determines that if the change pattern of the generated second feature value does not match a specific change pattern set in the first correspondence information that takes the occurrence of a criminal act as a solution, the determination unit 1005 determines whether the target It is assessed that no criminal acts have occurred against these persons. More specifically, the determination unit 1005 determines whether the target It may be assessed that no criminal act has occurred against the person.

More specifically, the determination unit 1005 inputs the generated change pattern of the second feature amount into the first model described above, and uses the first model to determine the occurrence situation of the criminal act against the target person. May be evaluated. More specifically, the determination unit 1005 evaluates the occurrence situation of the criminal act against the target person based on the solution of the occurrence situation of the criminal act outputted by the first model in response to the input.

Specifically, the determination unit 1005 may compare the second feature amount with a threshold value if the generated second feature amount is the amount of change in the emotional feature value. For example, the threshold value is set in advance by the user. Specifically, the determination unit 1005 evaluates that a criminal act against the target person has occurred when the generated second feature amount is equal to or greater than the threshold value. Specifically, when the generated second feature amount is less than the threshold value, the determination unit 1005 evaluates that no criminal act has occurred against the target person.

Thereby, the determination unit 1005 can accurately evaluate the occurrence of a criminal act. The determination unit 1005 can, for example, accurately evaluate whether a criminal act has occurred. Specifically, the determination unit 1005 uses the second feature amount based on partial audio data, which is easy to read the audio characteristics and is easy to use for evaluating the occurrence situation of criminal acts. It is possible to accurately evaluate the occurrence of criminal acts against a person.

The determination unit 1005 may evaluate the occurrence of a criminal act against the target person, for example, based on the generated first feature amount and the generated second feature amount. Specifically, the determination unit 1005 evaluates the occurrence of a criminal act against the target person based on the generated change pattern of the first feature amount and the generated change pattern of the second feature amount.

More specifically, the determination unit 1005 calculates the amount of change in the generated first feature amount. The amount of change is, for example, the difference between the first feature amounts generated for each of two different speech sections. Specifically, the amount of change is the difference between the first feature amount generated for the first utterance section and the first feature amount generated for the last utterance section in the target period.

More specifically, the determination unit 1005 calculates the amount of change in the generated second feature amount. The amount of change is, for example, the difference between the second feature amounts generated for each of two different speech sections. Specifically, the amount of change is the difference between the second feature amount generated for the first utterance section and the second feature amount generated for the last utterance section in the target period.

More specifically, if the amount of change in the generated first feature amount is equal to or greater than the threshold value, and the amount of change in the generated second feature amount is equal to or greater than the threshold value, the determination unit 1005 determines that a criminal act has been committed against the target person. Evaluate that this has occurred. More specifically, when the amount of change in the generated first feature amount is less than a threshold value, or the amount of change in the generated second feature amount is less than a threshold value, the determination unit 1005 determines that a criminal act has been committed against the target person. It is evaluated that this has not occurred.

More specifically, the determination unit 1005 collates the combination of the third correspondence information described above, the generated change pattern of the first feature amount, and the generated change pattern of the second feature amount. The occurrence of criminal acts against the target person may be evaluated. More specifically, the determination unit 1005 determines that the combination of the generated change pattern of the first feature amount and the generated change pattern of the second feature amount is a third response whose solution is the occurrence of a criminal act. Determine whether the information matches a specific change pattern set in the information.

More specifically, if the determination unit 1005 determines that the combination matches a specific change pattern set in the third correspondence information, where the answer is the occurrence of a criminal act, the determination unit 1005 determines that a criminal act has occurred against the target person. I rate it as having done so. More specifically, if the determination unit 1005 determines that the combination does not match the specific change pattern set in the third correspondence information, where the answer is the occurrence of a criminal act, the determination unit 1005 determines that a criminal act against the target person has occurred. Evaluate that it has not occurred.

More specifically, the determination unit 1005 determines that the combination of the generated change pattern of the first feature amount and the generated change pattern of the second feature amount is a third response in which the solution is that no criminal act occurs. Determine whether the information matches a specific change pattern set in the information. More specifically, if the determination unit 1005 determines that the combination matches a specific change pattern set in the third correspondence information, where the solution is that no criminal act has occurred, the determination unit 1005 determines that a criminal act has occurred against the target person. You can evaluate it as not having done so.

More specifically, the determination unit 1005 inputs the combination of the generated change pattern of the first feature amount and the generated change pattern of the second feature amount into the third model described above. The occurrence of criminal acts against a target person may be evaluated using the three models. More specifically, the determination unit 1005 evaluates the occurrence situation of the criminal act against the target person based on the solution of the occurrence situation of the criminal act outputted by the third model in response to the input.

Specifically, the determination unit 1005 may compare the generated first feature amount with a first threshold value, and may also compare the generated second feature amount with a second threshold value. The first threshold value is set in advance by the user, for example. The second threshold value is set in advance by the user, for example. Specifically, the determination unit 1005 evaluates that a criminal act against the target person has occurred when the first feature amount is greater than or equal to the first threshold and the second feature amount is greater than or equal to the second threshold. Specifically, when the first feature amount is less than the threshold value or the second feature amount is less than the second threshold value, the determination unit 1005 evaluates that a criminal act against the target person has not occurred.

Thereby, the determination unit 1005 can accurately evaluate the occurrence of a criminal act. The determination unit 1005 can, for example, accurately evaluate whether a criminal act has occurred. Specifically, the determination unit 1005 can use a first feature amount based on partial biometric data in which noise superposition tends to be relatively small among the biometric data. Further, specifically, the determination unit 1005 may use a second feature amount based on partial audio data, which is easy to read the characteristics of the audio data and is easy to use for evaluating the occurrence situation of criminal acts. can. Therefore, the determination unit 1005 can accurately evaluate the occurrence of a criminal act against the target person.

The output unit 1006 outputs the processing result of at least one of the functional units. The output format is, for example, displaying on a display, printing out to a printer, transmitting to an external device via network I/F 503, or storing in a storage area such as memory 502 or recording medium 505. Thereby, the output unit 1006 can notify the user of the processing results of at least one of the functional units, thereby improving the usability of the information processing apparatus 100.

The output unit 1006 outputs, for example, the result of the evaluation of the occurrence of a criminal act by the determination unit 1005. Specifically, the output unit 1006 outputs the result of the evaluation of the occurrence of the criminal act by the determination unit 1005 so that the target person can refer to it. Specifically, the output unit 1006 may output the result of the evaluation of the occurrence of a criminal act by the determination unit 1005 so that a person related to the target person can refer to the result. The related person is, for example, a close relative. Specifically, the output unit 1006 may output the result of the evaluation of the occurrence of a criminal act by the determination unit 1005 so that it can be referenced by a predetermined organization such as the police or a bank. More specifically, the output unit 1006 transmits to the alert output device 203 the result of the evaluation of the occurrence of a criminal act by the determination unit 1005. Thereby, the output unit 1006 can make it easier to prevent criminal damage caused by criminal acts.

For example, when the determination unit 1005 evaluates that a criminal act has occurred, the output unit 1006 may output a notification or an alert indicating that a criminal act has occurred. Specifically, the output unit 1006 outputs a notification or alert indicating that a criminal act has occurred so that the target person can refer to it. Specifically, the output unit 1006 may output a notification or alert indicating that a criminal act has occurred so that a person related to the target person can refer to the notification or alert. The related person is, for example, a close relative. Specifically, the output unit 1006 may output a notification or alert indicating that a criminal act has occurred so that it can be referenced by a predetermined organization such as the police or a bank. More specifically, the output unit 1006 transmits a notification or alert indicating that a criminal act has occurred to the alert output device 203. Thereby, the output unit 1006 can make it easier to prevent criminal damage caused by criminal acts.

Here, a case has been described in which the information processing device 100 evaluates the occurrence of a criminal act based on audio data, but the present invention is not limited to this. For example, by analyzing voice data, another computer can generate text data that shows the content of the conversation between the target person and another person, or analysis data that shows the voice characteristics between the target person and the other person. may be generated. In this case, the information processing device 100 receives, for example, text data indicating the content of a conversation between the target person and another person, or analysis data indicating voice characteristics between the target person and another person, etc. from another computer. receive. The information processing device 100 uses, for example, text data indicating the content of a conversation between a target person and another person, or analysis data indicating voice characteristics between the target person and another person, instead of audio data. do.

Here, a case has been described in which the information processing device 100 includes the acquisition unit 1001, the classification unit 1002, the processing unit 1003, the calculation unit 1004, the determination unit 1005, and the output unit 1006, but the invention is not limited to this. . For example, the information processing device 100 may not include any of the functional units. Specifically, the information processing apparatus 100 may not include the processing unit 1003.

(Example of operation of information processing device 100)
Next, an example of the operation of the information processing device 100 will be described using FIGS. 11 to 13.

11 to 13 are explanatory diagrams showing operation examples of the information processing device 100. In FIG. 11, the information processing device 100 generates audio data indicating the conversation voice between the receiver and the caller over the phone during the call period from the start of the call to the end of the call, when the call of the receiver who could be the victim ends. get. The caller may be a scammer. The information processing device 100 acquires biometric data indicating temporal changes in the recipient's heart rate during the call period.

The information processing apparatus 100 extracts the recipient's voice 1101 and the sender's voice 1102 based on the voice data. Based on the extracted recipient's voice 1101 and caller's voice 1102, the information processing device 100 determines the call period, the listening period in which the recipient listened to the caller's utterances, and the utterances other than the recipient's utterances. The content is classified into the utterance section in which the content was uttered.

The information processing device 100, for example, refers to the filter information management table 600 and obtains a voice pattern corresponding to "silence", a voice pattern corresponding to "agishoku", and the like. Based on the receiver's voice 1101 and the caller's voice 1102, the information processing apparatus 100 determines that the caller is speaking during the call period and that the receiver's voice 1101 has a voice pattern corresponding to "silence". The appearing sections are classified into silent sections and listening sections.

Based on the receiver's voice 1101 and the caller's voice 1102, the information processing device 100 determines that the caller is speaking during the call period and that the receiver's voice 1101 has a voice pattern corresponding to "Aitsuchi". The appearing sections are classified into agreeing sections and listening sections. The information processing device 100 combines consecutive listening sections. The information processing apparatus 100 classifies, for example, a section in which the receiver speaks other than the intermittent section as a speech section, based on the receiver's voice 1101. Next, the explanation will move on to FIG. 12.

In FIG. 12, the information processing apparatus 100 generates the recipient's emotional value for each utterance interval based on the partial voice corresponding to the utterance interval out of the recipient's voice 1101. The emotion value is a characteristic value of emotion. The emotional value is, for example, negative or positive. The emotional value may be, for example, "regrettable" or "motivated". For example, for each utterance section, the information processing device 100 considers the tone, volume, or keyword of the recipient's voice based on the partial speech corresponding to the utterance section, and calculates the receiver's performance in the utterance section. Generate emotional values.

In the example of FIG. 12, the information processing device 100 specifically considers the tone, voice volume, keyword, etc. of the recipient's partial voice “Did you say that until yesterday↓” corresponding to the first utterance section? Then, the receiver's emotional value ``negative ``regrettable'''' in the first utterance section is generated. The arrow indicates whether the tone at the end of the voice is rising or falling. Specifically, the information processing device 100 takes into consideration the tone, voice volume, keyword, etc. of the recipient's voice "I understand. I'll go right away ↑" corresponding to the second utterance section, and performs the second utterance section. The receiver's emotional value "positive 'motivated'" in the utterance section is generated.

The information processing device 100 generates a change pattern of the recipient's emotional value across a plurality of utterance sections. The information processing device 100 refers to the emotion pattern information management table 700 and determines whether or not a fraudulent act has occurred by the sender against the recipient, based on the generated change pattern.

Thereby, the information processing apparatus 100 can accurately determine whether or not a fraudulent act has occurred by the sender against the recipient. The information processing device 100 can, for example, do not need to generate the emotional value of the receiver for a section such as the mutual agreement section where the length of time is relatively short and it is difficult to generate the emotional value of the receiver. The information processing device 100 can generate the recipient's emotional value for a section, such as a speaking section, which has a relatively long time length and is easy to accurately generate the recipient's emotional value. For example, the information processing apparatus 100 can accurately determine whether or not a fraudulent act has occurred by the sender against the recipient, based on a relatively accurate change pattern of the emotional value of the recipient.

In FIG. 13, the information processing apparatus 100 refers to the filter information management table 600 and obtains a silence filter corresponding to "silence", a mutual agreement filter corresponding to "commitment", and the like. The silence filter is, for example, a filter for removing breathing-based noise from biological data. The feedback filter is, for example, a filter for removing noise based on body movements from biological data.

The information processing device 100 performs noise removal processing by applying a silence filter to the partial biometric data corresponding to the silent period among the biometric data indicating the temporal change in the recipient's heart rate during the call period. do. The information processing device 100 performs noise removal processing by applying a matching filter to the partial biometric data corresponding to the matching section among the biometric data indicating temporal changes in the recipient's heart rate during the call period. do.

After performing the noise removal process, the information processing device 100 calculates the listening interval from the heart rate at the start of the listening interval for each listening interval based on the biological data indicating the time change in the recipient's heart rate during the call period. Calculate the amount of change in heart rate up to the end of the period. If the calculated amount of change is equal to or greater than the threshold value in at least one of the listening sections, the information processing device 100 determines that a fraudulent act by the sender against the recipient has occurred. If the calculated amount of change is less than the threshold value for any listening section, the information processing device 100 determines that no fraud has occurred by the sender against the receiver.

After performing the noise removal process, the information processing device 100 calculates the listening interval from the heart rate at the start of the listening interval for each listening interval based on the biological data indicating the time change in the recipient's heart rate during the call period. A change pattern up to the end of the heart rate may be generated. The information processing device 100 may refer to the biometric pattern information management table 800 and determine whether or not a fraudulent act by the sender against the recipient has occurred based on the generated change pattern.

Thereby, the information processing apparatus 100 can accurately determine whether or not a fraudulent act has occurred by the sender against the recipient. For example, the information processing device 100 generates a heart rate change pattern for an interval, such as a speaking interval, in which noise such as body movement noise tends to be relatively large and is difficult to utilize for evaluating the occurrence of criminal acts. You can get away with it without having to do it. For example, the information processing device 100 generates a heart rate change pattern for an interval, such as a listening interval, in which superimposition of noise such as body movement noise is likely to be relatively small and can be easily utilized for evaluating the occurrence of criminal acts. be able to. For this reason, the information processing device 100 accurately determines whether or not a fraudulent act has occurred by the sender against the recipient, based on a heart rate change pattern in which noise such as body movement noise is relatively small. can be made possible.

The information processing device 100 can appropriately perform noise removal processing for each section corresponding to the voice pattern using a filter corresponding to the voice pattern. The information processing device 100 uses a filter that takes into consideration, for example, what kind of noise tends to be included and how large the noise tends to be, depending on the voice pattern. can do. Therefore, the information processing device 100 can accurately remove noise from biometric data. Then, the information processing device 100 can accurately generate a heart rate change pattern, and can accurately determine whether or not a fraudulent act has occurred by the sender against the recipient.

(Overall processing procedure)
Next, an example of the overall processing procedure executed by the information processing apparatus 100 will be described using FIG. 14. The overall processing is realized by, for example, the CPU 501 shown in FIG. 5, a storage area such as a memory 502 or a recording medium 505, and a network I/F 503.

FIG. 14 is a flowchart illustrating an example of the overall processing procedure. The information processing apparatus 100 detects the start of a call by a target person (step S1401).

Next, the information processing apparatus 100 acquires voice data of the target person during the target period at a predetermined timing (step S1402). The predetermined timing is, for example, the timing at which the call ends. The target period is, for example, the call period from the start of the call to the end of the call. The information processing apparatus 100 then acquires biometric data of the target person during the target period at a predetermined timing (step S1403).

Next, the information processing device 100 analyzes the acquired audio data, classifies the target period into a speaking interval and a listening interval, and classifies the listening interval into a silent interval and a chirping interval (step S1404). . Then, the information processing apparatus 100 selects either the speaking section or the listening section as a processing target in order from the beginning of the target period (step S1405).

Next, the information processing device 100 determines whether the selected section is a listening section (step S1406). Here, if the selected section is the listening section (step S1406: Yes), the information processing device 100 moves to the process of step S1407. On the other hand, if the selected section is not a listening section but a speaking section (step S1406: No), the information processing device 100 moves to the process of step S1410.

In step S1407, the information processing apparatus 100 acquires the biometric data of the target person in the listening section from among the biometric data of the target person in the target period (step S1407). Next, the information processing apparatus 100 performs noise removal processing on the biometric data of the target person in the listening section (step S1408).

For example, the information processing device 100 applies a filter corresponding to silence to partial data related to the silence section of the biological data of the target person in the listening section to remove noise. Further, the information processing device 100 applies a filter corresponding to the agreement to partial data related to the agreement section among the biometric data of the target person in the listening section, for example, to remove noise.

Then, the information processing device 100 calculates the first feature amount of the biometric data of the target person in the acquired listening section (step S1409). After that, the information processing apparatus 100 moves to the process of step S1412.

In step S1410, the information processing apparatus 100 extracts the target person's voice data in the utterance period from the target person's voice data in the target period (step S1410). Next, the information processing device 100 calculates a second feature amount of the target person's voice data in the extracted speech section (step S1411). The information processing apparatus 100 then proceeds to the process of step S1412.

In step S1412, the information processing apparatus 100 determines whether all sections have been selected as processing targets (step S1412). Here, if any section is not selected (step S1412: No), the information processing apparatus 100 returns to the process of step S1405. On the other hand, if all sections are selected (step S1412: Yes), the information processing apparatus 100 moves to the process of step S1413.

In step S1413, the information processing device 100 determines whether a fraudulent act has occurred based on the calculated change pattern of the first feature amount and the calculated change pattern of the second feature amount (step S1413). Then, the information processing device 100 ends the overall processing. Thereby, the information processing device 100 can accurately determine whether or not a fraudulent act has occurred.

Although a case has been described here in which the predetermined timing is the timing at which the call ends, the predetermined timing is not limited to this. For example, the predetermined timing may be at regular intervals from the start of the call. In this case, the period of interest may be a period from a predetermined timing to a certain time before. In this case, the information processing apparatus 100 may return to the process of step S1402 after performing the process of step S1413.

Here, the information processing apparatus 100 may perform the processing of some steps in FIG. 14 by changing the order of the processing. For example, the order of the processing in steps S1402 and S1403 can be changed. Further, the information processing apparatus 100 may omit the processing of some steps in FIG. For example, the processing in steps S1408 and S1409 can be omitted.

As described above, according to the information processing apparatus 100, it is possible to acquire audio data indicating a conversation of a target person during a target period. According to the information processing device 100, based on the acquired audio data, it is possible to specify sections of the target period that match each of the one or more set audio patterns. According to the information processing device 100, for each identified section, a noise removal process corresponding to one of the one or more voice patterns that matches the section is performed on the biometric data of the target person in the section. It can be implemented. According to the information processing device 100, it is possible to evaluate the occurrence of a criminal act against a target person based on the target person's biometric data after performing noise removal processing in each of the identified sections. Thereby, the information processing device 100 can appropriately remove noise from the biometric data of the target person, and can accurately evaluate the occurrence of criminal acts against the target person.

The information processing device 100 can include a storage unit that stores one or more voice patterns in association with a filter that removes noise from biometric data. According to the information processing device 100, for each specified section, the information processing device 100 refers to the storage unit and uses a filter associated with one of the voice patterns that matches the section with respect to the biometric data of the target person in the section. Then, noise removal processing can be performed. Thereby, the information processing apparatus 100 can implement noise removal processing using a filter.

According to the information processing device 100, it is possible to acquire audio data indicating a conversation of a target person during a target period via a telephone. According to the information processing device 100, it is possible to evaluate the occurrence of fraud against a target person via telephone based on the target person's biometric data after performing noise removal processing in each specified section. can. Thereby, the information processing device 100 can easily prevent fraud damage caused by fraudulent acts via the telephone.

According to the information processing device 100, information regarding the target person in each specified interval is based on the biological data of the target person after performing noise removal processing corresponding to one of the voice patterns that matches the interval in each of the identified intervals. Features can be generated. According to the information processing device 100, it is possible to evaluate the occurrence of a criminal act against a target person based on the change pattern of the generated feature amount. Thereby, the information processing device 100 can take into account the change pattern of the feature amount, and can accurately evaluate the occurrence of a criminal act against the target person.

According to the information processing device 100, for each specified section, a noise removal process corresponding to one of the voice patterns that matches the section is performed on biometric data indicating a measured value regarding the heartbeat or pulse of the target person in the section. It can be implemented. According to the information processing device 100, a feature quantity that is the amount of change in a measured value regarding the heartbeat or pulse of the target person in each of the specified intervals is generated based on the biological data after performing the noise removal process in each of the identified intervals. be able to. According to the information processing device 100, if the generated feature amount is equal to or greater than the threshold value, it can be evaluated that a criminal act against the target person has occurred. According to the information processing device 100, if the generated feature amount is less than the threshold value, it can be evaluated that no criminal act has occurred against the target person. Thereby, the information processing device 100 can accurately evaluate whether a criminal act has occurred.

According to the information processing device 100, for each specified section, a noise removal process corresponding to one of the voice patterns that matches the section is performed on biometric data indicating a measured value regarding the heartbeat or pulse of the target person in the section. It can be implemented. According to the information processing device 100, a feature amount that is a statistical value of a measured value regarding the heartbeat or pulse of the target person in the specified section is generated based on the biological data after performing the noise removal process in each specified section. be able to. According to the information processing device 100, if the amount of change in the generated feature amount is equal to or greater than the threshold value, it can be evaluated that a criminal act against the target person has occurred. According to the information processing device 100, if the amount of change in the generated feature amount is less than the threshold value, it can be evaluated that no criminal act has occurred against the target person. Thereby, the information processing device 100 can accurately evaluate whether a criminal act has occurred.

Note that the information processing method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a PC or a workstation. The information processing program described in this embodiment is recorded on a computer-readable recording medium, and executed by being read from the recording medium by the computer. The recording medium includes a hard disk, a flexible disk, a CD (Compact Disc)-ROM, an MO (Magneto Optical disc), a DVD (Digital Versatile Disc), and the like. Furthermore, the information processing program described in this embodiment may be distributed via a network such as the Internet.

Regarding the embodiments described above, the following additional notes are further disclosed.

(Additional note 1) Obtain audio data showing the conversation of a target person during a certain period,
Based on the acquired audio data, identify a section of the period that matches each of the one or more set audio patterns,
For each of the identified sections, perform a noise removal process on the biometric data of the target person in the section, corresponding to any one of the one or more voice patterns that matches the section;
Based on the biometric data of the target person after performing noise removal processing corresponding to one of the voice patterns that matched the interval in each of the identified intervals, the occurrence status of the criminal act against the target person is determined. evaluate,
An information processing program that causes a computer to perform processing.

(Additional note 2) The processing to be performed is
With reference to a storage unit that stores each of the one or more voice patterns in association with a filter that removes noise from biometric data, for each of the identified sections, the biometric data of the target person in the section is The information processing according to appendix 1, wherein noise removal processing is performed using a filter associated with one of the voice patterns that matches the interval among the one or more voice patterns. program.

(Additional Note 3) The audio data indicates a conversation of the target person during the period via telephone,
The process to be evaluated is
Based on the biometric data of the target person after performing the noise removal process corresponding to any of the voice patterns that match the interval in each of the identified intervals, the target person's response to the target person's response via the telephone is The information processing program according to appendix 1 or 2, characterized in that the information processing program evaluates the occurrence of fraudulent acts against a person.

(Additional note 4) Based on the biometric data of the target person in each of the identified intervals after performing noise removal processing corresponding to any of the voice patterns that match the interval, the target person in the interval is determined. Generate features related to
The process to be evaluated is
The information processing program according to appendix 1 or 2, characterized in that the information processing program evaluates the occurrence of a criminal act against the target person based on the generated change pattern of the feature amount.

(Additional note 5) The processing to be performed is:
For each of the identified sections, correspond to any one of the one or more voice patterns that matches the section with respect to biometric data indicating a measured value regarding the heartbeat or pulse of the target person in the section. Perform noise removal processing to
The process to generate the
Based on biometric data indicating a measured value regarding the heartbeat or pulse of the target person after performing noise removal processing corresponding to any of the voice patterns that match the specified sections in each of the specified sections, Generating a feature amount that is an amount of change in a measured value regarding the heartbeat or pulse of the target person,
The process to be evaluated is
If the generated feature amount is greater than or equal to a threshold value, it is evaluated that a criminal act against the target person has occurred, and if the generated feature amount is less than the threshold value, a criminal act against the target person has not occurred. The information processing program according to appendix 4, characterized in that the information processing program is evaluated as follows.

(Additional note 6) The processing to be performed is
For each of the identified sections, correspond to any one of the one or more voice patterns that matches the section with respect to biometric data indicating a measured value regarding the heartbeat or pulse of the target person in the section. Perform noise removal processing to
The process to generate the
Based on biometric data indicating a measured value regarding the heartbeat or pulse of the target person after performing noise removal processing corresponding to any of the voice patterns that match the specified sections in each of the specified sections, Generate a feature amount that is a statistical value of a measured value regarding the heartbeat or pulse of the target person,
The process to be evaluated is
If the amount of change in the generated feature amount is greater than or equal to the threshold, it is evaluated that a criminal act against the target person has occurred, and if the amount of change in the generated feature amount is less than the threshold, a crime against the target person is evaluated. The information processing program according to appendix 4, characterized in that it evaluates that the act has not occurred.

(Additional note 7) Obtain audio data showing the conversation of the target person during a certain period,
Based on the acquired audio data, identify a section of the period that matches each of the one or more set audio patterns,
For each of the identified sections, perform a noise removal process on the biometric data of the target person in the section, corresponding to any one of the one or more voice patterns that matches the section;
Based on the biometric data of the target person after performing noise removal processing corresponding to one of the voice patterns that matched the interval in each of the identified intervals, the occurrence status of the criminal act against the target person is determined. evaluate,
An information processing method characterized in that processing is performed by a computer.

(Additional note 8) Obtain audio data indicating the conversation of the target person during a certain period,
Based on the acquired audio data, identify a section of the period that matches each of the one or more set audio patterns,
For each of the identified sections, perform a noise removal process on the biometric data of the target person in the section, corresponding to any one of the one or more voice patterns that matches the section;
Based on the biometric data of the target person after performing noise removal processing corresponding to one of the voice patterns that matched the interval in each of the identified intervals, the occurrence status of the criminal act against the target person is determined. evaluate,
An information processing device comprising a control section.

100 Information processing device 101 Voice data 102 Biometric data 200 Information processing system 201 Voice data acquisition device 202 Biometric data acquisition device 203 Alert output device 210 Network 301,401 User 302,402 Fraudster 310,414 Smartphone 311,907 Biosensor 403 Relationship Person 411 Wearable terminal 412 Smart speaker 413 Server 415 Landline phone 500,900 Bus 501,901 CPU
502,902 Memory 503,903 Network I/F
504,904 Recording medium I/F
505,905 Recording medium 600 Filter information management table 700 Emotion pattern information management table 800 Biological pattern information management table 906 Audio sensor 908 Output device 1000 Storage section 1001 Acquisition section 1002 Classification section 1003 Processing section 1004 Calculation section 1005 Judgment section 1006 Output section 1101 ,1102 Audio

Claims (5)

Obtain audio data showing the conversation of a target person during a certain period,
Based on the acquired audio data, identify a section of the period that matches each of the one or more set audio patterns,
For each of the identified sections, perform a noise removal process on the biometric data of the target person in the section, corresponding to any one of the one or more voice patterns that matches the section;
Based on the biometric data of the target person after performing noise removal processing corresponding to one of the voice patterns that matched the interval in each of the identified intervals, the occurrence status of the criminal act against the target person is determined. evaluate,
An information processing program that causes a computer to perform processing. The process to be performed is
With reference to a storage unit that stores each of the one or more voice patterns in association with a filter that removes noise from biometric data, for each of the identified sections, the biometric data of the target person in the section is The information according to claim 1, wherein noise removal processing is performed using a filter associated with one of the one or more audio patterns that matches the interval. Processing program. the audio data indicates a conversation of the target person during the period via a telephone;
The process to be evaluated is
Based on the biometric data of the target person after performing the noise removal process corresponding to any of the voice patterns that match the interval in each of the identified intervals, the target person's response to the target person's response via the telephone is 3. The information processing program according to claim 1, wherein the information processing program evaluates the occurrence of fraud against a person. Obtain audio data showing the conversation of a target person during a certain period,
Based on the acquired audio data, identify a section of the period that matches each of the one or more set audio patterns,
For each of the identified sections, perform a noise removal process on the biometric data of the target person in the section, corresponding to any one of the one or more voice patterns that matches the section;
Based on the biometric data of the target person after performing noise removal processing corresponding to one of the voice patterns that matched the interval in each of the identified intervals, the occurrence status of the criminal act against the target person is determined. evaluate,
An information processing method characterized in that processing is performed by a computer. Obtain audio data showing the conversation of a target person during a certain period,
Based on the acquired audio data, identify a section of the period that matches each of the one or more set audio patterns,
For each of the identified sections, perform a noise removal process on the biometric data of the target person in the section, corresponding to any one of the one or more voice patterns that matches the section;
Based on the biometric data of the target person after performing noise removal processing corresponding to one of the voice patterns that matched the interval in each of the identified intervals, the occurrence status of the criminal act against the target person is determined. evaluate,
An information processing device comprising a control section.

Images