JP2023127393A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

To provide an information processing device capable of detecting water wetting on a surface of a terminal device using sound output from a speaker and acquired by a microphone, an information processing method, and an information processing program.SOLUTION: In an information processing system 1, a server device 100 comprises: an instruction unit which issues an instruction to output sound from a speaker SK attached to a housing of a terminal device 10; a detection unit which detects a foreign matter adhering to the housing of the terminal device by using active acoustic sensing on the basis of sound acquired by a microphone MK attached to the housing of the terminal device; and a learning unit which learns characteristics of sound with the foreign matter adhering to the housing of the terminal device and constructs a model. The detection unit also detects the foreign matter adhering to the housing of the terminal device by using the model.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Active acoustic sensing technology has been disclosed that recognizes hand gestures by attaching a speaker and microphone to the arm.

Study of hand gesture recognition method using active acoustic sensing, Information Processing Society of Japan Research Report, August 6, 2016, [Retrieved November 22, 2021], Internet <URL: https://ipsj.ixsq.nii. ac.jp/ej/?action=repository_action_common_download&item_id=171767&item_no=1&attribute_id=1&file_no=1＞

However, the above-mentioned conventional technology merely recognizes gestures, and there is room to provide further services to users. For example, in recent years, with the advent of waterproof smart devices, it has become easy to operate devices near water such as the ocean or pool, around water such as in the kitchen or bathroom, or outdoors during rainy weather. However, in the case of capacitive touch sensing, if there are water droplets on the screen, erroneous touch detection may occur (a touch may occur at the location of the water drop, the touch position may be shifted due to the water drop, etc.).

The present application has been made in view of the above, and aims to detect water wetness on the surface of a terminal device using sound output from a speaker and acquired by a microphone.

The information processing device according to the present application includes an instruction unit that issues a command to output sound from a speaker attached to a casing of a terminal device, and an active acoustic sensing system based on sound acquired by a microphone attached to the casing of the terminal device. and a detection unit that detects foreign matter adhesion to the casing of the terminal device using the following.

According to one aspect of the embodiment, water wetness on the surface of a terminal device can be detected using sound output from a speaker and acquired by a microphone.

FIG. 1 is an explanatory diagram showing an overview of an information processing method according to an embodiment. FIG. 2 is a diagram illustrating a configuration example of an information processing system according to an embodiment. FIG. 3 is a diagram illustrating a configuration example of a terminal device according to an embodiment. FIG. 4 is a diagram illustrating a configuration example of a server device according to an embodiment. FIG. 5 is a diagram showing an example of a user information database. FIG. 6 is a diagram showing an example of a history information database. FIG. 7 is a diagram showing an example of a detection information database. FIG. 8 is a flowchart showing the processing procedure according to the embodiment. FIG. 9 is a diagram showing an example of the hardware configuration.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail below with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited to this embodiment. In addition, in the following embodiments, the same parts are given the same reference numerals, and redundant explanations will be omitted.

[1. Overview of information processing method]
First, with reference to FIG. 1, an overview of an information processing method performed by an information processing apparatus according to an embodiment will be described. FIG. 1 is an explanatory diagram showing an overview of an information processing method according to an embodiment. Note that, in FIG. 1, an example will be described in which water wetness on the surface of a terminal device is detected by sound output from a speaker and acquired by a microphone.

As shown in FIG. 1, the information processing system 1 includes a terminal device 10 and a server device 100. The terminal device 10 and the server device 100 are connected to be able to communicate with each other by wire or wirelessly via a network N (see FIG. 2). In this embodiment, the terminal device 10 cooperates with the server device 100.

The terminal device 10 is a smart device such as a smartphone or a tablet terminal used by a user U (user), and is connected to an arbitrary server device via a wireless communication network such as 4G (Generation) or LTE (Long Term Evolution). This is a mobile terminal device that can communicate. Further, the terminal device 10 has a screen such as a liquid crystal display, which has a touch panel function, and displays data such as content through a tap operation, a slide operation, a scroll operation, etc. from the user U using a finger or a stylus. Accepts various operations on. Note that an operation performed on an area of the screen where content is displayed may be an operation on the content. Further, the terminal device 10 may be not only a smart device but also an information processing device such as a desktop PC (Personal Computer) or a notebook PC.

The server device 100 cooperates with the terminal device 10 of each user U, and provides API (Application Programming Interface) services for various applications (hereinafter referred to as apps) and various data to the terminal device 10 of each user U. It is an information processing device that provides information, and is realized using computers, cloud systems, etc.

Further, the server device 100 may be an information processing device that provides some kind of web service online to the terminal device 10 of each user U. For example, the server device 100 provides web services such as Internet connection, search service, SNS (Social Networking Service), electronic commerce (EC), electronic payment, online games, online banking, online trading, accommodation/ticket reservation, Services such as video/music distribution, news, maps, route searches, route guidance, route information, operation information, weather forecasts, etc. may be provided. In reality, the server device 100 may cooperate with various servers that provide the above-mentioned Web services, mediate the Web services, or may be in charge of processing the Web services.

Note that the server device 100 can acquire user information regarding the user U. For example, the server device 100 acquires information regarding user U's attributes such as user U's gender, age, and residential area. The server device 100 stores and manages identification information indicating the user U (user ID, etc.) as well as information regarding attributes of the user U.

Further, the server device 100 acquires various kinds of history information (log data) indicating the actions of the user U from the terminal device 10 of the user U or from various servers based on the user ID or the like. For example, the server device 100 acquires a location history, which is a history of user U's location, date and time, from the terminal device 10. The server device 100 also acquires a search history, which is a history of search queries input by the user U, from a search server (search engine). Additionally, the server device 100 acquires a viewing history, which is a history of content viewed by the user U, from the content server. The server device 100 also acquires a purchase history (payment history) that is a history of product purchases and payment processing by the user U from the electronic commerce server and the payment processing server. Further, the server device 100 may acquire the listing history and the sales history, which are the listing history of the user U on the marketplace, from the electronic commerce server or the payment processing server. The server device 100 also acquires a posting history, which is a history of postings by the user U, from a posting server or SNS server that provides a word-of-mouth posting service.

[1-1. Water wetness detection using active acoustic sensing]
Generally, in active acoustic sensing technology, a speaker generates an acoustic signal, and a microphone captures the reflected sound and the sound propagated through the housing as the acoustic signal. Then, the acquired acoustic signal is analyzed using FFT (Fast Fourier Transformation), and machine learning is performed using the signal strength of each frequency band as a feature quantity.

In this embodiment, the server device 100 outputs sound from a speaker attached to the casing of the terminal device 10 of each target user U, and acquires the sound with a microphone attached to the casing of the terminal device 10. Wetness (water droplets) on the surface of the terminal device 10 is detected using active acoustic sensing.

As shown in FIG. 1, the server device 100 gives an instruction (instruction) to output sound from the speaker SK attached to the housing of the terminal device 10 of each user U via the network N (see FIG. 2). (Step S1). The command (instruction) may be data or a signal. Note that if the speaker SK is independent from the terminal device 10, the server device 100 may directly issue a command to the speaker SK to output sound.

Next, the terminal device 10 of the user U outputs sound from the speaker SK attached to the casing of the terminal device 10 in response to the command from the server device 100 (step S2). For example, speaker SK is a vibration speaker that can be used for active acoustic sensing. Note that the speaker SK may be externally attached to the surface of the casing, or may be built-in/mounted in the casing.

Next, the terminal device 10 of the user U acquires the sound using the microphone MK attached to the casing of the terminal device 10 (step S3). For example, microphone MK is a piezo microphone that can be used for active acoustic sensing. Note that the microphone MK may be externally attached to the surface of the casing, or may be built-in/mounted in the casing.

Next, the terminal device 10 of the user U transmits information regarding the sound acquired by the microphone MK to the server device 100 via the network N (see FIG. 2) (step S4). Note that when the microphone MK is independent from the terminal device 10, the microphone MK may directly transmit information regarding the acquired sound to the server device 100.

Next, the server device 100 receives information regarding the sound acquired by the microphone MK from the terminal device 10 of each user U, and detects water wetness of the casing of the terminal device 10 using active acoustic sensing. (Step S5). The housing of the terminal device 10 includes a screen DP. The server device 100 detects water on the screen DP of the casing.

Note that the server device 100 detects water in the casing of the terminal device 10 based on the characteristics (characteristics) of the sound when the casing of the terminal device 10 is wet and the characteristics of the sound when the casing is not wet. Wetness may also be detected. At this time, the server device 100 constructs (generates, updates) a model that has learned the characteristics of the sound when the casing is wet, and uses the model to detect whether the casing of the terminal device 10 is wet.

For example, the server device 100 learns the characteristics of the sound output from the speaker SK (output sound) and the characteristics of the sound acquired by the microphone MK (obtained sound) when the housing of the terminal device 10 is wet with water. and build the model. At this time, the server device 100 learns the characteristics of the acquired sound (or both the characteristics of the output sound and the characteristics of the acquired sound) and constructs a model. Alternatively, the server device 100 constructs a model by learning the difference between the output sound and the acquired sound.

The server device 100 learns the characteristics of the acquired sound (or the characteristics of the difference between the output sound and the acquired sound) when a predetermined location (location, part, part) of the housing of the terminal device 10 is wet. A model may be built that shows which areas are wet. For example, when the acquired sound is input, the server device 100 outputs information indicating which part of the housing of the terminal device 10 (back surface, screen, microphone part, speaker part, charging connector part, etc.) is wet. You can also build a model.

At this time, the server device 100 may learn and build a model for each type of terminal device 10 (model, size, etc.). Further, the server device 100 may learn and build a model for each peripheral device, part, or accessory (cover, ring, etc.) of the terminal device 10.

Further, the server device 100 may change a threshold value regarding determination of contact with a touch panel or the like of the casing of the terminal device 10 based on the detection result of water wetting of the casing of the terminal device 10 . Further, the server device 100 may determine whether the operation on the terminal device 10 is valid or invalid based on the detection result of water wetness of the casing of the terminal device 10. In this way, by using active acoustic sensing technology on a smart device to detect the adhesion of water droplets (a state in which water droplets are attached), erroneous touch operations can be reduced. If water droplets are present, the touch sensitivity threshold is changed so that the touch input is not detected by signals such as those caused by water droplets. Further, since a water droplet spreads, but the shape of a finger does not change as much as a water droplet, a threshold value may be provided for the value of the contact area in addition to the touch sensitivity.

In addition, the server device 100 receives the result that "there is a mysterious touch" and "water wetness is detected (water droplet detection)" on the terminal device 10 of each user U. You may also perform a control such as "

In addition, the server device 100 transmits information regarding the detection result of water wetness of the casing of the terminal device 10 of each user U via the network N (see FIG. 2) to the terminal device 10 to be detected (or the terminal device thereof). and the other party's terminal device 10) that is communicating with the other party.

Further, the server device 100 may detect not only water wetting (water droplets) on the surface of the terminal device 10 but also dirt (dust, oil film, air bubbles, etc.) on the surface of the terminal device 10. At this time, the server device 100 may construct a model that has learned the sound characteristics of a wet state and the sound characteristics of a dirty state, and use the model to determine whether the object is wet or dirty.

In addition, the sound transmitted through the air also becomes a characteristic of the right hand, left hand, etc. Therefore, the server device 100 constructs a model that learns characteristics such as right and left hands from sounds transmitted through the air, and uses the model to determine whether the user U is touching the terminal device 10 with the right hand or with the left hand. It may also be possible to determine whether

As described above, in the present embodiment, the housing of the terminal device 10 of the target user U includes a speaker SK capable of outputting an output sound including sound of a predetermined wavelength, and an output sound output from the speaker SK. Attach an available microphone MK. Then, the server device 100 detects water wetting in the casing according to the characteristics of the sound acquired by the microphone MK. Thereby, the server device 100 can detect water wetness on the surface of the terminal device 10 of the user U, and can determine whether the suspicious operation is due to water wetness. Further, the server device 100 can detect, for example, water droplets that the user U cannot see (visually recognize) with the naked eye, water droplets that the user U is not aware of, and the like.

In addition, in the above-mentioned embodiment, other terminal devices 10 or the terminal device 10 itself of the target user U may be used instead of the server device 100. That is, the subject of the treatment is not limited to the server device 100, but also other terminal devices 10 or the target user U's terminal device 10 itself, using active acoustic sensing to detect the surface of the user U's terminal device 10. Water wetness may also be detected.

For example, the other terminal device 10 outputs sound from a speaker SK attached to the casing of user U's terminal device 10, and acquires the sound with a microphone MK attached to the casing of user U's terminal device 10. Wetness of the surface of the terminal device 10 of the user U is detected using active acoustic sensing. As a result, other terminal devices 10 can detect water wetness on the surface of user U's terminal device 10, and can determine whether the suspicious communication content from user U's terminal device 10 is due to water wetness. You will be able to determine whether there is any such thing.

In addition, the terminal device 10 of the user U performs active acoustic sensing in which the speaker SK attached to the casing of the terminal device 10 outputs sound and the microphone MK attached to the casing of the terminal device 10 acquires the sound. is used to detect water wetness on the surface of the terminal device 10. This allows the terminal device 10 of the user U to self-diagnose whether the surface of the terminal device 10 is wet with water. Further, the terminal device 10 of the user U can detect, for example, water droplets that the user U cannot see (visually recognize) with the naked eye, water droplets that the user U is not aware of, and the like.

[2. Configuration example of information processing system]
Next, the configuration of the information processing system 1 including the server device 100 according to the embodiment will be described using FIG. 2. FIG. 2 is a diagram showing a configuration example of the information processing system 1 according to the embodiment. As shown in FIG. 2, the information processing system 1 according to the embodiment includes a terminal device 10 and a server device 100. These various devices are connected via a network N so that they can communicate by wire or wirelessly. The network N is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network) such as the Internet.

Furthermore, the number of devices included in the information processing system 1 shown in FIG. 2 is not limited to what is illustrated. For example, in FIG. 2, only one terminal device 10 is shown for simplification of illustration, but this is just an example and is not limited, and there may be two or more terminal devices.

The terminal device 10 is an information processing device used by the user U. For example, the terminal device 10 may be a smart device such as a smartphone or a tablet terminal, a feature phone, a PC (Personal Computer), a PDA (Personal Digital Assistant), a game console or AV device with a communication function, a car navigation system, a smart watch, or the like. These include wearable devices such as head-mounted displays, smart glasses, etc.

The terminal device 10 also supports wireless communication networks such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), Bluetooth (registered trademark), and wireless LAN (Local The server device 100 can be connected to the network N via short-range wireless communication such as an area network) and can communicate with the server device 100.

The server device 100 is, for example, a computer such as a PC or a blade server, or a mainframe or a workstation. Note that the server device 100 may be realized by cloud computing.

[3. Configuration example of terminal device]
Next, the configuration of the terminal device 10 will be explained using FIG. 3. FIG. 3 is a diagram showing a configuration example of the terminal device 10. As shown in FIG. As shown in FIG. 3, the terminal device 10 includes a communication section 11, a display section 12, an input section 13, a positioning section 14, a sensor section 20, a control section 30 (controller), and a storage section 40. Be prepared.

(Communication Department 11)
The communication unit 11 is connected to a network N (see FIG. 2) by wire or wirelessly, and transmits and receives information to and from the server device 100 via the network N. For example, the communication unit 11 is realized by a NIC (Network Interface Card), an antenna, or the like.

(Display section 12)
The display unit 12 is a display device that displays various information such as position information. For example, the display unit 12 is a liquid crystal display (LCD) or an organic electro-luminescent display (EL display). Further, the display unit 12 is a touch panel type display, but is not limited to this.

(Input section 13)
The input unit 13 is an input device that receives various operations from the user U. For example, the input unit 13 includes buttons for inputting characters, numbers, and the like. Note that the input unit 13 may be an input/output port (I/O port), a USB (Universal Serial Bus) port, or the like. Further, when the display section 12 is a touch panel display, a part of the display section 12 functions as the input section 13. Further, the input unit 13 may be a microphone or the like that receives voice input from the user U. The microphone may be wireless.

(Positioning unit 14)
The positioning unit 14 receives a signal (radio wave) sent from a GPS (Global Positioning System) satellite, and based on the received signal, determines position information (for example, latitude and longitude). That is, the positioning unit 14 positions the terminal device 10 . Note that GPS is just one example of GNSS (Global Navigation Satellite System).

Further, the positioning unit 14 can measure the position using various methods other than GPS. For example, the positioning unit 14 may use various communication functions of the terminal device 10 to measure the position as an auxiliary positioning means for position correction and the like, as described below.

(Wi-Fi positioning)
For example, the positioning unit 14 positions the terminal device 10 using the Wi-Fi (registered trademark) communication function of the terminal device 10 or the communication network provided by each communication company. Specifically, the positioning unit 14 performs Wi-Fi communication, etc., and determines the position of the terminal device 10 by measuring the distance to nearby base stations and access points.

(Beacon positioning)
Further, the positioning unit 14 may use the Bluetooth (registered trademark) function of the terminal device 10 to measure the position. For example, the positioning unit 14 measures the position of the terminal device 10 by connecting to a beacon transmitter connected by a Bluetooth (registered trademark) function.

(geomagnetic positioning)
Furthermore, the positioning unit 14 positions the terminal device 10 based on the geomagnetic pattern of the structure measured in advance and the geomagnetic sensor included in the terminal device 10 .

(RFID positioning)
Further, for example, if the terminal device 10 has an RFID (Radio Frequency Identification) tag function equivalent to a contactless IC card used at station ticket gates, stores, etc., or has a function to read an RFID tag. In this case, the location where the terminal device 10 used the terminal device 10 is recorded together with the information that the payment was made. The positioning unit 14 may measure the position of the terminal device 10 by acquiring such information. Further, the position may be determined by an optical sensor, an infrared sensor, or the like provided in the terminal device 10.

The positioning unit 14 may position the terminal device 10 using one or a combination of the above-mentioned positioning means, if necessary.

(sensor section 20)
The sensor unit 20 includes various sensors mounted on or connected to the terminal device 10. Note that the connection may be a wired connection or a wireless connection. For example, the sensors may be a detection device other than the terminal device 10, such as a wearable device or a wireless device. In the example shown in FIG. 3, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, an atmospheric pressure sensor 23, an air temperature sensor 24, a sound sensor 25, an optical sensor 26, a magnetic sensor 27, and an image sensor ( camera) 28.

Note that each of the sensors 21 to 28 described above is merely an example and is not limited to the above. That is, the sensor section 20 may be configured to include a portion of each of the sensors 21 to 28, or may include other sensors such as a humidity sensor in addition to or instead of each of the sensors 21 to 28. .

The acceleration sensor 21 is, for example, a three-axis acceleration sensor, and detects the physical movement of the terminal device 10, such as the moving direction, speed, and acceleration of the terminal device 10. The gyro sensor 22 detects physical movements of the terminal device 10 such as tilt in three axes directions based on the angular velocity of the terminal device 10 and the like. The atmospheric pressure sensor 23 detects the atmospheric pressure around the terminal device 10, for example.

Since the terminal device 10 is equipped with the above-mentioned acceleration sensor 21, gyro sensor 22, atmospheric pressure sensor 23, etc., it is possible to implement technologies such as pedestrian autonomous navigation (PDR) using these sensors 21 to 23, etc. It becomes possible to measure the position of the terminal device 10 using the . This makes it possible to obtain indoor position information that is difficult to obtain using positioning systems such as GPS.

For example, a pedometer using the acceleration sensor 21 can calculate the number of steps, walking speed, and distance walked. Furthermore, by using the gyro sensor 22, it is possible to know the direction of travel of the user U, the direction of the line of sight, and the inclination of the user's body. Further, from the atmospheric pressure detected by the atmospheric pressure sensor 23, it is also possible to know the altitude and the number of floors where the terminal device 10 of the user U is located.

The temperature sensor 24 detects, for example, the temperature around the terminal device 10. The sound sensor 25 detects, for example, sounds around the terminal device 10. The optical sensor 26 detects the illuminance around the terminal device 10 . The magnetic sensor 27 detects, for example, the earth's magnetism around the terminal device 10. The image sensor 28 captures an image of the surroundings of the terminal device 10.

The above-mentioned atmospheric pressure sensor 23, temperature sensor 24, sound sensor 25, optical sensor 26, and image sensor 28 each detect atmospheric pressure, temperature, sound, and illuminance, and capture images of the surroundings, so that the terminal device 10 It is possible to detect the surrounding environment and situation. Furthermore, it is possible to improve the accuracy of the location information of the terminal device 10 based on the environment and situation around the terminal device 10.

(Control unit 30)
The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM, an input/output port, etc., and various circuits. Further, the control unit 30 may be configured with hardware such as an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control section 30 includes a transmitting section 31, a receiving section 32, and a processing section 33.

(Transmission unit 31)
The transmitting unit 31 receives, for example, various information input by the user U using the input unit 13, various information detected by the sensors 21 to 28 mounted on or connected to the terminal device 10, and information measured by the positioning unit 14. The location information of the terminal device 10 and the like can be transmitted to the server device 100 via the communication unit 11.

(Receiving unit 32)
The receiving unit 32 can receive various information provided from the server device 100 and requests for various information from the server device 100 via the communication unit 11 .

(Processing unit 33)
The processing unit 33 controls the entire terminal device 10, including the display unit 12 and the like. For example, the processing unit 33 can output various information transmitted by the transmitting unit 31 and various information received from the server device 100 by the receiving unit 32 to the display unit 12 for display.

In the present embodiment, the processing unit 33 outputs sound (output sound) from the speaker SK attached to the casing of the terminal device 10, and acquires the sound (obtained sound) with the microphone MK attached to the casing of the terminal device 10. do. At this time, the processing unit 33 may output sound from the speaker SK in response to a command to output sound from the speaker SK received by the receiving unit 32 from the server device 100 or another terminal device 10. Further, the processing unit 33 may transmit information regarding the sound acquired by the microphone MK from the transmitting unit 31 to the server device 100 or other terminal device 10.

(Storage unit 40)
The storage unit 40 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an optical disk. Ru. The storage unit 40 stores various programs, various data, and the like.

[4. Configuration example of server device]
Next, the configuration of the server device 100 according to the embodiment will be described using FIG. 4. FIG. 4 is a diagram showing a configuration example of the server device 100 according to the embodiment. As shown in FIG. 4, the server device 100 includes a communication section 110, a storage section 120, and a control section 130.

(Communication Department 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card). Further, the communication unit 110 is connected to the network N (see FIG. 2) by wire or wirelessly.

(Storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD, an SSD, or an optical disk. As shown in FIG. 4, the storage unit 120 includes a user information database 121, a history information database 122, and a detection information database 123.

(User information database 121)
The user information database 121 stores user information regarding the user U. For example, the user information database 121 stores various information such as user U's attributes. FIG. 5 is a diagram showing an example of the user information database 121. In the example shown in FIG. 5, the user information database 121 has items such as "user ID (identifier),""age,""gender,""home,""worklocation," and "interest."

“User ID” indicates identification information for identifying user U. Note that the "user ID" may be user U's contact information (telephone number, email address, etc.), or may be identification information for identifying user U's terminal device 10.

Moreover, "age" indicates the age of the user U identified by the user ID. Note that the "age" may be information indicating the specific age of the user U (for example, 35 years old, etc.), or may be information indicating the age of the user U (for example, 30s, etc.) . Alternatively, the "age" may be information indicating the date of birth of the user U, or may be information indicating the generation of the user U (for example, born in the 1980s). Furthermore, “gender” indicates the gender of the user U identified by the user ID.

Moreover, "home" indicates the location information of the home of the user U identified by the user ID. In the example shown in FIG. 5, "home" is illustrated as an abstract code such as "LC11", but it may also be latitude and longitude information. Furthermore, for example, "home" may be a region name or address.

Moreover, "work place" indicates the location information of the work place (school in the case of a student) of the user U identified by the user ID. In the example shown in FIG. 5, the "work location" is illustrated as an abstract code such as "LC12", but it may also be latitude/longitude information or the like. Further, for example, the "work location" may be a region name or address.

Moreover, "interest" indicates the interest of the user U identified by the user ID. That is, "interest" indicates an object in which the user U identified by the user ID has a high interest. For example, "interest" may be a search query (keyword) that the user U inputs into a search engine. In the example shown in FIG. 5, one "interest" is shown for each user U, but there may be a plurality of "interests".

For example, in the example shown in FIG. 5, the age of the user U identified by the user ID "U1" is "20s", and the gender is "male". Further, for example, the user U identified by the user ID "U1" indicates that his home is "LC11". Further, for example, the user U identified by the user ID "U1" indicates that the work location is "LC12". Further, for example, the user U identified by the user ID "U1" indicates that he is interested in "sports."

Here, in the example shown in FIG. 5, abstract values such as "U1", "LC11", and "LC12" are used for illustration, but "U1", "LC11", and "LC12" have specific values. It is assumed that information such as character strings and numerical values is stored. Below, abstract values may be illustrated in diagrams related to other information as well.

Note that the user information database 121 is not limited to the above, and may store various information depending on the purpose. For example, the user information database 121 may store various information regarding the terminal device 10 of the user U. In addition, the user information database 121 includes information such as demographic (demographic attributes), psychographic (psychological attributes), geographic (geographical attributes), behavioral (behavioral attributes), etc. of user U. Information regarding attributes may also be stored. For example, the user information database 121 includes name, family composition, place of birth (locality), occupation, position, income, qualifications, type of residence (single-family house, condominium, etc.), presence or absence of a car, commuting/commuting time, commuting/commuting, etc. Memorizes information such as routes, commuter pass sections (stations, lines, etc.), frequently used stations (other than the stations closest to your home or work), lessons (location, time zone, etc.), hobbies, interests, lifestyle, etc. You can.

(History information database 122)
The history information database 122 stores various information related to history information (log data) indicating the actions of the user U. FIG. 6 is a diagram showing an example of the history information database 122. In the example shown in FIG. 6, the history information database 122 includes items such as "user ID,""locationhistory,""searchhistory,""browsinghistory,""purchasehistory," and "posting history."

“User ID” indicates identification information for identifying user U. Further, “position history” indicates a position history that is a history of the user U's position and movement. Further, “search history” indicates a search history that is a history of search queries input by the user U. In addition, “browsing history” indicates a browsing history that is a history of contents that the user U has viewed. Moreover, "purchase history" indicates a purchase history that is a history of purchases by user U. Moreover, "posting history" indicates a posting history that is a history of postings by user U. Note that the "posting history" may include questions regarding user U's belongings.

For example, in the example shown in FIG. 6, user U identified by user ID "U1" moves as per "location history #1", searches as per "search history #1", and searches as per "view history #1". This indicates that the content was viewed according to "history #1", a predetermined product, etc. was purchased at a predetermined store etc. according to "purchase history #1", and the content was posted according to "posting history".

Here, in the example shown in FIG. 6, abstracts such as "U1", "location history #1", "search history #1", "browsing history #1", "purchase history #1" and "posting history #1" are used. Although the figures are illustrated using ``U1'', ``Location history #1'', ``Search history #1'', ``Browsing history #1'', ``Purchase history #1'' and ``Posting history #1'', It is assumed that information such as specific character strings and numerical values is stored.

Note that the history information database 122 is not limited to the above, and may store various information depending on the purpose. For example, the history information database 122 may store the user U's usage history of a predetermined service. Further, the history information database 122 may store the user U's visit history to a physical store, visit history to a facility, or the like. Further, the history information database 122 may store the payment history of user U's payment using the terminal device 10 (electronic payment).

(Detection information database 123)
The detection information database 123 stores various information regarding detection results such as water wetting. FIG. 7 is a diagram showing an example of the detection information database 123. In the example shown in FIG. 7, the detection information database 123 has items such as "user ID", "terminal", "output sound", "obtained sound", "wet water", "dirt", and "location". .

“User ID” indicates identification information for identifying user U. Further, "terminal" indicates the type (model, size, etc.) of the terminal device 10 of the user U. Note that the "terminal" may also include information regarding peripheral devices, parts, or accessories (cover, ring, etc.) of the terminal device 10.

Moreover, "output sound" indicates the characteristics of the sound (output sound) output from the speaker SK attached to the housing of the terminal device 10. Furthermore, “obtained sound” indicates the characteristics of the sound (obtained sound) obtained by the microphone NK attached to the housing of the terminal device 10.

Furthermore, “wet” indicates whether or not there is water such as water droplets attached to the casing of the terminal device 10.
Further, "dirt" indicates the presence or absence of dirt attached to the casing of the terminal device 10. Note that "dirt" may be information indicating the type of dirt (dust, oil film, air bubbles, etc.).

Further, "location" indicates a location (location, portion, region) of the housing of the terminal device 10 where water or dirt has occurred. Note that the "location" may separately indicate a location where "wet water" has occurred and a location where "dirt" has occurred.

For example, in the example shown in FIG. The speaker SK outputs the sound indicated by "output sound #1", the microphone MK attached to the terminal device 10 acquires the sound indicated by "obtained sound #1", and as a result of active acoustic sensing, the sound indicated by "water wetness" is detected. ” indicates that it has been detected that “dirt” has occurred on the “screen” of the terminal device 10.

Here, in the example shown in FIG. 7, abstract values such as "U1", "terminal #1", "output sound #1", and "obtained sound #1" are used for illustration, but "U1", " It is assumed that information such as specific character strings and numerical values is stored in "terminal #1", "output sound #1", and "obtained sound #1".

Note that the detection information database 123 is not limited to the above, and may store various information depending on the purpose. For example, the detection information database 123 may store a model constructed by learning the characteristics of the output sound and the acquired sound (or the difference thereof). Further, the detection information database 123 may store items to be adjusted based on the detection result of water wetness of the housing of the terminal device 10. Further, the detection information database 123 may store information indicating whether the user U is touching the terminal device 10 with the right hand or the left hand.

(Control unit 130)
Returning to FIG. 4, the explanation will be continued. The control unit 130 is a controller, and controls the server device 100 using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array). This is realized by executing various programs (corresponding to an example of an information processing program) stored in an internal storage device using a storage area such as a RAM as a work area. In the example shown in FIG. 4, the control unit 130 includes an acquisition unit 131, an instruction unit 132, a detection unit 133, a learning unit 134, an adjustment unit 135, and a provision unit 136.

(Acquisition unit 131)
The acquisition unit 131 acquires a search query input by the user U. For example, when the user U inputs a search query into a search engine or the like and performs a keyword search, the acquisition unit 131 acquires the search query via the communication unit 110. That is, the acquisition unit 131 acquires, via the communication unit 110, a keyword input by the user U into a search window of a search engine, site, or application.

The acquisition unit 131 also acquires user information regarding the user U via the communication unit 110. For example, the acquisition unit 131 acquires identification information indicating the user U (user ID, etc.), location information of the user U, attribute information of the user U, etc. from the terminal device 10 of the user U. Further, the acquisition unit 131 may acquire identification information indicating the user U, attribute information of the user U, etc. at the time of user U's user registration. The acquisition unit 131 then registers the user information in the user information database 121 of the storage unit 120.

Further, the acquisition unit 131 acquires various types of history information (log data) indicating the behavior of the user U via the communication unit 110. For example, the acquisition unit 131 acquires various types of history information indicating the behavior of the user U from the terminal device 10 of the user U or from various servers based on the user ID or the like. The acquisition unit 131 then registers various types of history information in the history information database 122 of the storage unit 120.

(Instruction part 132)
The command unit 132 issues a command to the terminal device 10 of each target user U to output sound from the speaker SK attached to the casing of the terminal device 10 via the communication unit 110.

(Detection unit 133)
The detection unit 133 receives the sound acquired by the microphone MK attached to the casing of the terminal device 10 via the communication unit 110, and based on the sound acquired by the microphone MK attached to the casing of the terminal device 10, Active acoustic sensing is used to detect foreign matter adhesion (wet water, dirt, etc.) on the casing of the terminal device 10.

The detection unit 133 detects the adhesion of a foreign object to the casing of the terminal device 10 based on the characteristics of the sound when the casing of the terminal device 10 has the foreign object attached and the characteristics of the sound when the casing of the terminal device 10 has no foreign object.

(Learning section 134)
The learning unit 134 learns the characteristics of the sound when a foreign object is attached to the casing of the terminal device 10 and constructs a model. At this time, the detection unit 133 detects the adhesion of foreign matter to the casing of the terminal device 10 using the model.

For example, the learning unit 134 learns the characteristics of the acquired sound acquired by the microphone MK with a foreign object attached to the housing of the terminal device 10, or the characteristics of the output sound output from the speaker SK and the acquired sound acquired by the microphone MK. Build a model by learning both sound characteristics and sound characteristics.

In addition, the learning unit 134 learns the difference between the characteristics of the output sound output from the speaker SK and the characteristics of the acquired sound acquired by the microphone MK with a foreign object attached to the housing of the terminal device 10 to create a model. To construct.

Further, the learning unit 134 learns the characteristics of the acquired sound when a predetermined part of the housing of the terminal device 10 has foreign matter attached, and constructs a model indicating which part is wet. At this time, the detection unit 133 uses the model to detect (or estimate) which part of the housing of the terminal device 10 is wet.

Further, the learning unit 134 learns and constructs a model for each type of terminal device 10. Further, the learning unit 134 learns and constructs a model for each peripheral device, part, or accessory of the terminal device 10.

The learning unit 134 also learns the characteristics of the sound when the casing of the terminal device 10 has foreign matter attached to it, and the characteristics of the sound when the casing of the terminal device 10 has dirt attached, and determines whether the casing of the terminal device 10 has foreign matter or dirt. Build a model that shows At this time, the detection unit 133 uses the model to detect (or estimate) whether foreign matter is attached or dirty. That is, the detection unit 133 uses the model to detect (or estimate) whether the casing of the terminal device 10 has foreign matter or dirt attached to it.

(Adjustment section 135)
The adjustment unit 135 performs various controls based on the detection result of foreign matter adhering to the casing of the terminal device 10 of each user U. For example, the adjustment unit 135 may change a threshold value for determining contact with a touch panel or the like on the casing of the terminal device 10 based on the detection result of foreign matter adhesion on the casing of the terminal device 10 . Further, the adjustment unit 135 may determine whether the operation on the terminal device 10 is valid or invalid based on the detection result of foreign matter adhesion to the housing of the terminal device 10.

In addition, based on the results that "there is a mysterious touch" and "foreign object has been detected (foreign object detection)" on the terminal device 10 of each user U, the adjustment unit 135 cancels the operation because of foreign object. You may also perform a control such as "

(Providing unit 136)
The providing unit 136 transmits, via the communication unit 110, information regarding the detection result of foreign matter adhesion on the casing of the terminal device 10 of each user U to the terminal device 10 to be detected (or the other party communicating with the terminal device). terminal device 10).

[5. Processing procedure]
Next, a processing procedure by the terminal device 10 and the server device 100 according to the embodiment will be described using FIG. 8. FIG. 8 is a flowchart showing the processing procedure according to the embodiment. Note that the processing procedure shown below is repeatedly executed by the control unit 30 of the terminal device 10 and the control unit 130 of the server device 100.

As shown in FIG. 8, the instruction unit 132 of the server device 100 sends a message from the speaker SK attached to the housing of the terminal device 10 to the terminal device 10 of each target user U via the communication unit 110. A command to output sound is issued (step S101). At this time, the speaker SK attached to the housing of the terminal device 10 outputs sound.

Subsequently, the detection unit 133 of the server device 100 receives, via the communication unit 110, the sound acquired by the microphone MK attached to the housing of the terminal device 10 (step S102). At this time, the microphone MK attached to the housing of the terminal device 10 acquires the sound output from the speaker SK, and transmits information regarding the acquired sound to the server device 100.

Subsequently, the detection unit 133 of the server device 100 detects water wetness of the casing of the terminal device 10 using active acoustic sensing based on the sound acquired by the microphone MK attached to the casing of the terminal device 10 ( Step S103).

Next, the learning unit 134 of the server device 100 learns the characteristics of the sound when the casing of the terminal device 10 is wet and constructs a model (step S104). For example, the learning unit 134 learns the characteristics of the acquired sound acquired by the microphone MK while the housing of the terminal device 10 is wet, or the characteristics of the output sound output from the speaker SK and the acquired sound acquired by the microphone MK. Build a model by learning both sound characteristics and sound characteristics. Alternatively, the learning unit 134 creates a model by learning the difference between the characteristics of the output sound output from the speaker SK and the characteristics of the acquired sound acquired by the microphone MK while the housing of the terminal device 10 is wet. To construct.

Next, the learning unit 134 of the server device 100 learns the characteristics of the acquired sound when a predetermined part of the housing of the terminal device 10 is wet, and constructs a model indicating which part is wet (step S105). At this time, the detection unit 133 uses the model to detect (or estimate) which part of the housing of the terminal device 10 is wet.

Subsequently, the learning unit 134 of the server device 100 learns and constructs a model for each type of terminal device 10 and each peripheral device/part/accessory of the terminal device 10 (step S106).

Subsequently, the learning unit 134 of the server device 100 learns the characteristics of the sound when the casing of the terminal device 10 is wet and the characteristics of the sound when the casing of the terminal device 10 is dirty. A model indicating whether the object is wet or dirty is constructed (step S107). At this time, the detection unit 133 uses the model to detect (or estimate) whether the object is wet or dirty.

Subsequently, the adjustment unit 135 of the server device 100 performs various controls based on the detection result of the water wetness of the casing of the terminal device 10 of each user U (step S108).

Subsequently, the providing unit 136 of the server device 100 transmits information regarding the detection result of water wetness of the casing of the terminal device 10 of each user U to the terminal device 10 to be detected (or its terminal) via the communication unit 110. The information is provided to the other party's terminal device 10 that is communicating with the device (step S109).

[6. Modified example]
The terminal device 10 and server device 100 described above may be implemented in various different forms other than the above embodiments. Therefore, a modification of the embodiment will be described below.

In the above embodiment, part or all of the processing executed by the server device 100 may actually be executed by the terminal device 10. For example, the process may be completed stand-alone (by the terminal device 10 alone). In this case, it is assumed that the terminal device 10 has the functions of the server device 100 in the above embodiment. Further, in the embodiment described above, since the terminal device 10 cooperates with the server device 100, it appears to the user U that the terminal device 10 is also executing the processing of the server device 100. That is, from another point of view, it can be said that the terminal device 10 includes the server device 100.

Furthermore, in the embodiment described above, a plurality of speakers SK and microphones MK may each be attached to the housing of one terminal device 10. The number of speakers SK and microphones MK may not be the same. For example, the sound output from one speaker SK attached to the housing of the terminal device 10 may be acquired by each of the microphones NK attached to a plurality of locations on the housing of the terminal device 10. On the contrary, the sound output from the speakers SK attached to a plurality of locations on the housing of the terminal device 10 may be acquired by one microphone NK attached to the housing of the terminal device 10. Note that if the microphone MK cannot properly acquire sound when sound is output from a plurality of speakers SK at the same time, the timing at which sound is output from each speaker SK may be shifted. At this time, the server device 100 may estimate the water-wet location of the casing of the terminal device 10 based on which speaker SK outputs the sound and which microphone MK acquires the sound.

Further, in the above embodiment, a plurality of pairs (sets) of a pair of speakers SK and microphones MK may be installed in the housing of one terminal device 10. In this case, the number of speakers SK and microphones MK is the same. For example, a pair of speaker SK and microphone MK may be installed on the front side (screen side) and the back side of one terminal device 10, respectively. Note that if the microphone MK cannot properly acquire sound when sound is output from a plurality of speakers SK at the same time, the timing at which sound is output from each speaker SK may be shifted.

[7. effect〕
As described above, the information processing device (terminal device 10 and server device 100) according to the present application includes a command unit 132 that issues a command to output sound from the speaker SK attached to the housing of the terminal device 10; The terminal device 10 includes a detection unit 133 that detects foreign matter adhesion to the casing of the terminal device 10 using active acoustic sensing based on the sound acquired by the microphone MK attached to the casing of the terminal device 10 .

The detection unit 133 detects the adhesion of a foreign object to the casing of the terminal device 10 based on the characteristics of the sound when the casing of the terminal device 10 has the foreign object attached and the characteristics of the sound when the casing of the terminal device 10 has no foreign object.

Further, the information processing device according to the present application further includes a learning unit 134 that learns the characteristics of the sound in a state where a foreign object is attached to the casing of the terminal device 10 and constructs a model. The detection unit 133 detects foreign matter adhesion to the casing of the terminal device 10 using the model.

The learning unit 134 learns the characteristics of the acquired sound acquired by the microphone MK with a foreign object attached to the housing of the terminal device 10, or the characteristics of the output sound output from the speaker SK and the acquired sound acquired by the microphone MK. Build a model by learning both features.

The learning unit 134 constructs a model by learning the difference between the characteristics of the output sound output from the speaker SK and the characteristics of the acquired sound acquired by the microphone MK with a foreign object attached to the housing of the terminal device 10. .

The learning unit 134 learns the characteristics of the acquired sound when a predetermined part of the housing of the terminal device 10 has foreign matter attached, and constructs a model indicating which part is wet.

The learning unit 134 learns and constructs a model for each type of terminal device 10.

The learning unit 134 learns and constructs a model for each peripheral device, part, or accessory of the terminal device 10.

The learning unit 134 learns the characteristics of the sound when the casing of the terminal device 10 is wet and the characteristics of the sound when the casing of the terminal device 10 is dirty, and indicates whether the casing is wet or dirty. Build the model.

By using any one or a combination of the above-described processes, the information processing device according to the present application can detect water wetness on the surface of the terminal device using the sound output from the speaker and acquired by the microphone.

[8. Hardware configuration]
Further, the terminal device 10 and the server device 100 according to the embodiments described above are realized by, for example, a computer 1000 having a configuration as shown in FIG. The following will explain the server device 100 as an example. FIG. 9 is a diagram showing an example of the hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I/F (Interface) 1060, an input I/F 1070, and a network I/F 1080 are connected to a bus. 1090.

The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and performs various processes. The arithmetic device 1030 is realized by, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used by the calculation device 1030 for various calculations. Further, the secondary storage device 1050 is a storage device in which data used by the calculation device 1030 for various calculations and various databases are registered, and includes a ROM (Read Only Memory), an HDD (Hard Disk Drive), and an SSD (Solid Disk Drive). This is realized using flash memory, etc. The secondary storage device 1050 may be a built-in storage or an external storage. Further, the secondary storage device 1050 may be a removable storage medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) memory card. Further, the secondary storage device 1050 may be a cloud storage (online storage), a NAS (Network Attached Storage), a file server, or the like.

The output I/F 1060 is an interface for transmitting information to be output to the output device 1010 that outputs various information such as a display, a projector, and a printer. (Digital Visual Interface) and HDMI (registered trademark) (High Definition Multimedia Interface). Further, the input I/F 1070 is an interface for receiving information from various input devices 1020 such as a mouse, keyboard, keypad, button, scanner, etc., and is realized by, for example, a USB or the like.

Further, the output I/F 1060 and the input I/F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, output device 1010 and input device 1020 may be wireless devices.

Moreover, the output device 1010 and the input device 1020 may be integrated like a touch panel. In this case, the output I/F 1060 and the input I/F 1070 may also be integrated as an input/output I/F.

Note that the input device 1020 is, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like.

Network I/F 1080 receives data from other devices via network N and sends it to computing device 1030, and also sends data generated by computing device 1030 to other devices via network N.

Arithmetic device 1030 controls output device 1010 and input device 1020 via output I/F 1060 and input I/F 1070. For example, the arithmetic device 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

For example, when the computer 1000 functions as the server device 100, the arithmetic unit 1030 of the computer 1000 realizes the functions of the control unit 130 by executing a program loaded onto the primary storage device 1040. Further, the arithmetic device 1030 of the computer 1000 may load a program obtained from another device via the network I/F 1080 onto the primary storage device 1040, and execute the loaded program. Furthermore, the arithmetic unit 1030 of the computer 1000 may cooperate with other devices via the network I/F 1080, and may call and use program functions, data, etc. from other programs of other devices.

[9. others〕
Although the embodiments of the present application have been described above, the present invention is not limited to the contents of these embodiments. Furthermore, the above-mentioned components include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those that are in a so-called equivalent range. Furthermore, the aforementioned components can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the constituent elements can be made without departing from the gist of the embodiments described above.

Further, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured.

For example, the server device 100 described above may be realized by a plurality of server computers, and depending on the function, it may be realized by calling an external platform etc. using an API (Application Programming Interface), network computing, etc. Can be changed flexibly.

Furthermore, the above-described embodiments and modifications can be combined as appropriate within a range that does not conflict with the processing contents.

Further, the above-mentioned "section, module, unit" can be read as "means", "circuit", etc. For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

1 Information processing system 10 Terminal device SK Speaker MK Microphone 100 Server device 110 Communication unit 120 Storage unit 121 User information database 122 History information database 123 Detection information database 130 Control unit 131 Acquisition unit 132 Command unit 133 Detection unit 134 Learning unit 135 Adjustment Section 136 Provision Section

Claims (11)

a command unit that issues a command to output sound from a speaker attached to a housing of the terminal device;
a detection unit that uses active acoustic sensing to detect foreign matter adhesion to the casing of the terminal device based on sound acquired by a microphone attached to the casing of the terminal device;
An information processing device comprising: The detection unit detects the adhesion of a foreign object to the casing of the terminal device based on the characteristics of the sound when the casing of the terminal device has the foreign object attached and the characteristics of the sound when the casing of the terminal device has no foreign object attached. The information processing device according to claim 1, characterized in that: a learning unit that constructs a model by learning the characteristics of sound in a state where a foreign object is attached to the housing of the terminal device;
Furthermore,
3. The information processing apparatus according to claim 1, wherein the detection unit detects adhesion of foreign matter to a casing of the terminal device using the model. The learning unit is configured to learn the characteristics of the acquired sound acquired by the microphone with a foreign object attached to the housing of the terminal device, or the characteristics of the output sound output from the speaker and the acquired sound acquired by the microphone. The information processing device according to claim 3, wherein the information processing device constructs a model by learning both the characteristics and the characteristics. The learning unit constructs a model by learning the difference between the characteristics of the output sound output from the speaker and the characteristics of the acquired sound acquired by the microphone with a foreign object attached to the casing of the terminal device. The information processing device according to claim 3, characterized in that: Claim 3, wherein the learning unit learns the characteristics of the acquired sound when a predetermined part of the housing of the terminal device has foreign matter attached, and constructs a model indicating which part is wet. The information processing device described. The information processing apparatus according to claim 3, wherein the learning unit constructs a model by learning for each type of the terminal device. The information processing device according to claim 3, wherein the learning unit constructs a model by learning for each peripheral device, part, or accessory of the terminal device. The learning unit learns the characteristics of the sound when the casing of the terminal device is wet with water and the characteristics of the sound when the casing of the terminal device is dirty, and indicates whether the casing of the terminal device is wet or dirty. The information processing device according to claim 3, wherein the information processing device constructs a model. An information processing method executed by an information processing device, the method comprising:
a command step for issuing a command to output sound from a speaker attached to the housing of the terminal device;
a detection step of detecting foreign matter adhesion to the casing of the terminal device using active acoustic sensing based on sound acquired by a microphone attached to the casing of the terminal device;
An information processing method characterized by comprising: a command procedure for issuing a command to output sound from a speaker attached to the housing of the terminal device;
a detection procedure of detecting foreign matter adhesion to the casing of the terminal device using active acoustic sensing based on sound acquired by a microphone attached to the casing of the terminal device;
An information processing program that allows a computer to execute.

Images