JP7408419B2 - Information processing systems and electronic equipment - Google Patents

Description

The present application relates to information processing systems and electronic devices.

In recent years, mobile phones and systems for children have been proposed that are equipped with various security functions (crime prevention functions) for the purpose of preventing children from being involved in crimes such as kidnapping.

Japanese Patent Application Publication No. 2009-53855 Japanese Patent Application Publication No. 2008-102751

In order to eradicate criminal acts such as child abduction, it is necessary to construct a system that can act as a deterrent to deter such criminal acts.

An information processing system according to one aspect includes a user terminal and an information processing apparatus that processes data acquired by the user terminal. The user terminal includes a communication unit that transmits and receives various information to and from the information processing device, an audio input unit that is arranged to be able to input sounds surrounding the user terminal, and a device that captures images of the surroundings of the user terminal. an audio acquisition unit that acquires the input audio input by the audio input unit; an image acquisition unit that acquires the image captured by the imaging unit; and a comparison of the input audio. a storage unit that stores in advance information on comparison audio for performing the comparison; If it is determined that they do not match, the information processing device uses the audio data of the input audio and the image data of the image acquired by the image acquisition unit as terminal acquired audio data and terminal acquired image data, respectively, via the communication unit. and a determination unit that transmits the information to the user. The information processing device includes a communication unit that receives the terminal-acquired audio data and the terminal-acquired image data from the user terminal, and a comparison image data storage that stores comparison image data for comparing the terminal-acquired image data. a risk word storage section that stores information on risk words; a risk storage section that stores risk levels associated with age, gender, and emotion; A voice analysis unit that divides and acquires text data corresponding to voice data into morphemes; and an image analysis unit that analyzes the terminal-acquired image data and acquires the age, gender, and emotion of the person shown in the terminal-acquired image. and, if the terminal acquired image data and the comparison image data do not match, the match rate between the morpheme acquired by the voice analysis unit and the dangerous word, the age, gender, and age acquired by the image analysis unit. and a risk level determining unit that derives a risk level indicating whether or not a child who is a user of the user terminal is likely to be involved in a crime based on the risk level associated with the emotion.

Further, an electronic device according to one aspect is an electronic device that executes processing based on audio and images, and includes an audio input section arranged to be able to input sounds surrounding the electronic device; an imaging unit arranged to be able to capture surrounding images; an audio acquisition unit that acquires the input audio input by the audio input unit; an image acquisition unit that acquires the image captured by the imaging unit; A voice storage unit that stores in advance information on a comparison voice for comparison with voice, an image storage unit that stores in advance information on a comparison image for comparison with the image, and information on dangerous words. A risk word storage unit, a risk level storage unit that stores risk levels associated with age, gender, and emotion, and an input voice data acquired by the voice acquisition unit are analyzed, and the data of the input voice is A voice analysis unit that classifies and acquires the corresponding text data into morphemes, and an image that analyzes data of the image acquired by the image acquisition unit and acquires the age, gender, and emotion of the person shown in the image. an analysis unit, a voice determination unit that determines whether the frequency of the input voice acquired by the voice acquisition unit and the frequency of the comparison voice match, an image acquired by the image acquisition unit, and the image If the image determination unit determines whether the comparison image stored in the storage unit matches, and the audio determination unit determines that the frequency of the input audio and the frequency of the comparison audio do not match; , and/or when it is determined that the image acquired by the image acquisition unit and the comparison image stored in the image storage unit do not match, the morpheme acquired by the speech analysis unit and the dangerous word and whether or not the user of the electronic device is likely to be involved in a crime, based on the matching rate and the degree of risk associated with the age, gender, and emotion acquired by the image analysis unit. and a risk level determination unit that derives a level of risk indicating the level of risk.

FIG. 1 is a schematic diagram showing an overview of an information processing system according to an embodiment. FIG. 2 is a diagram illustrating a configuration example of an information processing system according to an embodiment. FIG. 3 is a diagram illustrating a configuration example of a user terminal according to the embodiment. FIG. 4 is a diagram illustrating an example of the internal configuration of a controller included in a user terminal according to the embodiment. FIG. 5 is a diagram illustrating a configuration example of an information processing device according to an embodiment. FIG. 6 is a diagram illustrating an overview of information stored in the dangerous word storage unit according to the embodiment. FIG. 7 is a diagram illustrating an overview of information stored in the risk storage unit according to the embodiment. FIG. 8 is a diagram illustrating an example of the matching rate of dangerous words according to the embodiment. FIG. 9 is a flowchart illustrating an example of a processing procedure of the information processing system according to the embodiment. FIG. 10 is a flowchart illustrating an example of a processing procedure of the information processing system according to the embodiment. FIG. 11 is a flowchart illustrating an example of a processing procedure of the information processing system according to the embodiment.

Embodiments according to the present application will be described in detail with reference to the drawings. In the following description, similar components may be denoted by the same reference numerals. Furthermore, duplicate explanations may be omitted.

<<1. Introduction >>
Children are easily influenced by their emotions and have difficulty making objective and logical judgments, so when they encounter a situation where there is a possibility of a criminal act such as kidnapping, there is a high possibility that they will not be able to take appropriate action. Furthermore, even if parents are aware of their child's schedule and location, they may not necessarily be aware of their child's current situation, so they may encounter a situation where their child may commit a criminal act such as kidnapping. However, it is difficult to act immediately. Furthermore, even if a criminal act such as kidnapping occurs, administrative agencies such as the police can take action against such criminal act only after it has been reported, so a certain amount of time has passed since the criminal act occurred. As a result, the initial response will be delayed. In this way, it is extremely difficult to prevent children from encountering criminal acts such as kidnapping, but it is important to prevent children from encountering criminal acts as much as possible and take action as quickly as possible against criminal acts. It is desirable to build a crime prevention system that can take the following measures.

Therefore, in this application, we propose an information processing system and electronic equipment that enable investigative agencies to take action as quickly as possible against criminal acts, and that can eradicate children and others who become involved in criminal acts. .

<<2. Overview of information processing system >>
An overview of an information processing system according to an embodiment will be described using FIG. 1. FIG. 1 is a schematic diagram showing an overview of an information processing system according to an embodiment.

The information processing system according to the embodiment shown in FIG. 1 includes a user terminal 1 carried by a child P ₁ and an information processing device 200 that executes processing to prevent the child P ₁ from becoming a victim of crime. .

In FIG. 1, the symbol "P ₁ " indicates a child. The code " _P2 " indicates a third party other than the child and the child's guardian, and may or may not be a suspicious person. The code " _P3 " indicates the child's guardian. Further, in FIG. 1, the symbol "PS" indicates a police station.

The user terminal 1 is a communication device for children, such as a smartphone, which has a communication function. The user terminal 1 may be various terminals such as a mobile phone, a smart device (smartphone or tablet), a wearable terminal, a PDA (Personal Digital Assistant), a portable music player, a game terminal, a laptop personal computer, and the like.

Further, the user terminal 1 has a voice input function and an imaging function. The user terminal 1 can acquire the voice of a third person _P2 who is present around the child _P1 by using the voice input function. Further, the user terminal 1 can obtain an image of a third party _P2 around the child by using the imaging function. Further, the user terminal 1 has a function of outputting a warning sound and a function of displaying that the warning sound is being outputted.

Further, the user terminal 1 can connect to a network such as the Internet and perform information communication with the information processing device 200 to send and receive various information.

The information processing device 200 is an information processing device that executes risk determination processing, and is constructed from a plurality of general-purpose information processing devices such as a server device or a workstation placed on a network such as the Internet. This will be realized using cloud systems, etc.

An example of processing by an information processing system including such a user terminal 1 and information processing device 200 will be described.

The user terminal 1 acquires an input voice (for example, the voice of a third party _P2 ), and determines whether the frequency of the acquired input voice matches the frequency of a comparison voice held in advance (step S1 ). Here, the comparison voice corresponds to, for example, the voices of the guardian _P3 , relatives, etc., recorded in advance and recorded on the user terminal 1.

When the user terminal 1 determines that the frequency of the input voice and the frequency of the comparison voice do not match, the user terminal 1 activates a camera (not shown) and displays the image captured by the camera (for example, the face image of the third party _P2) . image) is acquired (step S2). Then, the user terminal 1 transmits the audio data of the input voice acquired in step S1 and the image data of the image acquired in step S2 to the information processing device 200 (step S3). Further, the user terminal 1 can transmit information on the current location of the child _P1 and terminal information specifying the user terminal 1 to the information processing device 200 (step S3). Information on the current location of the child P ₁ and terminal information specifying the user terminal 1 are enclosed when reporting to the police management device 400 . Examples of the terminal information include a telephone number and an individual identification number assigned to the user terminal 1.

The information processing device 200 executes voice analysis processing based on the voice data received from the user terminal 1 (terminal acquired voice data) (step S4). For example, the information processing device 200 analyzes voice data and divides text data corresponding to the voice data into morphemes.

Furthermore, the information processing device 200 executes image analysis processing based on the image data (terminal acquired image data) (step S5). For example, the information processing device 200 analyzes image data and obtains the age, gender, and emotion of the person shown in the image.

The information processing device 200 derives a risk level indicating whether or not the child P1 who is the user of the user terminal ₁ is likely to be involved in a crime based on the voice analysis processing result and the image analysis processing result. A determination process is executed (step S6). The information processing device 200 analyzes the user terminal based on the match rate between morphemes and risk words obtained through voice analysis processing, and the degree of risk associated with age, gender, and emotion obtained through image analysis processing. A risk level indicating whether there is a high possibility that child _P1 , who is user No. 1, will be involved in a crime is derived. The information processing device 200 includes a risk word storage unit that stores risk words, and a risk storage unit that stores risk levels associated with age, gender, and emotion.

The information processing device 200 determines a process to be performed on at least one of the child _{P 1 ,} the guardian P ₃ , and the police station PS, depending on the risk level of the child P 1 derived through the risk level determination process. do. For example, if the risk level of the third party P ₂ exceeds 90%, the information processing device 200 transmits a message to the guardian device 300 to notify the child P ₁ of the situation, _and The current situation of child _P1 is notified to child P1 (step S7). Further, the information processing device 200 transmits, for example, a message to the police management device 400 to request emergency dispatch, and reports to the police station PS that the child P1 is in a dangerous situation (step S8). This report can include, in addition to the audio data and image data received from the user terminal 1, information on the current location of the user terminal 1 and terminal information that specifies the user terminal 1. Further, the information processing device 200 transmits a command signal to the user terminal 1 to cause the user terminal 1 to output a warning sound (step S9).

In accordance with the command signal received from the information processing device 200, the user terminal 1 outputs a warning sound (step S10) and displays an image indicating that the warning sound is being outputted (step S11).

In this way, in the information processing system according to the embodiment, the information processing device 200 determines the degree of risk of the current situation of the child P ₁ instead of the child P ₁ for whom it is difficult to make an objective and logical judgment. Then, the information processing device 200 notifies the guardian P ₃ of the current situation of the child P ₁ according to the determined risk level, and also reports to the police station PS, which is an administrative agency. Then, the information processing device 200 transmits the audio data and image data acquired by the user terminal 1 when reporting to the police station PS. Therefore, according to the information processing system according to the embodiment, for example, it is possible to make the guardian P 3 aware of the danger of the child P ₁ , and when the child P ₁ is involved in a crime, it is possible to make the guardian P ₃ understand the danger of the child P 1. This information can be provided to the police station PS, which is the investigating agency. This enables the police station PS, which is an investigative agency, to take swift action against criminal acts.

Furthermore, the audio data and image data that the information processing device 200 encloses when reporting to the police station PS can be information for proving a crime. Therefore, if the information processing system according to the embodiment is widely recognized in society, it can serve as a deterrent to deter criminal acts such as child abduction. As a result, it is possible to eradicate children who become involved in criminal acts.

<<3. Functional configuration etc. >>
<3-1. System configuration>
The configuration of the information processing system according to the embodiment will be explained using FIG. 2. FIG. 2 is a diagram illustrating a configuration example of an information processing system according to an embodiment.

As shown in FIG. 2, the information processing system SYS according to the embodiment includes a user terminal 1, an information processing device 200, a guardian device 300, a police management device 400, and a GPS satellite 500. The information processing system SYS operates in cooperation with the user terminal 1 and the information processing device 200, so that the information processing system SYS, on behalf of the user of the user terminal 1 (for example, child P ₁ ), can monitor the risk level of the current situation of the user. Achieve appropriate processing.

The information processing system SYS can include a plurality of user terminals 1 and a guardian device 300. In the example shown in FIG. 2, the information processing system SYS includes user terminals 1 _a to 1 _n , etc. as user terminals 1. The information processing system SYS also includes guardian devices 300 _a to 300 _n as the guardian device 300 . In addition, the information processing system SYS includes GPS satellites 500 _a to 500 _n as GPS satellites 500 . Note that the information processing system SYS may include a plurality of information processing devices 200 or a plurality of police management devices 400.

The user terminal 1 is a communication device for children that has a communication function. The user terminal 1 may be a mobile terminal such as a mobile phone, a smart device (smartphone or tablet), a wearable terminal, a PDA (Personal Digital Assistant), a portable music player, a game terminal, a laptop personal computer, or the like. Further, the user terminal 1 may be an M2M (Machine to Machine) device or an IoT (Internet of Things) device. The user terminal 1 is connected to the network NW and can communicate with other devices via the network NW.

The information processing device 200 is an information processing device that executes risk level determination processing. The information processing device 200 is realized by a general-purpose information processing device such as a server device or a workstation placed on a network NW such as the Internet, a cloud system constructed by a plurality of information processing devices, or the like. The information processing device 200 is connected to the network NW and can communicate with other devices via the network NW.

The guardian device 300 is a communication device held by the guardian P ₃ (see FIG. 1) of the child P ₁ (see FIG. 1). The guardian device 300 may be a device such as a mobile phone, a smart device (smartphone or tablet), a wearable terminal, a PDA (Personal Digital Assistant), a portable music player, a game terminal, a personal computer, or the like. In the embodiment, the guardian device 300 can receive a message from the information processing device 200 and display the received message. The guardian device 300 is connected to the network NW and can communicate with other devices via the network NW.

The police management device 400 is a system managed and operated by the police station PS. The police management device 400 is, for example, a system that quickly and accurately responds to emergency calls from residents and quickly dispatches police cars and police officers to the scene, thereby supporting lifesaving and response to incidents and accidents. including. The police management device 400 is equipped with a pointing device function, etc., and can accurately receive information from callers through high operability and efficient input using a dedicated pen. An effective search function can assist in early identification of the location of occurrence. In the embodiment, the police management device 400 can receive a report from the information processing device 200 and record audio data and image data included in the received report. The police management device 400 is connected to the network NW and can communicate with other devices via the network NW.

The network NW interconnects the user terminal 1, the information processing device 200, the guardian device 300, and the police management device 400. The network NW may include public line networks such as the Internet, telephone line networks, and satellite communication networks, various LANs (Local Area Networks) including Ethernet (registered trademark), WANs (Wide Area Networks), and the like. Further, the network NW may include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network). Further, the network NW may include a wireless communication network such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

The GPS satellite 500 is a satellite that transmits (provides) information that can specify the geographic location of a communication device to a communication device that has a GPS (Global Positioning System) communication function. The communication device (for example, user terminal 1) obtains the information output from the GPS satellite 500 and analyzes it to obtain position information (coordinate information) of itself (user terminal 1). In addition to GPS, the location information may be coordinate information obtained by a positioning function such as Beidou, QZSS (Quasi-Zenith Satellite System), Galileo, or A-GPS (Assisted Global Positioning System). . Typically, information regarding latitude, longitude, altitude, and positioning error may be included. Alternatively, the location information may be registered in an information management device managed by the NRA (National Regulatory Authority) or its commissioned organization, for example. Alternatively, for example, the coordinates may be X-axis, Y-axis, and Z-axis coordinates with the origin at a specific geographic location. Further, an identifier indicating outdoor/indoor may be added together with such coordinate information.

<3-2. User terminal>
FIG. 3 is a diagram illustrating a configuration example of a user terminal according to the embodiment. As shown in FIG. 3, the user terminal 1 includes a touch screen display 2, a button 3, an illuminance sensor 4, a proximity sensor 5, a communication unit 6, a receiver 7, a microphone 8, a storage 9, and a controller. 10, a speaker 11, a camera 12, a camera 13, a connector 14, an acceleration sensor 15, a direction sensor 16, a gyroscope 17, and a GPS receiver 18.

The touch screen display 2 has a display 2A and a touch screen 2B.

The display 2A is a liquid crystal display (LCD), an organic EL display (OELD), or an inorganic EL display (IELD). display devices such as scence Display). The display 2A displays objects such as characters, images, symbols, and figures on the screen.

The touch screen 2B detects contact or proximity of a finger, pen, stylus pen, etc. to the touch screen 2B. The touch screen 2B can detect the position on the touch screen 2B when a plurality of fingers, pens, stylus pens, etc. touch or come close to the touch screen 2B. The touch screen 2B notifies the controller 10 of the contact or proximity of a finger to the touch screen 2B together with the detected position.

The controller 10 detects at least one of the following: contact or proximity detected by the touch screen 2B, a detected position, a change in the detected position, a duration of contact or proximity, an interval at which contact or proximity was detected, and a number of times contact was detected. The type of gesture can be determined based on the Gestures that the controller 10 determines via the touch screen 2B include, for example, touch, long touch, release, swipe, tap, double tap, long tap, drag, flick, pinch in, and pinch out. but not limited to.

The detection method of the touch screen 2B may be any method such as a capacitance method, a resistive film method, a surface acoustic wave method, an infrared method, and a load detection method. In the following description, in order to simplify the description, it is assumed that the user touches the touch screen 2B using a finger in order to operate the user terminal 1.

Button 3 accepts operation input from the user. The number of buttons 3 may be any number. The controller 10 detects an operation on the button 3 by cooperating with the button 3. Operations on the button 3 include, but are not limited to, click, double click, triple click, push, and multi-push, for example. The button 3 may be assigned various functions such as a menu button, a power-on button, a power-off button, a reset button, and the like. The button 3 may be assigned a function of executing voice input processing.

The illuminance sensor 4 detects illuminance. Illuminance is the value of the luminous flux incident on a unit area of the measurement surface of the illuminance sensor 4. The illuminance sensor 4 is used, for example, to adjust the brightness of the display 2A.

The proximity sensor 5 detects the presence of nearby objects in a non-contact manner. The proximity sensor 5 includes a light emitting element that emits infrared light and a light receiving element that receives reflected light of the infrared light emitted from the light emitting element. The illuminance sensor 4 and the proximity sensor 5 may be configured as one sensor.

The communication unit 6 communicates wirelessly. The wireless communication standards supported by the communication unit 6 include, for example, cellular phone communication standards such as 2G, 3G, 4G, and 5G, and short-range wireless communication standards. Communication standards for cellular phones include, for example, LTE (Long Term Evolution), W-CDMA (registered trademark) (Wideband Code Division Multiple Access), CDMA2000, PDC (Personal Digital Cellular), and GSM (registered trademark) (Global System for mobile communications), PHS (Personal Handy-phone System), etc.

Examples of short-range wireless communication standards supported by the communication unit 6 include WiMAX (registered trademark) (Worldwide interoperability for Microwave Access), IEEE802.11, Bluetooth (registered trademark), IrDA (Infrared Data Association), and NFC ( (registered trademark) (Near Field Communication), WPAN (Wireless Personal Area Network), etc. The communication unit 6 may support one or more of the communication standards mentioned above.

The communication unit 6 can connect to the network NW and support a communication protocol for realizing information communication with the information processing device 200. The controller 10 performs information communication to send and receive various information to and from the information processing device 200 via the communication unit 6 . The communication unit 6 transmits the audio data and image data acquired by the controller 10 to the information processing device 200.

Receiver 7 and speaker 11 are sound output units. The receiver 7 and the speaker 11 output the sound signal transmitted from the controller 10 as sound. The receiver 7 is used, for example, to output the other party's voice during a call. The speaker 11 is used, for example, to output ringtones and music. Further, the speaker 11 is used in the embodiment to output a warning sound. One of the receiver 7 and the speaker 11 may also serve as the other. The microphone 8 is a sound input section. The microphone 8 converts the user's voice etc. into a sound signal and transmits it to the controller 10.

Storage 9 stores programs and data. The storage 9 is also used as a work area for temporarily storing processing results of the controller 10. Storage 9 may include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. Storage 9 may include multiple types of storage media. The storage 9 may include a combination of a portable storage medium, such as a memory card, an optical disk, or a magneto-optical disk, and a storage medium reading device. The storage 9 may include a storage device used as a temporary storage area, such as a RAM (Random Access Memory).

Some or all of the programs and data stored in the storage 9 may be downloaded from other devices such as cloud storage that can be accessed via wireless communication by the communication unit 6. Part or all of the programs and data stored in the storage 9 may be stored in a non-transitory storage medium readable by a reading device included in the storage 9. Some or all of the programs and data stored in the storage 9 may be stored in a non-transitory storage medium readable by a reading device connected to the connector 14. Examples of non-transitory storage media include optical disks such as CD (registered trademark), DVD (registered trademark), and Blu-ray (registered trademark), magneto-optical disks, magnetic storage media, memory cards, and solid-state storage media. including but not limited to.

The programs stored in the storage 9 include applications that are executed in the foreground or background, and control programs that support the operation of the applications. For example, the application causes the display 2A to display a screen and causes the controller 10 to execute processing in accordance with a gesture detected via the touch screen 2B. The control program is, for example, an OS. Applications and control programs may be introduced into the storage 9 via wireless communication by the communication unit 6 or via a non-transitory storage medium.

The programs stored in the storage 9 can each provide functions for realizing processing related to various operations of the user terminal 1. The functions provided by the program stored in the storage 9 include the function of adjusting the brightness of the display 2A based on the detection result of the illuminance sensor 4. The functions provided by the program stored in the storage 9 include the function of invalidating the operation on the touch screen 2B based on the detection result of the proximity sensor 5. The functions provided by the programs stored in the storage 9 include the function of controlling communication with external devices connected via the connector 14. The functions provided by the programs stored in the storage 9 include functions for performing various controls such as changing information displayed on the display 2A in accordance with gestures determined based on the detection results of the touch screen 2B. Functions for realizing processes related to various operations of the user terminal 1 may be provided by a control program 9A described later.

The storage 9 stores, for example, a control program 9A, a security application 9B, audio data 9X, image data 9Y, and comparison audio data 9Z.

The control program 9A provides functions for controlling the user terminal 1. By executing the control program 9A, the controller 10 starts or ends an application depending on a detected operation, for example.

The crime prevention application 9B provides various functions for realizing operations related to crime prevention of the user terminal 1. The controller 10 executes operations related to crime prevention of the user terminal 1, which will be described later, by executing the crime prevention application 9B. Note that the control program 9A may take on some of the functions provided by the security application 9B. Furthermore, some of the functions provided by the security application 9B may be implemented as a program created exclusively to implement processing specific to the embodiment, or as an add-on to an existing program or application.

The audio data 9X is audio data input by the microphone 8. The audio data 9X corresponds to, for example, audio data of a third party _P2 speaking to the child _P1 who is the user of the user terminal 1.

The image data 9Y is data of an image captured by the camera 12 or the camera 13. The image data 9Y corresponds to, for example, image data of a face image of a third party _P2 who speaks to the child _P1 who is the user of the user terminal 1.

The comparison audio data 9Z is comparison audio information for comparison with the input audio (audio data 9X). In the comparison audio data 9Z, for example, the frequency of a previously recorded audio is recorded as information on the comparison audio. The comparison voice corresponds to the voice of a person who is safe for the child _P1 , such as the guardian _P3 (see FIG. 1) or a relative.

The program and data shown in FIG. 3 are exemplified as necessary for explaining the embodiment, and do not need to be particularly limited to the example shown in FIG. 3. For example, mail applications that provide e-mail functions for creating, sending, receiving, and displaying e-mails, browser applications that provide web browsing functions for displaying web pages, etc. are stored as necessary. obtain. Further, the data shown in FIG. 3 is an example of what is deemed necessary for explaining the embodiment, and does not need to be particularly limited to the example shown in FIG. 3. For example, setting data (file) including information regarding various settings regarding the operation of the user terminal 1 may be stored.

The controller 10 is realized by, for example, a processor (arithmetic processing unit) such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). For example, the controller 10 is realized by a processor included in the controller 10 executing various programs stored in a storage device inside the user terminal 1 using a RAM or the like as a work area. Note that the controller 10 may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The controller 10 comprehensively controls the operation of the user terminal 1 and implements various processes. Specifically, the controller 10 executes instructions included in a program stored in the storage 9 while referring to data stored in the storage 9 as necessary. The controller 10 controls the functional units according to data and instructions, thereby realizing various functions. The functional units include, for example, the display 2A, the communication unit 6, the receiver 7, and the speaker 11, but are not limited thereto. The controller 10 may change control depending on the detection result of the detection unit. The detection unit includes, for example, the touch screen 2B, the button 3, the illuminance sensor 4, the proximity sensor 5, the microphone 8, the camera 12, the camera 13, the acceleration sensor 15, the direction sensor 16, the gyroscope 17, and the GPS receiver 18. Not limited to these.

The controller 10 controls the operation of the user terminal 1 by executing a program stored in the storage 9. The controller 10 can adjust the brightness of the display 2A based on the detection result of the illuminance sensor 4, for example, by executing the control program 9A. By executing the control program 9A, the controller 10 can disable the operation on the touch screen 2B based on the detection result of the proximity sensor 5, for example. The controller 10 can control communication with an external device connected via the connector 14, for example, by executing the control program 9A. By executing the control program 9A, the controller 10 can perform various controls such as changing the information displayed on the display 2A in accordance with gestures determined based on the detection results of the touch screen 2B.

Further, the controller 10 executes operations related to crime prevention of the user terminal 1 by executing the crime prevention application 9B. FIG. 4 is a diagram illustrating an example of the internal configuration of a controller included in a user terminal according to the embodiment.

As shown in FIG. 4, the controller 10 includes an audio acquisition section 101, an image acquisition section 102, a determination section 103, and a notification section 104. Each block included in the controller 10, that is, the audio acquisition unit 101, the image acquisition unit 102, the determination unit 103, and the notification unit 104, is a functional block that represents a function that is realized when the controller 10 executes the security application 9B. . These functional blocks may be software blocks or hardware blocks. For example, each of the above functional blocks may be one software module realized by software (including a microprogram), or one circuit block on a semiconductor chip (die). Of course, each functional block may be one processor or one integrated circuit. The configuration method and configuration unit of the functional blocks are arbitrary.

The audio acquisition unit 101 records the audio data of the input audio acquired from the microphone 8 in the storage 9 as audio data 9X. The time for recording audio data by the audio acquisition unit 101 can be set arbitrarily. The audio data acquired by the audio acquisition unit 101 may include audio data of a voice uttered by a third party _P2 who is a suspicious person. The audio acquisition unit 101 activates the microphone 8 at a predetermined timing to enable audio input. The predetermined timing can be set arbitrarily. For example, the schedule (time zone) in which the child _P1 goes out can be set as the activation time of the microphone 8. Alternatively, the audio acquisition unit 101 can activate the microphone 8 according to the detection result of the acceleration sensor 15 or activate the microphone 8 based on the position based on the GPS receiver 18. Further, the time for recording audio data by the audio acquisition unit 101 can also be set arbitrarily. The audio acquisition unit 101 sends audio data of the input audio acquired from the microphone 8 to the determination unit 103.

The image acquisition unit 102 acquires image data of an image captured by the camera 12 (or camera 13) according to a signal from the determination unit 103. The image acquisition unit 102 may adjust the imaging direction of the camera 12 (or camera 13), for example, in accordance with the direction in which the third party _P2 is located. For example, when the microphone 8 has directivity, the image acquisition unit 102 regards the audio input direction of the microphone 8 as the direction in which the third party P ₂ is located, and changes the imaging direction of the camera 12 (or camera 13). Adjustable. The image acquisition unit 102 records image data of an image acquired from the camera 12 (or camera 13) in the storage 9 as image data 9Y. The image data acquired by the image acquisition unit 102 may include image data of an image (face image) of a third person P ₂ who is a suspicious person.

The determination unit 103 determines whether the frequency of the input voice acquired by the voice acquisition unit 101 matches the frequency of the comparison voice recorded in the comparison voice data 9Z. Whether or not they match is determined by whether the frequency of the input voice is within a range of ±5% of the frequency of the comparison voice. When determining that the frequencies match, the determining unit 103 discards the audio data 9X recorded in the storage 9. On the other hand, if the determination unit 103 determines that the frequency of the input audio acquired by the audio acquisition unit 101 and the frequency of the comparison audio do not match, the determination unit 103 activates the camera 12 (and/or the camera 13) and instructs imaging. Outputs a signal to The determination unit 103 may start the camera 12 and the camera 13, or may start either the camera 12 or the camera 13. Further, when the determination unit 103 activates the camera 12 (and/or the camera 13), the determination unit 103 may activate either the camera 12 or the camera 13 based on the input direction of the audio. Further, the determination unit 103 may change the direction of the camera 12 (and/or the camera 13) based on the input direction of the voice in accordance with the direction in which the third party _P2 is located.

When the image data is acquired by the image acquisition unit 102, the determination unit 103 acquires the image data acquired by the image acquisition unit 102 and also acquires the audio data 9X from the storage 9. When acquiring the audio data 9X from the storage 9, the determination unit 103 acquires, for example, audio data recorded within 1 second before and after the time when the audio data that does not match the frequency of the comparison audio is recorded. The determination unit 103 sends the audio data acquired from the audio data 9X and the image data acquired from the image acquisition unit 102 to the communication unit 6 as terminal acquired audio and terminal acquired images.

Note that the determination unit 103 can transmit information on the current location of the child P ₁ who is the user of the user terminal 1 and terminal information that specifies the user terminal 1 to the information processing device 200 together with the terminal acquired audio and the terminal acquired image. The determining unit 103 calculates the position of the user (child P ₁ ) based on the signal demodulated by the GPS receiver 18, and specifies the calculated position as the current position of the user. Further, the determination unit 103 can acquire a telephone number, an individual identification number, etc. stored in the storage 9 as terminal information.

The notification unit 104 executes notification processing based on the command signal that the communication unit 6 receives from the information processing device 200. For example, the notification unit 104 outputs a warning sound based on the command signal from the speaker 11, and displays an image indicating that the warning sound is being outputted on the display 2A.

The camera 12 is an in-camera that images an object facing the front face of the user terminal 1. The camera 13 is an out camera that images an object facing the back face of the user terminal 1. The camera 12 and/or the camera 13 captures, for example, an image (face image) of a third person P ₂ around the child P ₁ .

Connector 14 is a terminal to which other devices are connected. The connector 14 is USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), MHL (Mobile High-Definition Link), Light Peak, Thunderbolt (registered trademark), It may be a general-purpose terminal such as a LAN connector or an earphone microphone connector. Connector 14 may be a specially designed terminal such as a dock connector. Devices connected to connector 14 include, but are not limited to, for example, aircraft, chargers, external storage, communication devices, and information processing devices.

Acceleration sensor 15 detects the direction and magnitude of acceleration acting on user terminal 1 . The detection result of the acceleration sensor 15 can also be used to detect the number of steps taken by the user of the user terminal 1. The orientation sensor 16 detects the direction of earth's magnetic field. Gyroscope 17 detects the angle and angular velocity of user terminal 1 . The detection results of the acceleration sensor 15, the orientation sensor 16, and the gyroscope 17 are used in combination to detect changes in the position and orientation of the user terminal 1. The GPS receiver 18 can receive radio signals in a predetermined frequency band from the GPS satellites 500. The GPS receiver 18 demodulates the received radio signal and sends the processed signal to the controller 10.

FIG. 3 shows an example of the configuration of the user terminal 1, and may be modified as appropriate without departing from the gist of the present application. The user terminal 1 illustrated in FIG. 3 includes a camera 12 and a camera 13, but may also include one camera that can be switched between an inside camera and an outside camera, for example. Although the user terminal 1 illustrated in FIG. 3 includes multiple types of sensors to detect the position and orientation of the user terminal 1, it is not necessary to include some of these sensors. Alternatively, the user terminal 1 may include other types of sensors for detecting at least one of the position and orientation of the user terminal 1.

<3-3. Information processing equipment>
FIG. 5 is a diagram illustrating a configuration example of an information processing device according to an embodiment. As shown in FIG. 5, the information processing device 200 includes a communication section 210, a storage section 220, and a control section 230.

The communication unit 210 is an interface such as a NIC (Network Interface Card) for exchanging various information with other devices connected to the network NW. The communication unit 210 supports communication protocols for performing information communication for transmitting and receiving various information among the user terminal 1, the guardian device 300, and the police management device 400. The control unit 230 communicates information with the user terminal 1 , the guardian device 300 , and the police management device 400 via the communication unit 210 .

The storage unit 220 stores programs and data that enable the control unit 230 to perform various processes. The storage unit 24 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 220 can also be used as a work area for temporarily storing processing results of the control unit 230.

As shown in FIG. 5, the storage unit 220 includes a comparison image data storage unit 221, a risk word storage unit 222, a risk level storage unit 223, a voice analysis program 224, an image analysis program 225, and a risk level determination program. 226 and a contact database 227.

The comparison image data storage section 221 is image data for comparison with terminal acquired images. The comparison image data storage unit 221 stores image data of facial images of people who are safe for the child P1, such as the guardian _P3 (see FIG. ₁ ) and relatives.

The dangerous word storage unit 222 stores information on dangerous words. Dangerous words correspond to information such as common phrases used by criminals when kidnapping children. Dangerous words can be set based on information on past incidents. FIG. 6 is a diagram showing an overview of information stored in the dangerous word storage unit 222 according to the embodiment. It is a figure which shows an example of the information of a dangerous word. As shown in FIG. 6, the information on dangerous words stored in the dangerous word storage section 222 is configured by associating dangerous words with synonyms of the dangerous words. Note that FIG. 6 shows an example of information on dangerous words, and there is no need to be particularly limited to the example shown in FIG. 6. For example, synonyms of the dangerous word may be included in the dangerous word, and synonyms and the like may be further associated with each other to configure information on the dangerous word.

The risk storage unit 223 stores risk levels associated with age, gender, and emotion obtained as an analysis result by an image analysis unit 232, which will be described later. FIG. 7 is a diagram showing an overview of information stored in the risk storage unit 223 according to the embodiment. The risk level stored in the risk level storage unit 223 can be set based on criminal tendencies based on information on past incidents. For example, in Figure 7, when comparing gender, the risk level is higher for men, and the risk level is higher for older age groups and "negative" emotions, regardless of gender. It shows. Note that FIG. 7 shows an example of the degree of risk, and there is no need to be particularly limited to the example shown in FIG. For example, the age setting shown in FIG. 7 does not need to be set within the range shown in FIG. 7, and may be set within any range such as every 5 years or every 10 years. Emotions are classified into three categories: positive (favorable, positive emotions), neutral (neutral emotions), and negative (pessimistic, negative emotions). If evaluation is not necessary (for example, if there is almost no risk of responding positively based on crime trends), it may be classified into two categories: neutral and negative.

The speech analysis program 224 provides a function for realizing the processing of the speech analysis section 231, which will be described later. For example, the speech analysis program 224 provides a function of analyzing input speech data (terminal-acquired speech data), converting it into text data, and dividing the text data corresponding to the terminal-acquired speech data into morphemes.

The image analysis program 225 provides functions for realizing the processing of the image analysis section 232, which will be described later. For example, the image analysis program 225 analyzes input image data (terminal acquired image data), acquires the age, gender, and emotion (mental state) of the person shown in the image data, and acquires the age, gender, and emotion (mental state) of the person shown in the image data. Provides a function to output emotional information.

The risk level determination program 226 provides a function for realizing the processing of the risk level determination unit 233, which will be described later. For example, the risk level determination program 226 determines whether the user of the user terminal 1 (for example, a child It is possible to provide a function for deriving the degree of risk indicating whether or not P ₁ ) is likely to be involved in a crime.

The contact database 227 is configured with contact information when notifying the user of the user terminal 1 (for example, child P ₁ ) about the status. For example, the contact information includes the telephone number and e-mail address of the guardian device 300 (see FIG. 1, etc.) used by the guardian P ₃ of the child P ₁ .

The control unit 230 is a controller that controls each unit of the information processing device 200. The control unit 230 is realized by a processor (arithmetic processing unit) such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). For example, the control unit 230 is realized by a processor executing various programs stored in a storage device inside the information processing device 200 using the RAM as a work area. Note that the control unit 230 is realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). CPUs, MPUs, ASICs, and FPGAs can all be considered controllers.

As shown in FIG. 5, the control unit 230 includes a voice analysis unit 231, an image analysis unit 232, a risk determination unit 233, and a communication processing unit 234. Each block included in the control section 230, that is, the voice analysis section 231, the image analysis section 232, the risk level determination section 233, and the communication processing section 234 is a functional block indicating a function realized by the control section 230, respectively. These functional blocks may be software blocks or hardware blocks. For example, each of the above functional blocks may be one software module realized by software (including a microprogram), or one circuit block on a semiconductor chip (die). Of course, each functional block may be one processor or one integrated circuit. The configuration method and configuration unit of the functional blocks are arbitrary.

The audio analysis unit 231 analyzes audio data received from the user terminal 1 (terminal acquired audio data). Specifically, the voice analysis unit 231 analyzes input voice data (terminal acquired voice data), converts it into text data, and divides the text data corresponding to the terminal acquired voice data into morphemes. The voice analysis section 231 outputs the analysis result to the risk determination section 233. Among the morphemes, the speech analysis unit 231 may output morphemes corresponding to predetermined parts of speech (such as nouns and verbs) that can be compared with dangerous words. The voice analysis unit 231 is realized, for example, by a processor executing the voice analysis program 224.

The image analysis unit 232 analyzes image data received from the user terminal 1 (terminal acquired image data). Specifically, the image analysis unit 232 analyzes the input image data (terminal acquired image data) and acquires the age, gender, and emotion (mental state) of the person depicted in the image data. The image analysis section 232 outputs the analysis result to the risk determination section 233. The image analysis unit 232 is realized, for example, by a processor executing the image analysis program 225.

The risk level determination unit 233 determines the user's risk level based on the match rate with dangerous words based on voice analysis by the voice analysis unit 231 and the risk level associated with age, gender, and emotion based on image analysis by the image analysis unit 232. A risk level indicating whether there is a high possibility that the user of the terminal 1 (for example, child P ₁ ) will be involved in a crime is derived.

Specifically, the risk determination unit 233 acquires the comparison image data stored in the comparison image data storage unit 221, and determines whether the image received from the user terminal 1 (terminal acquired image) and the comparison image match. Determine whether or not. Whether or not they match is determined based on whether the matching rate between the terminal acquired image and the comparison image exceeds 95%. If the risk level determination unit 233 determines that the image received from the user terminal 1 (terminal acquired image) and the comparison image do not match, the risk level determination unit 233 refers to the information on the risk word stored in the risk word storage unit 222 and determines whether the voice The match rate between the morphemes included in the speech analysis results obtained from the analysis unit 231 and the risky words is calculated. FIG. 8 is a diagram illustrating an example of the matching rate of dangerous words according to the embodiment. As shown in FIG. 8, for example, when all of the morphemes included in the speech analysis result obtained from the speech analysis section 231 correspond to risky words, the risk determination section 233 calculates the matching rate as 100%. Furthermore, if half of the morphemes included in the speech analysis result obtained from the speech analysis section 231 correspond to risky words, the risk determination section 233 calculates the matching rate as 50%. Furthermore, if one-fourth of the morphemes included in the speech analysis result obtained from the speech analysis section 231 correspond to risky words, the risk determination section 233 calculates the matching rate as 25%. Then, the risk level determination unit 233 compares the matching rate with the dangerous word with a predetermined threshold value, and if the matching rate with the dangerous word exceeds the predetermined threshold value, derives the matching rate with the dangerous word as the risk level based on the voice analysis result.

Further, the risk level determination unit 233 obtains the level of risk corresponding to the image analysis result by the image analysis unit 232 based on the risk level information stored in the risk level storage unit 223 . For example, if the age, gender, and emotion obtained as the image analysis result from the image analysis unit 232 are "18 years old, female, positive," the risk determination unit 233 determines the risk level corresponding to the image analysis result as "0. %”. Then, the risk level determination unit 233 compares the level of risk corresponding to the acquired image analysis result based on the level of risk information stored in the level of risk storage unit 223 with a predetermined threshold value, and if the level of risk exceeds the predetermined threshold value, , the obtained degree of risk is derived as a degree of risk based on the image analysis results.

The risk level determining unit 233 integrates the level of risk based on the voice analysis result and the level of risk based on the image analysis result, and determines the final level of risk. For example, if the risk level based on the voice analysis result is "25%" and the risk level based on the image analysis result is "0%", the risk level determination unit 233 sets the final risk level to "25%". decide.

The risk level determining unit 233 selects at least one of the user terminal 1 (child P ₁ ), the guardian device 300 (guardian P ₃ ), and the police management device 400 (police station PS) according to the determined risk level. Decide what action to take. For example, if the risk level of the third party _P2 exceeds 75%, the information processing device 200 decides to send a message requesting emergency dispatch to the police management device 400, for example. This message can include, in addition to the audio data and image data received from the user terminal 1, information on the current location of the user terminal 1 and terminal information that specifies the user terminal 1. Further, the risk level determining unit 233 decides to transmit a message to the guardian device 300 to notify the child P ₁ of the current situation (danger). Further, the risk level determining unit 233 determines to transmit a command signal to the user terminal 1 to cause the user terminal 1 to output a warning sound.

The communication processing unit 234 executes communication processing for the user terminal 1, the guardian device 300, and the police management device 400 according to the determination by the risk level determination unit 233.

<<4. Processing procedure example >>
An example of a processing procedure of the information processing system according to the embodiment will be described using FIGS. 9 to 11. 9 to 11 are diagrams illustrating an example of the processing procedure of the information processing system according to the embodiment.

<4-1. Processing of the entire system>
First, the overall processing flow of the information processing system SYS will be explained using FIG. As shown in FIG. 9, the user terminal 1 acquires input audio (step S101) and stores the acquired audio data in the storage 9. The user terminal 1 determines whether the frequency of the input voice matches the frequency of the comparison voice (the voice of the guardian P ₃ (see FIG. 1) registered in advance) (step S102). Whether or not they match is determined by whether the frequency of the input voice is within a range of ±5% of the frequency of the comparison voice.

When the user terminal 1 determines that the frequency of the input voice and the frequency of the comparison voice match (step S102; Yes), the process returns to step S101. At this time, the user terminal 1 may discard the audio data of the input audio as is, or may record it in the storage 9 once and discard it after the fact.

On the other hand, if the user terminal 1 determines that the frequency of the input audio and the frequency of the comparison audio do not match (step S102; No), the user terminal 1 activates the camera 12 (and/or the camera 13) and acquires image data ( Step S103).

After acquiring the image data, the user terminal 1 transmits predetermined audio data and image data to the information processing device 200 (step S104). Here, the user terminal 1 stores, as the predetermined audio data, for example, audio recorded within 1 second before and after the time when audio data that does not match the frequency of the comparison audio is recorded from among the audio data stored in the storage 9. Get data. Further, the user terminal 1 can transmit location information indicating the current location of the user terminal 1 and terminal information for specifying the user terminal 1 along with audio data and image data.

When the information processing device 200 receives audio data and image data from the user terminal 1, it performs audio analysis processing and image analysis processing, respectively (steps S105 and S106).

The information processing device 200 derives the risk level by executing a risk level determination process (see FIGS. 10 and 11) based on the voice analysis result and the image analysis result (step S107). Then, the information processing device 200 determines whether the derived risk level is 75% or less (step S108).

If the information processing device 200 determines that the degree of risk is not less than 75% (step S108; No), that is, if the degree of risk exceeds 75%, the information processing device 200 sends a message requesting emergency dispatch to the police station PS in charge. The information is transmitted to the police management device 400 (step S109).

Subsequently, the information processing device 200 sends a message to the guardian device 300 owned by the guardian (for example, the guardian P ₃ ) notifying the child (for example, the child P ₁ ) who is the user of the user terminal 1 of the danger. Transmit (step S110).

Subsequently, the information processing device 200 transmits a command signal for instructing the output of a warning sound to the user terminal 1 (step S111), and ends the processing of the information processing device 200 shown in FIG. 9.

If the information processing device 200 determines that the degree of risk is 75% or less in step S108 (step S108; Yes), it determines whether the degree of risk is 50% or less (step S112).

If the information processing device 200 determines that the degree of risk is not less than 50% (step S1112; No), that is, if the degree of risk is in the range of 51% to 75%, the information processing device 200 performs the processing procedures of steps S110 and S111 above. Execute. That is, the information processing device 200 transmits a message to the guardian device 300 and a command signal to cause the user terminal 1 to output a warning sound, and ends the processing of the information processing device 200 shown in FIG. 9.

On the other hand, if the information processing device 200 determines in step S112 that the risk level is 50% or less (step S112; Yes), it determines whether the risk level is 30% or less (step S113). .

If the information processing device 200 determines that the degree of risk is not less than 30% (step S113; No), that is, if the degree of risk is within the range of 31% to 50%, the information processing device 200 executes the processing procedure of step S111 above. . That is, the information processing device 200 transmits a command signal to cause the user terminal 1 to output a warning sound, and ends the processing of the information processing device 200 shown in FIG. 9 .

On the other hand, if the information processing device 200 determines in step S113 that the risk level is 30% or less (step S113; Yes), it erases the audio data and image data received from the user terminal 1 (step S114). , the processing of the information processing apparatus 200 shown in FIG. 9 ends. Note that if the audio data has been deleted in the risk level determination process based on the audio analysis result, which will be described later with reference to FIG. 10, the information processing device 200 will omit the audio data deletion procedure in step S114. Furthermore, if the image data has been erased in the risk level determination process based on the image analysis result, which will be described later with reference to FIG. 11, the information processing apparatus 200 will omit the image data erase procedure in step S114.

When the user terminal 1 receives a command signal instructing the output of a warning sound from the information processing device 200 (step S115), the user terminal 1 outputs a warning sound from the speaker 11 (step S116).

Then, the user terminal 1 causes the display 2A to display an image indicating that the warning sound is being outputted (step S117), and ends the process of the user terminal 1 shown in FIG. 9.

In the processing procedure shown in FIG. 9, either the audio analysis process in step S105 or the image analysis process in step S106 may be performed first.

<4-2. Voice-based risk determination process>
The flow of the risk level determination process based on the voice analysis result by the information processing device 200 will be explained using FIG. 10 . As shown in FIG. 10, the risk determination unit 233 acquires the voice analysis result by the voice analysis unit 231 (step S201).

Subsequently, the risk level determination unit 233 calculates the match rate between the voice analysis result and the risk word (step S202).

Then, the risk level determining unit 233 determines whether the matching rate calculated in step S202 is 10% or less (step S203).

If the risk determination unit 233 determines that the match rate calculated in step S202 is 10% or less (step S203; Yes), the risk determination unit 233 uses the audio data received from the user terminal 1 and stored in the storage 9 (terminal acquired audio data). ) (step S204), and the risk level determination process based on the voice analysis results shown in FIG. 10 is ended.

On the other hand, if the risk determination unit 233 determines that the match rate calculated in step S202 is not less than 10% (step S203; No), it outputs the match rate as the risk level (step S205), and The risk level determination process based on the voice analysis result shown in FIG.

<4-3. Image-based risk determination process>
The flow of risk determination processing based on the image analysis result by the information processing device 200 will be described using FIG. 11. As shown in FIG. 11, the risk determination unit 233 determines whether the terminal acquired image, which is the image received from the user terminal 1, matches the comparison image registered in advance in the comparison image data storage unit 221. Determination is made (step S301).

If the risk level determination unit 233 determines that the terminal acquired image and the comparison image do not match (step S301; No), the risk level determination unit 233 acquires the level of risk based on the image analysis result from the risk level storage unit 223 (step S302).

The risk level determining unit 233 determines whether the risk level acquired in step S302 is 10% or less (step S303).

If the risk level determination unit 233 determines that the level of risk acquired in step S302 is 10% or less (step S303; Yes), the risk level determination unit 233 uses the image data (terminal acquired image data) received from the user terminal 1 and stored in the storage 9. ) (step S304), and the risk level determination process based on the image analysis results shown in FIG. 11 is completed.

On the other hand, if the risk determination unit 233 determines that the risk is not 10% or less (step S303; No), it outputs the risk obtained in step S302 (step S305), and performs the image analysis shown in FIG. The risk determination process based on the result is ended.

The risk level determination process based on the voice analysis result described above using FIG. 10 and the risk level determination process based on the voice analysis result described above using FIG. 11 may be performed in an integrated manner. That is, when the risk level based on the voice analysis result exceeds 10%, the risk level determination unit 233 outputs the matching rate as the level of risk, and executes the risk level determination process based on the image analysis result shown in FIG. You can do it like this.

As described above, in the information processing system SYS according to the embodiment, the information processing device 200 uses a user ( _{child P1} ) Determine the degree of risk of the current situation. Then, the information processing device 200 notifies a predetermined contact point (for example, a guardian P 3 ) of the current situation of the user (child P ₁ ) according to the determined degree of risk, and also notifies the police station, which is the investigating agency, of the current situation of the user (child P ₁ ). Report to PS. Then, the information processing device 200 transmits the audio data and image data acquired by the user terminal 1 when reporting to the police station PS. Thereby, the information processing device 200 performs audio analysis processing and image analysis processing based on the audio and images from the user terminal 1, and can determine the degree of risk for the user of the user terminal 1. Therefore, according to the information processing system SYS according to the embodiment, for example, it is possible to make the guardian P 3 understand the danger of the child P ₁ , and when the child P ₁ is involved in a crime, it is possible to make the guardian P ₃ understand the danger of the child P 1. This information can be provided to the police station PS, which is the investigating agency. This enables investigative agencies to take swift action against criminal acts.

Furthermore, the audio data and image data that the information processing device 200 encloses when reporting to the police station PS can be information for proving a crime. Therefore, the information processing system SYS according to the embodiment is widely recognized in society, which can serve as a deterrent against criminal acts such as child abduction. As a result, it is possible to eradicate children who become involved in criminal acts.

<<5. Modified example >>

In the embodiment described above, the risk level determination process based on the image analysis result by the risk level determination unit 233 may derive the level of risk by further taking into account information about the surrounding environment obtained by analyzing the image data acquired by the terminal. . For example, if the surrounding environment deviates from the normal action range of the child P1 who is the user of the user terminal ₁ , the risk level will increase according to the image analysis result obtained from the risk level storage unit 223. Weighting may be performed at different times. Such processing can be realized by adding a function to the image analysis program 225 that analyzes the surrounding environment of image data and outputs information indicating the surrounding environment.

Furthermore, in the embodiment described above, the risk level determination process based on the image analysis result by the risk level determination unit 233 derives the level of risk by further taking into consideration the degree of blurring of the image obtained by analyzing the image data acquired by the terminal. Good too. For example, if the degree of blur in the image exceeds a predetermined threshold, that is, if it can be determined that the situation is not in normal use, the risk level will be increased based on the image analysis results obtained from the risk level storage unit 223. The degree of risk may be weighted. Such processing can be realized by adding a function to the image analysis program 225 to analyze image blur and output the specific image blur.

Furthermore, in the above embodiment, the risk level determination process based on the voice analysis result and the image analysis result executed in the information processing device 200 is the same as the risk level determination process based on the voice analysis result or the risk level determination process based on the image analysis result. Either may be implemented.

For example, when the information processing device 200 performs only the risk level determination process based on the voice analysis result, if the level of risk based on the voice analysis result exceeds a threshold value, the information processing device 200 transmits a message to the guardian device 300 and the police management device 400. , a command signal instructing the user terminal 1 to output a warning sound can be transmitted.

Further, for example, when the information processing device 200 only performs a risk level determination process based on the image analysis result, if the risk level corresponding to the image analysis result exceeds a threshold value, the information processing device 200 may send information to the guardian device 300 and the police management device 400. It is possible to send a message and send a command signal that instructs the user terminal 1 to output a warning sound.

Furthermore, in the embodiments described above, it is conceivable that the voice (frequency) of the guardian, etc. changes with the age of the guardian. Therefore, for example, if the voice (frequency) of the guardian of the user of the user terminal 1 (for example, child P ₁ ) is registered in advance as the comparison voice data 9Z, the user terminal 1 Changes in voice depending on age may be predicted, and the voice based on the prediction result may be used for comparison with input voice. Prediction of changes in voice according to age may be performed, for example, by machine learning using a trained model that learns the relationship between changes in human voice and age and predicts changes in voice according to age. However, the processing may be performed by processing the voice using parameters obtained in advance.

Furthermore, in the embodiments described above, it is conceivable that the face image of the guardian, etc. also changes with the age of the guardian. Therefore, for example, if the face image of the guardian of the user of the user terminal 1 (for example, child P ₁ ) is registered in the comparison image data storage unit 221 in advance, the information processing device 200 protects the voice of the guardian. Changes in the facial image according to the person's age may be predicted, and the facial image based on the prediction result may be used for comparison with the terminal-acquired image. Prediction of changes in facial images according to age is performed using, for example, a trained model that uses machine learning to learn the relationship between changes in human facial images and age, and predicts changes in facial images according to age. Alternatively, the processing may be performed by processing a face image using parameters obtained in advance.

In addition, the risk level determination process based on the voice analysis result and the image analysis result executed in the information processing device 200 is performed in the user terminal 1. may be implemented.

For example, when the user terminal 1 performs a risk determination process based on the audio analysis result and the image analysis result, an electronic device functioning as the user terminal 1 has an audio input device arranged to input sounds surrounding the electronic device. an imaging unit arranged to be able to capture an image of the surroundings of the electronic device; an audio acquisition unit that acquires input audio input by the audio input unit; and an audio acquisition unit that acquires the image captured by the imaging unit. an image acquisition unit; an audio storage unit that stores in advance information on a comparison audio for comparison with the input audio; and an image storage unit that stores in advance information on a comparison image for comparison with the image; A risk word storage unit that stores information on risk words, a risk level storage unit that stores risk levels associated with age, gender, and emotion, and analyzes input voice data acquired by the voice acquisition unit. , a voice analysis unit that classifies and acquires text data corresponding to the input voice data into morphemes; and a voice analysis unit that analyzes the image data acquired by the image acquisition unit and determines the age and gender of the person shown in the image. , an image analysis unit that acquires the emotion, a voice determination unit that determines whether the frequency of the input voice acquired by the voice acquisition unit and the frequency of the comparison voice match, and the image acquisition unit acquires the emotion. an image determining unit that determines whether the input image matches the comparison image stored in the image storage unit; and an audio determining unit that determines whether the input audio frequency and the comparison audio frequency match. If it is determined that they do not match, and/or if it is determined that the image acquired by the image acquisition unit and the comparison image stored in the image storage unit do not match, the image is acquired by the voice analysis unit. The user of the electronic device is involved in a crime based on the matching rate between the morpheme and the dangerous word, and the degree of risk associated with the age, gender, and emotion acquired by the image analysis unit. What is necessary is to include a risk level determination unit that derives a risk level indicating whether or not the possibility is high.

The voice input section can be realized by the microphone 8 included in the user terminal 1. The imaging unit can be realized by cameras 12 and 13 included in the user terminal 1. The audio storage unit and the image storage unit can be realized by the storage 9 included in the user terminal 1. The audio acquisition unit, image acquisition unit, audio analysis unit, image analysis unit, determination unit, and risk determination unit can be realized by the controller 10 included in the user terminal 1. For example, this can be realized by a processor included in the controller 10 executing a program for realizing the processing functions of a voice acquisition section, a voice analysis section, an image analysis section, a determination section, and a risk determination section.

Further, for example, when the user terminal 1 performs a risk determination process based on the voice analysis result, the electronic device functioning as the user terminal 1 includes the voice input section, the voice acquisition section, the voice storage section, It is sufficient to include the risk word storage section, the voice analysis section, the voice determination section, and the risk level determination section. In this case, the risk level determination unit derives the match rate and risk level between the morpheme acquired by the speech analysis unit and the risk word.

Further, for example, when the user terminal 1 performs a risk determination process based on the image analysis result, an electronic device functioning as the user terminal 1 includes the above-mentioned imaging section, the above-mentioned image acquisition section, the above-mentioned image storage section, and the above-mentioned It is sufficient to include a risk storage section, the image determination section, and the risk determination section. In this case, the risk level determination unit acquires and derives the risk level associated with the age, gender, and emotion acquired by the image analysis unit from the risk level storage unit.

<<6. Others＞＞
The information processing device 200 according to the embodiment may be implemented with a dedicated computer system or a general-purpose computer system.

The program executed by the information processing device 200 may be stored in a disk device included in a server device on a network such as the Internet, so that the information processing device 200 can download the program. Further, the above-described processing functions executed by the information processing device 200 may be realized by collaboration between an OS (Operating System) and application software. In this case, the parts other than the OS may be stored on a medium and distributed, or the parts other than the OS may be stored in a server device so that the information processing device 200 can download them.

Further, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured.

Furthermore, the embodiments described above can be combined as appropriate in areas where the processing contents do not conflict. Further, the order of each step shown in each figure showing the processing procedure according to the embodiment can be changed as appropriate.

Although each embodiment of the present disclosure has been described above, the technical scope of the present disclosure is not limited to each of the above-mentioned embodiments as is, and various changes can be made without departing from the gist of the present disclosure. be. Furthermore, components of different embodiments and modifications may be combined as appropriate.

Moreover, the effects in each embodiment described in this specification are merely examples and are not limited, and other effects may also be provided.

1 User terminal 2 Touch screen display 2A Display 2B Touch screen 3 Button 4 Illuminance sensor 5 Proximity sensor 6 Communication unit 7 Receiver 8 Microphone 9 Storage 9A Control program 9B Security application 9X Audio data 9Y Image data 9Z Comparison audio data 10 Controller 11 Speaker 12 , 13 Camera 14 Connector 15 Acceleration sensor 16 Direction sensor 17 Gyroscope 18 GPS receiver 200 Information processing device 210 Communication section 220 Storage section 221 Comparison image data storage section 222 Danger word storage section 223 Risk storage section 224 Voice analysis program 225 Image analysis Program 226 Risk level determination program 227 Contact database 230 Control unit 231 Voice analysis unit 232 Image analysis unit 233 Risk level determination unit 234 Contact processing unit 300 Guardian device 400 Police management device

Claims (5)

An information processing system including a user terminal and an information processing device that processes data acquired by the user terminal,
The user terminal is
a communication unit that transmits and receives various information to and from the information processing device;
an audio input unit arranged to be able to input audio surrounding the user terminal;
an imaging unit arranged to be able to capture an image of the surroundings of the user terminal;
an audio acquisition unit that acquires the input audio input by the audio input unit;
an image acquisition unit that acquires an image captured by the imaging unit;
a storage unit that stores in advance information on comparison audio for comparison with the input audio;
It is determined whether the frequency of the input voice and the frequency of the comparison voice match, and if it is determined that the frequency of the input voice and the frequency of the comparison voice do not match, the a determination unit that transmits audio data of the input audio and image data of the image acquired by the image acquisition unit to the information processing device as terminal acquired audio data and terminal acquired image data, respectively;
The information processing device includes:
a communication unit that receives the terminal acquired audio data and the terminal acquired image data from the user terminal;
a comparison image data storage unit that stores comparison image data for comparison with the terminal acquired image data;
a dangerous word storage unit that stores information about dangerous words;
a risk level storage unit that stores risk levels associated with age, gender, and emotion;
a speech analysis unit that analyzes the terminal-obtained speech data and divides text data corresponding to the terminal-obtained speech data into morphemes and obtains the text data;
an image analysis unit that analyzes the terminal-obtained image data and obtains the age, gender, and emotion of the person shown in the terminal-obtained image;
If the terminal acquired image data and the comparison image data do not match, the match rate between the morpheme acquired by the voice analysis unit and the dangerous word, and the age, gender, and emotion acquired by the image analysis unit an information processing system comprising: a risk level determining unit that derives a risk level indicating whether or not a child who is a user of the user terminal is likely to be involved in a crime based on a risk level associated with the crime; The risk determination unit includes:
The information processing system according to claim 1, wherein when it is determined that the degree of risk exceeds a predetermined threshold, a predetermined signal instructing the output of a warning sound is transmitted to the user terminal via the communication unit. It further includes a contact database for pre-registering predetermined contact information,
The risk determination unit includes:
If it is determined that the degree of risk exceeds a predetermined threshold, a notification indicating the status of the child who is the user of the user terminal is sent to the contacts registered in the contact database via the communication unit. The information processing system according to claim 2. The user terminal is
a sound output unit that outputs an alarm sound in response to reception of the predetermined signal;
The information processing system according to claim 2 or 3, further comprising: a display unit that displays an image indicating that an alarm sound is being outputted in response to reception of the predetermined signal. An electronic device that performs processing based on audio and images,
an audio input unit arranged to be able to input audio surrounding the electronic device;
an imaging unit arranged to be able to capture an image of the surroundings of the electronic device;
an audio acquisition unit that acquires the input audio input by the audio input unit;
an image acquisition unit that acquires an image captured by the imaging unit;
an audio storage unit that stores in advance information on comparison audio for comparison with the input audio;
an image storage unit that stores in advance information on a comparison image for comparison with the image;
a dangerous word storage unit that stores information about dangerous words;
a risk level storage unit that stores risk levels associated with age, gender, and emotion;
a speech analysis unit that analyzes input speech data acquired by the speech acquisition unit, classifies text data corresponding to the input speech data into morphemes, and obtains the text data;
an image analysis unit that analyzes the data of the image acquired by the image acquisition unit and acquires the age, gender, and emotion of the person shown in the image;
a voice determination unit that determines whether the frequency of the input voice acquired by the voice acquisition unit and the frequency of the comparison voice match;
an image determination unit that determines whether the image acquired by the image acquisition unit and the comparison image stored in the image storage unit match;
If the audio determination unit determines that the frequency of the input audio and the frequency of the comparison audio do not match, and/or the image acquired by the image acquisition unit and the image stored in the image storage unit If it is determined that the comparison images do not match, the morpheme obtained by the speech analysis unit and the risk word are matched with each other, and the age, gender, and emotion obtained by the image analysis unit are associated with each other. an electronic device, comprising: a risk level determination unit that derives a risk level indicating whether or not a user of the electronic device is likely to be involved in a crime, based on the determined risk level.

Images