JP2019200607A - Information processing system and information processing method - Google Patents

Abstract

To provide an information processing system which uses a robot that makes many people feel familiar.SOLUTION: In an information processing system 10 including a robot 20 and an information processing device 40, the information processing device 40 includes: a camera 49; a first transmission unit configured to transmit data related to an image captured by the camera 49 to a network; a first receiving unit configured to receive response data created based on the data related to the image transmitted from the first transmission unit; and a second transmission unit configured to transmit voice data to the robot 20 based on the response data received by the first receiving unit. The robot 20 includes a third receiving unit configured to receive the voice data transmitted from the second transmission unit, and a speaker that outputs voices based on the voice data received by the third receiving unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing system and an information processing method.

  A mobile robot dialogue system using a service robot that operates in the same space as humans based on dialogue and information obtained from environmental sensors is proposed (Patent) Reference 1). The service robot has a size close to that of a human being and operates autonomously based on a scenario given in advance while grasping an environmental situation in real time.

JP-A-2018-67100

  However, depending on the user, there may be a sense of resistance or fear that a service robot having a size close to that of a human will operate independently. In one aspect, an object is to provide an information processing system using a robot that makes many people feel familiar.

  In an information processing system including a robot and an information processing device, the information processing device is transmitted from a camera, a first transmission unit that transmits data related to an image captured by the camera to a network, and the first transmission unit. A first reception unit that receives response data created based on the data related to the image, and a second transmission unit that transmits voice data to the robot based on the response data received by the first reception unit. The robot includes a third receiving unit that receives the audio data transmitted by the second transmitting unit, and a speaker that outputs audio based on the audio data received by the third receiving unit.

  In one aspect, an information processing system using a robot that makes many people feel friendly can be provided.

It is explanatory drawing explaining the outline | summary of operation | movement of an information processing system. It is a perspective view of a robot. It is a perspective view of a robot. It is a perspective view of a charging stand. It is explanatory drawing explaining the internal structure of a robot. It is explanatory drawing explaining operation | movement of a robot. It is explanatory drawing explaining the structure of an information processing system. It is explanatory drawing explaining the structure of an information processing system. It is explanatory drawing explaining the record layout of log | history DB. It is explanatory drawing explaining the record layout of conversation log DB. It is explanatory drawing explaining the record layout of bot DB. It is explanatory drawing explaining the example of a report. It is a flowchart explaining the flow of a process of a program. 10 is a flowchart for explaining a processing flow of a program according to the second embodiment. FIG. 10 is an explanatory diagram for explaining an outline of the operation of the guidance system according to the third embodiment. It is explanatory drawing explaining the record layout of coupon DB. FIG. 10 is a functional block diagram of an information processing system according to a fourth embodiment. FIG. 10 is an explanatory diagram illustrating a configuration of an information processing system according to a fifth embodiment.

[Embodiment 1]
FIG. 1 is an explanatory diagram for explaining an outline of the operation of the information processing system 10. In the following description, the information processing system 10 that is installed at a commercial facility information center and guides a store to a user such as a shopper will be described as an example. The installation location of the information processing system 10 is not limited to a commercial facility. The information processing system 10 can be installed in an arbitrary place such as a transportation facility such as a station or a tourist facility.

  The information processing system 10 includes a robot 20 and an information processing device 40. The information processing device 40 is a general-purpose information processing device such as a tablet or a personal computer, and is connected to the server 50 via a network. The information processing device 40 includes a display device 47 and a camera 49 that include a display panel such as a liquid crystal display panel or an organic EL (electro-luminescence) display panel.

  The exterior of the robot 20 has a shape that makes you feel the familiarity of a bird. The robot 20 includes a microphone 31 (see FIG. 2) and a speaker 32 (see FIG. 2). The robot 20 is smaller than the display device 47. More specifically, the height of the robot 20, that is, the length in the vertical direction is shorter than the length of the short side of the display device 47. The robot 20 is connected to the information processing apparatus 40 via short-range wireless communication such as Bluetooth (registered trademark).

  The robot 20 is placed on a substantially disc-shaped charging stand 36 that imitates a stump. The robot 20 is operated by power supplied through the charging stand 36 or a battery built in the robot 20.

  The user approaches the robot 20 and speaks, for example, “Is there a cake shop?” The microphone 31 detects voice, converts it into voice data that is an electrical signal, and transmits the voice data to the information processing apparatus 40 via short-range wireless communication. The camera 49 photographs the user. The camera 49 is arranged at a position where the user's face facing the robot 20 can be photographed. Audio data and an image captured by the camera 49 are transmitted from the information processing apparatus 40 to the server 50.

  The server 50 creates an appropriate answer based on the image and audio data, and transmits it to the information processing apparatus 40. Specifically, attributes such as the sex and age of the user are determined based on the image. A voice chat bot with a pseudo personality corresponding to a user attribute is selected from a plurality of voice chat bots with a pseudo personality defined in advance.

  Voice chatbot extracts “cake shop” recommended for users from multiple “cake shop” in the commercial facility, and includes voice data and images associated with the “cake shop” Send. Note that the function of the voice chat bot for interacting with the user in a natural language may be installed in the server 50 or provided by a cloud service. Since the voice chat bot is publicly known, a detailed description thereof is omitted.

  The information processing device 40 displays a video on the display device 47. The video is a still image or a moving image. The information processing apparatus 40 transmits voice data to the robot 20 via short-range wireless communication. The robot 20 emits sound based on the sound data from the speaker 32.

  By determining a user attribute based on an image photographed by the camera 49 and selecting a pseudo personality, it is possible to provide the information processing system 10 that provides information suitable for the user in an expression suitable for the user. By using the small robot 20 shaped like a bird, even a user who is not good at computers can speak without resistance and obtain necessary information.

  2 and 3 are perspective views of the robot 20. The robot 20 includes a substantially cylindrical body 201 and a substantially hemispherical head 202 that covers one end surface of the body 201. Two first light emitting units 251 and a speaker 32 are arranged on the head 202. The first light emitting unit 251 has an appearance like the eyes of the robot 20.

  A circular human sensor 331 is provided in front of the body portion 201. A circular second light emitting unit 252 is disposed below the human sensor 331. The microphone 31 is disposed below the second light emitting unit 252. Blade-type touch sensors 332 are arranged on the left and right sides of the body portion 201. The human sensor 331 and the touch sensor 332 are examples of the sensor 33 provided in the robot 20. Two legs 27 simulating bird legs are provided at the bottom of the body part 201.

  A connector 29 is disposed at a lower part on the rear side of the body portion 201. The connector 29 is, for example, a USB (Universal Serial Bus) connector. Under the connector 29, two chain holes 35 are provided. An antitheft chain (not shown) can be attached to the chain hole 35.

  Three switches 26 and two second charging terminals 28 are arranged on the lower surface of the body portion 201. By operating the switch 26 alone or in combination, the power of the robot 20 is turned on / off, the sensitivity of the microphone 31 is adjusted, the volume of the speaker 32 is adjusted, and the light intensity of the first light emitting unit 251 and the second light emitting unit 252 Adjustments can be made.

  FIG. 4 is a perspective view of the charging stand 36. The charging stand 36 has a substantially disc shape and has a substantially circular recess 361 on one surface. A first charging terminal 38 corresponding to the second charging terminal 28 is disposed at the center of the recess 361. A leg insertion portion 362 corresponding to the leg portion 27 is provided at the edge of the recess 361.

  A connector 39 is disposed on the side surface of the charging stand 36. The connector 39 is a USB connector, for example, and can be connected to an AC (Alternating Current) adapter.

  By placing the robot 20 on the charging base 36 with the legs 27 of the robot 20 and the leg insertion portions 362 of the charging base 36 aligned, a part of the robot 20 is placed in the recess 361 of the charging base 36. Inserted. The first charging terminal 38 and the second charging terminal 28 come into contact with each other, and power is supplied to the robot 20. In the state where the directions of the leg portion 27 and the leg insertion portion 362 do not match, the first charging terminal 38 and the second charging terminal 28 do not come into contact with each other, so that erroneous connection can be prevented.

  FIG. 5 is an explanatory diagram for explaining the internal structure of the robot 20. An outline of the structure of the actuator 34 disposed inside the head 202 will be described with reference to FIG. The actuator 34 includes a magnet 341, a coil 342, and a rotation shaft 343. The rotation shaft 343 is provided along the central axis of the body portion 201 and protrudes from the end surface of the body portion 201 into the head 202.

  The magnet 341 has a substantially rectangular parallelepiped shape that is long in the left-right direction of the robot 20. The magnet 341 is fixed inside the head 202 and can rotate with respect to the rotation shaft 343. The magnet 341 has an N pole and an S pole at the end in the longitudinal direction.

  A coil 342 is disposed on the rear side of the magnet 341. The coil 342 is fixed to the body portion 201. Two stoppers 344 are fixed to the end surface of the body portion 201.

  By applying a direct current to the coil 342, the magnet 341 rotates in the first direction. By reversing the direction of the direct current, the magnet 341 rotates in the second direction. The rotation range of the magnet 341 is limited to a predetermined range by the stopper 344.

  FIG. 6 is an explanatory diagram for explaining the operation of the robot 20. As the magnet 341 rotates, the head 202 of the robot 20 rotates relative to the body portion 201. Thereby, the swing motion of the robot 20 is performed.

  7 and 8 are explanatory diagrams for explaining the configuration of the information processing system 10. The configuration of the information processing apparatus 40 and the robot 20 will be described with reference to FIG.

  The information processing apparatus 40 includes a CPU (Central Processing Unit) 41, a main storage device 42, an auxiliary storage device 43, a first communication unit 46, a second communication unit 44, a touch panel 45, a camera 49, and a bus. The CPU 41 is an arithmetic control device that executes the program of the present embodiment. As the CPU 41, one or a plurality of CPUs or a multi-core CPU is used. The CPU 41 is connected to each hardware part constituting the information processing apparatus 40 via a bus.

The main storage device 42 is a storage device such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), or a flash memory. The main storage device 42 temporarily stores information necessary during the processing performed by the CPU 41 and a program being executed by the CPU 41.

  The auxiliary storage device 43 is a storage device such as SRAM, flash memory, or hard disk. The auxiliary storage device 43 stores a program to be executed by the CPU 41 and various data necessary for executing the program.

  The first communication unit 46 includes a first transmission unit 461 and a first reception unit 462. The first communication unit 46 is an interface that performs data communication between the information processing apparatus 40 and the network. The second communication unit 44 includes a second transmission unit 441 and a second reception unit 442. The second communication unit 44 is an interface that performs short-range wireless communication between the information processing apparatus 40 and the robot 20.

  The touch panel 45 includes a display unit 451 such as a liquid crystal display panel and an input unit 452 stacked on the display unit 451. A display device 47 having a touch panel 45 may be provided separately from the information processing device 40 and may be connected to the information processing device 40 by wire or wirelessly.

  The camera 49 can shoot still images and moving images based on control by the CPU 41. The camera 49 is provided separately from the information processing apparatus 40 and may be connected to the information processing apparatus 40 by wire or wirelessly.

  The robot 20 includes a CPU 21, a main storage device 22, an auxiliary storage device 23, a third communication unit 24, a light emitting unit 25, a microphone 31, a speaker 32, a sensor I / F 330, a sensor 33, an actuator I / F 340, and a bus. The CPU 21 is an arithmetic control device that executes the program of the present embodiment. As the CPU 21, one or a plurality of CPUs or a multi-core CPU is used. The CPU 21 is connected to each hardware part constituting the robot 20 via a bus.

  The main storage device 22 is a storage device such as SRAM, DRAM, or flash memory. The main storage device 22 temporarily stores information necessary during the processing performed by the CPU 21 and a program being executed by the CPU 21.

  The auxiliary storage device 23 is a storage device such as SRAM, flash memory, or hard disk. The auxiliary storage device 23 stores a program to be executed by the CPU 21 and various data necessary for executing the program.

  The third communication unit 24 includes a third transmission unit 241 and a third reception unit 242. The third communication unit 24 is an interface that performs short-range wireless communication with the information processing apparatus 40. The light emitting unit 25 includes the first light emitting unit 251 and the second light emitting unit 252 described above. The light emitting unit 25 is, for example, an LED (Light Emitting Diode), and emits light with luminance and color based on the control of the CPU 21.

  The sensor I / F 330 is an interface that connects the sensor 33 and the bus. The sensor 33 includes a gyro sensor 333 built in the body portion 201 in addition to the human sensor 331 and the touch sensor 332 described above. The CPU 21 detects the approach or contact of the user via the sensor 33 and the sensor I / F 330.

  The actuator I / F 340 is an interface that connects the actuator 34 and the bus. As described above, the actuator 34 includes the magnet 341 and the coil 342. Based on the control by the CPU 21, a direct current is supplied to the coil 342 and the robot 20 swings.

  The CPU 21 performs light emission or swinging motion of the light emitting unit 25 at random while detecting the approach or contact of the user by the sensor 33, for example. By operating at random, it is possible to give the user an impression of a living creature and feel familiarity. The CPU 21 may perform the light emission or swing motion of the light emitting unit 25 in conjunction with the sound detected by the microphone 31 or the sound output from the speaker 32. The CPU 21 may perform light emission or swing motion of the light emitting unit 25 based on a predetermined pattern.

  The user feels that the robot 20 is reacting to the user by the light emission and the swing motion. As a result, the user feels familiar with the robot 20 and can easily ask questions. The shape of the robot 20 may be an arbitrary shape such as a cat shape, a dog shape, or a human shape. The place where the actuator 34 operates may be any place other than the neck. A plurality of locations of the robot 20 may be operated by the actuator 34.

  The configuration of the server 50 will be described with reference to FIG. The server 50 includes a CPU 51, a main storage device 52, an auxiliary storage device 53, a fourth communication unit 54, and a bus. The CPU 51 is an arithmetic control device that executes the program of the present embodiment. As the CPU 51, one or a plurality of CPUs or a multi-core CPU is used. The CPU 51 is connected to each hardware unit constituting the server 50 via a bus.

  The main storage device 52 is a storage device such as SRAM, DRAM, or flash memory. The main storage device 52 temporarily stores information necessary during processing performed by the CPU 51 and a program being executed by the CPU 51.

  The auxiliary storage device 53 is a storage device such as SRAM, flash memory, or hard disk. The auxiliary storage device 53 stores a history DB (Database) 61, a conversation log DB 62, a bot DB 63, a program to be executed by the CPU 51, and various data necessary for executing the program.

  The history DB 61, the conversation log DB 62, and the bot DB 63 may be stored in the auxiliary storage device 43 of the information processing device 40. The history DB 61, the conversation log DB 62, and the bot DB 63 may be stored in an external mass storage device or the like connected to the server 50 or the information processing device 40.

  The fourth communication unit 54 includes an information transmission unit 541 and an information reception unit 542. The fourth communication unit 54 is an interface that performs data communication between the server 50 and the network. The server 50 may be a virtual machine that operates on a large computer. The server 50 may be configured by combining a plurality of computers or server machines.

  FIG. 9 is an explanatory diagram for explaining the record layout of the history DB 61. The history DB 61 is a DB that records the start time of the conversation between the information processing system 10 and the user, the user attributes, the conversation log file name, the conversation duration, and the guidance destination that is guided to the user in association with each other. The history DB 61 has a robot ID field, a start date / time field, an attribute field, a conversation log ID field, a conversation duration field, and a guidance destination field. The attribute field has a gender field and an age field.

  In the robot ID field, a robot ID (Identifier) uniquely assigned to the robot 20 corresponding to the user is recorded. In the start date / time field, the date / time when the user started the conversation with the information processing system 10 is recorded. In the sex field, the gender of the user estimated based on the image taken by the camera 49 is recorded. In the age field, the age of the user estimated based on the image taken by the camera 49 is recorded.

  In the conversation log ID field, a conversation log ID uniquely assigned to a conversation log exchanged between the user and the information processing system 10 is recorded. The duration of conversation between the user and the information processing system 10 is recorded in the conversation duration field. In the guidance destination field, a place where the information processing system 10 has guided the user in response to a question from the user is recorded. The history DB 61 has one record for one conversation between the user and the robot 20.

  FIG. 10 is an explanatory diagram for explaining the record layout of the conversation log DB 62. The conversation log DB 62 is a DB that records the conversation log ID, the date and time when each utterance was performed, the speaker, the utterance content, the screen displayed on the display device 47, and the user's reaction in association with each other. . The conversation log DB 62 has a conversation log ID field, a date / time field, a speaker field, an utterance content field, a display screen field, and a user response field. The user response field has a smile field and a whisper field.

  A conversation log ID is recorded in the conversation log ID field. In the date / time field, the date / time when the utterance was performed is recorded. In the speaker field, it is recorded whether the user or the robot 20 has spoken. The utterance content of the user or the robot 20 is recorded in the utterance content field.

  In the display screen field, the screen displayed on the display device 47 is recorded by, for example, an HTML (HyperText Markup Language) file. In this embodiment, since the display screen is updated when the robot 20 speaks, information is recorded in the display screen field only when the speaker is a robot.

  In the smile field, a result obtained by analyzing an image taken by the camera 49 and determining whether or not the user is smiling is recorded. In the sowing field, the result of determining whether the user is whispering by analyzing the image taken by the camera 49 is recorded. In the present embodiment, since an image captured by the camera 49 and a voice uttered by the user are simultaneously acquired and analyzed, data is recorded in the user response field only when the speaker is a user. The conversation log DB 62 has one record for one utterance.

  The smile field and the whispering field are examples of subfields of the user reaction field. An arbitrary subfield can be provided in which an image captured by the camera 49 and a user reaction obtained by analyzing a voice acquired by the microphone 31 are recorded. The history DB 61 and the conversation log DB 62 constitute a log recording unit that records the conversation between the user and the robot 20 and the reaction of the user.

  FIG. 11 is an explanatory diagram for explaining the record layout of the bot DB 63. The bot DB 63 is a DB that records a user attribute estimated by analyzing an image captured by the camera 49 and a bot ID uniquely assigned to the pseudo personality of the chat bot.

  The bot DB 63 has a user attribute field and a bot ID field. The user attribute field has a gender field and an age field. The estimated gender of the user is recorded in the gender field. The estimated age of the user is recorded in the age field. A bot ID is recorded in the bot ID field.

  For example, a chat bot with a pseudo personality speaking in a slow tone is set in the bot IDs “A04” and “B04” corresponding to the case where the user is an elderly person. For the bot IDs “A01” and “B01” corresponding to the case where the user is a child, a chat bot with a pseudo personality that speaks in a vocabulary that is easy for children to understand is set.

  For example, if the user's attribute cannot be determined because the user is wearing a mask or sunglasses, the corresponding bot ID “D01” is set to a pseudo-personal chatbot that makes a highly versatile response. Yes.

  The sex field and the age field are examples of subfields of the user attribute field. An arbitrary subfield corresponding to the attribute of the user that can be obtained by analyzing the image captured by the camera 49 and the voice acquired by the microphone 31 can be provided. In addition, since estimation of user attributes based on images and sounds has been conventionally performed, a detailed description thereof will be omitted.

  FIG. 12 is an explanatory diagram illustrating an example of a report. The report shown in FIG. 12 is an example of a report created based on data recorded in the history DB 61 and the conversation log DB 62. The report shown in FIG. 12 includes a location column 71, a start column 721, an end column 722, an age distribution graph column 731, a number of people graph column 732, and a time graph column 733.

  In the place column 71, the place where the robot 20 is installed is displayed. An analysis target period is displayed in the start column 721 and the end column 722. In the age distribution graph column 731, a bar graph indicating the age distribution of the user who has a conversation with the robot 20 is displayed. The horizontal axis is the estimated age of the user, and the vertical axis is the number of users.

  In the horizontal axis memory, for example, “before 10” means that the user is estimated to be in the early teens. The bar graph is a stacked bar graph that is stacked in the order of male, female and gender unknown from the bottom. In all graphs in FIG. 12, black indicates a man, left-left hatching indicates a woman, and right-down hatching indicates gender unknown.

  In the number graph column 732, a pie chart indicating the number of users who have a conversation with the robot 20 is displayed. In the time graph column 733, a pie chart showing the total conversation time of the user who had a conversation with the robot 20 is displayed.

  Each of the age distribution graph column 731, the number of people graph column 732, and the time graph column 733 can be created by analyzing the data recorded in the history DB 61 and the conversation log DB 62. With the report shown in FIG. 12, the administrator of the information processing system 10 can grasp the operating rate of the robot 20 and use it to improve the information processing system 10. Further, the user's question contents and data such as responses can be used for marketing materials such as facilities where the information processing system 10 is installed.

  The report can be created by any other personal computer obtained by acquiring the data recorded in the history DB 61 and the conversation log DB 62 in a CSV (Comma-Separated Values) format or the like. The administrator of the information processing system 10 may create a report using general-purpose spreadsheet application software or the like. The report layout and graph shown in FIG. 12 are exemplary.

  FIG. 13 is a flowchart for explaining the flow of processing of a program. The CPU 41 determines whether or not the approach or contact of the user to the robot 20 has been detected based on the image captured by the camera 49 (step S501). For example, the CPU 41 determines that the approach or proximity of the user has been detected when a human face is detected within a predetermined range in the image using a known face recognition technique. In step S501, the CPU 41 functions as a face detection unit that detects when the user's face is included in the image.

  If it is determined that the approach or contact of the user has been detected (YES in step S501), the CPU 41 extracts a feature amount by cutting out a portion in which the user is reflected from the image (step S502). Since various methods have been proposed for image feature amounts, detailed description thereof is omitted. Any feature suitable for estimating user attributes and user response in the image can be used. The feature amount is an example of data based on an image.

  CPU21 determines whether the sensor 33 detected the approach or contact of the user (step S601). When it determines with having detected (it is YES at step S601), CPU21 sets operation | movement of the light emission part 25 and the actuator 34 to ON, ie, an active state (step S602). Thereafter, the light emitting unit 25 and the actuator 34 operate randomly. The light emitting unit 25 and the actuator 34 may operate according to sound or based on a predetermined pattern.

  CPU21 transmits the audio | voice data which converted the audio | voice detected by the microphone 31 into the electrical signal to the information processing apparatus 40 via near field communication (step S603). The CPU 41 receives the audio data by the interruption process and temporarily stores it in the main storage device 22 or the auxiliary storage device 23.

  When it is determined that the user's approach or contact is not detected (NO in step S501), or after the end of step S502, the CPU 41 determines whether there is data to be transmitted to the server 50 (step S503). Here, the data to be transmitted to the server 50 is the feature amount of the image extracted in step S502 and the voice data received from the robot 20 by the interrupt process. In step S503, it is determined that there is data to be transmitted when either one or both of the feature amount and the audio data exist.

  If it is determined that there is no data to be transmitted to the server 50 (NO in step S503), the CPU 41 returns to step S501. If it is determined that there is data to be transmitted to the server 50 (YES in step S503), the CPU 41 transmits the feature amount and the voice data to the server 50 via the network (step S504). Note that the CPU 41 may extract the feature amount of the audio data and transmit it in step S504 instead of the audio data. The feature amount of the audio data is an example of data based on the audio data.

  The CPU 51 receives the feature amount and the audio data (step S701). The CPU 51 adds a new record to the conversation log DB 62 and records data in the date / time field, speaker field, utterance content field, and user response field.

  The CPU 51 determines whether or not the user has changed to another user (step S702). The CPU 512 can determine whether or not the user has changed based on the feature amount of the image, for example. The CPU 51 may determine whether or not the user has changed based on the elapsed time after the previous execution of step S702.

  If it is determined that the user has changed (YES in step S702), the CPU 51 creates a new record in the history DB 61 and records information related to the previous conversation. The CPU 51 records a new conversation log ID in the record recorded in the conversation log DB 62 immediately after step S701.

  The CPU 51 estimates the user attributes, specifically the gender and age of the user, based on the received feature amount (step S703). CPU51 fulfill | performs the function of an attribute estimation part by step S703.

  The CPU 51 searches the bot DB 63 using the estimated user attribute as a key, and acquires the bot ID from the extracted bot ID field of the record (step S704). The CPU 51 fulfills the function of a bot selection unit that selects a pseudo personality of the voice chat bot based on the user attribute in step S704.

  When it is determined that the user has not changed (NO in step S702), or after the end of step S704, the CPU 51 acquires response data for the user based on the bot ID (step S705). Specifically, the CPU 51 transmits the bot ID acquired in step S704 and the voice data acquired in step S701 to the voice chat bot service on the cloud, and acquires the voice data of the response generated by the voice chat bot.

  CPU51 acquires the video file displayed on the display apparatus 47 from a voice chat bot, or acquires the video file relevant to a response from a predetermined | prescribed server. The voice chat bot software may be executed on the server 50. The video file is recorded, for example, as an HTML file. The CPU 51 may acquire a URL (Uniform Resource Locator) relating to a WEB (World Wide Web) site related to the response instead of the video file.

  The CPU 51 transmits response data including a video file and audio data to the information processing apparatus 40 via the network (step S706). CPU41 receives response data, isolate | separates the audio | voice data in response data, and transmits to the robot 20 via near field communication (step S511). The CPU 21 receives the audio data by the interruption process and temporarily stores it in the main storage device 22 or the auxiliary storage device 23.

  The CPU 41 displays the video file on the display unit 451 (step S512). CPU41 determines whether a process is complete | finished (step S513). For example, when an end instruction is received from the administrator of the information processing system 10, the CPU 41 determines to end the process. If it is determined not to end the process (NO in step S513), the CPU 41 returns to step S501. If it is determined that the process is to be ended (YES in step S513), the CPU 41 ends the process.

  When it is determined that the approach or contact of the user has not been detected (NO in step S601), or after the end of step S603, the CPU 21 determines whether there is audio data output from the speaker 32 (step S611). ). If it is determined that there is audio data (YES in step S611), the CPU 21 converts the audio data into audio via the speaker 32 and outputs the audio (step S612).

  When it is determined that there is no audio data (NO in step S611), or after the end of step S612, the CPU 21 determines whether the sensor 33 has detected a user (step S613). When it determines with having detected (it is YES at step S613), CPU21 returns to step S603.

  When it determines with not detecting (it is NO at step S613), CPU21 sets the operation | movement of the light emission part 25 and the actuator 34 to OFF, ie, an inactive state (step S614). The CPU 21 returns to step S601. As described above, the robot 20 stops its operation, and when a new user approaches, the robot 20 starts operating in response to the user.

  In addition, CPU21 may transmit the detection result by the sensor 33 with audio | voice data in step S603. In this case, the CPU 41 transmits the detection result by the sensor 33 together with the feature amount and the audio data in step S504. The CPU 51 can also use the detection result of the sensor 33 for attribute estimation.

  According to the present embodiment, it is possible to provide the information processing system 10 using the robot 20 that makes many people feel familiar. Since the robot 20 is smaller than the display device 47, the user does not feel intimidation or fear. By using the robot 20, it is possible to provide the information processing system 10 that can easily attract the user's interest as compared to the case where an image such as an avatar is displayed on the display device 47.

  When the user approaches, the robot 20 reacts by operations such as light emission and swinging of the light emitting unit 25. As a result, it is possible to provide the information processing system 10 that allows even a user unfamiliar with a machine, a computer, etc. to easily grasp the opportunity to speak.

  Since the voice chat bot is used to communicate with the user using a natural language, the information processing system 10 that can be easily used by a user who does not have specialized knowledge can be provided. Since the pseudo personality of the voice chat bot is selected and used according to the user attribute estimated from the image, the information processing system 10 that provides the tone, vocabulary, and content information according to the user attribute can be provided.

  By using the robot 20 that is small and has limited functions, the information processing system 10 that can be introduced at low cost can be provided. By installing the information processing system 10 in many places and using it by many users, for example, it is possible to provide the information processing system 10 that collects marketing data such as interest trends of customers who have visited commercial facilities.

  By having the chain hole 35 for the anti-theft chain, it is possible to provide the robot 20 that can prevent theft even though it is small. The swing mechanism using the actuator 34 described with reference to FIG. 5 hardly generates noise. Therefore, it is possible to provide the robot 20 having a head swing mechanism that does not hinder voice communication.

  When the user is in proximity or in contact, the robot 20 can be provided that makes the user feel close by performing light emission and swinging motion. By mounting the camera 49 on the information processing apparatus 40 side, it is possible to provide the information processing system 10 that can photograph the user and perform image analysis even when the robot 20 is swinging.

  An information processing system 10 that can easily supply power to the robot 20 by placing the robot 20 on the charging stand 36 in a state in which the legs 27 of the robot 20 and the leg insertion portions 362 of the charging stand 36 are aligned. Can be provided. By placing the robot 20 imitating a bird on the charging stand 36 imitating a stump, the information processing system 10 in which the power supply mechanism to the robot 20 is not conspicuous can be provided. The power supply to the robot 20 may be performed via the connector 29.

  By using the robot 20 and the display device 47 together, it is possible to provide the information processing system 10 that can provide the user with information such as photographs and maps in addition to voice. By combining a plurality of types of information, the information processing system 10 that effectively provides information required by the user can be provided.

[Embodiment 2]
The present embodiment relates to an information processing system 10 that transmits an image captured by a camera 49 from an information processing apparatus 40 to a server 50. Description of portions common to the first embodiment is omitted.

  FIG. 14 is a flowchart for explaining the flow of processing of the program according to the second embodiment. The processing flow performed by the robot 20 is the same as the processing flow of the program of the first embodiment described with reference to FIG.

  The CPU 41 receives the audio data transmitted from the robot 20, and transmits it to the server 50 together with the image captured by the camera 49 (step S521). The CPU 51 receives image and audio data (step S711).

  The CPU 51 performs image analysis (step S712). The CPU 51 performs voice analysis (step S713). The CPU 51 adds a new record to the conversation log DB 62 and records data in the date / time field, speaker field, utterance content field, and user response field.

  The CPU 51 determines whether or not the user has changed to another user based on the image analysis result and the sound analysis result (step S714). If it is determined that the user has changed (YES in step S714), the CPU 51 creates a new record in the history DB 61 and records information related to the previous conversation. The CPU 51 records a new conversation log ID in the record recorded in the conversation log DB 62 immediately after step S713.

  The CPU 51 estimates the user's attribute based on the image analysis result and the sound analysis result (step S715). By using the image and the sound together, it is possible to estimate the user's attribute with higher accuracy than when estimating the attribute based only on the image.

  The CPU 51 searches the bot DB 63 using the estimated user attribute as a key, and acquires the bot ID from the extracted bot ID field of the record (step S716). When it is determined that the user has not changed (NO in step S714), or after the end of step S716, the CPU 51 acquires response data for the user based on the acquired bot ID (step S717).

  The CPU 51 transmits response data including a video file and audio data to the information processing apparatus 40 via the network (step S718). CPU41 receives response data, isolate | separates the audio | voice data in response data, and transmits to the robot 20 via near field communication (step S531).

  The CPU 41 displays the video file on the display unit 451 (step S532). CPU41 determines whether a process is complete | finished (step S533). If it is determined not to end the process (NO in step S533), the CPU 41 returns to step S521. If it is determined that the process is to be ended (YES in step S533), the CPU 41 ends the process.

  According to the present embodiment, it is possible to provide the information processing system 10 that can accurately estimate the user's attributes by combining image analysis and sound analysis. In addition, you may estimate a user's attribute based on only audio | voice data, without using the camera 49. FIG.

  According to the present embodiment, it is possible to provide an information processing system that determines a dialect or language based on, for example, voice analysis, and selects and responds to a chat bot with a pseudo personality that speaks the dialect or language. By responding in a language that the user is familiar with, the information processing system 10 that makes the user feel familiar can be provided.

[Embodiment 3]
The present embodiment relates to an information processing system 10 that issues a coupon based on a conversation with a user. Description of portions common to the first embodiment is omitted.

  FIG. 15 is an explanatory diagram for explaining the outline of the operation of the guidance system according to the third embodiment. The video displayed on the display device 47 includes a two-dimensional barcode 48 indicating the URL. The user photographs the two-dimensional barcode 48 using his / her smartphone 60 and accesses the URL, thereby displaying a coupon screen such as a discount coupon.

  When the user uses the coupon, the coupon use button 69 in the coupon screen is tapped. Thereby, it can record that a user used a coupon. In addition, it can be determined which coupon was used by putting coupon ID uniquely given to the coupon in URL shown in the two-dimensional barcode 48. FIG.

  FIG. 16 is an explanatory diagram illustrating the record layout of the coupon DB. The coupon DB is a DB that records a coupon ID, a conversation log ID, an issue date / time, and a use date / time in association with each other. The coupon DB has a coupon ID field, a conversation log ID field, a light emission date / time field, and a use date / time field.

  A coupon ID is recorded in the coupon ID field. The conversation log ID when the coupon is issued is recorded in the conversation log ID field. In the issue date field, the date and time when the coupon is displayed on the display device 47 is recorded. The date and time when the coupon use button 69 is tapped is recorded in the use date and time field.

  By issuing a coupon, the information processing system 10 with high user satisfaction can be provided. By collating the conversation ID when the coupon is issued with the usage history of the coupon, marketing data useful for the coupon issuing store can be obtained.

[Embodiment 4]
FIG. 17 is a functional block diagram of the information processing system 10 according to the fourth embodiment. The information processing system 10 includes a robot 20 and an information processing device 40. The information processing apparatus 40 includes a camera 49, a first transmission unit 461 that transmits data related to an image captured by the camera 49 to the network, and response data created based on the data related to the image transmitted from the first transmission unit 461. And a second transmitter 441 that transmits voice data to the robot based on the response data received by the first receiver 462.

  The robot 20 includes a third receiving unit 242 that receives the audio data transmitted by the second transmitting unit 441 and a speaker 32 that outputs audio based on the audio data received by the 242 third receiving unit.

[Embodiment 5]
The present embodiment relates to a mode for realizing the information processing system 10 of the present embodiment by operating a general-purpose computer and a program 97 in combination. FIG. 18 is an explanatory diagram illustrating the configuration of the information processing system 10 according to the fifth embodiment. Note that description of portions common to the first embodiment is omitted.

  The information processing system 10 according to the present embodiment includes a robot 20 and a computer 90 that are connected via short-range wireless communication. The computer 90 is connected to the server 50 via a network.

  The computer 90 includes a CPU 41, a main storage device 42, an auxiliary storage device 43, a first communication unit 46, a second communication unit 44, a touch panel 45, a camera 49, a reading unit 95, and a bus. The first communication unit 46 includes a first transmission unit 461 and a first reception unit 462. The second communication unit 44 includes a second transmission unit 441 and a second reception unit 442. The touch panel 45 includes a display unit 451 such as a liquid crystal display panel and an input unit 452 stacked on the display unit 451.

  The program 97 is recorded on a portable recording medium 96. The CPU 41 reads the program 97 via the reading unit 95 and stores it in the auxiliary storage device 43. The CPU 21 may read the program 97 stored in the semiconductor memory 98 such as a flash memory mounted on the computer 90. Further, the CPU 41 may download the program 97 from the first communication unit 46 and another server computer (not shown) connected via the network and store it in the auxiliary storage device 43.

  The program 97 is installed as a control program for the computer 90, loaded into the main storage device 42, and executed. Thereby, the computer 90 functions as the information processing apparatus 40 described above.

The technical features (components) described in each embodiment can be combined with each other, and new technical features can be formed by combining them.
The embodiments disclosed herein are illustrative in all respects and should not be considered as restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meaning described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 10 Information processing system 20 Robot 201 Torso part 202 Head 21 CPU
22 Main storage device 23 Auxiliary storage device 24 3rd communication part 241 3rd transmission part 242 3rd reception part 25 Light emission part 251 1st light emission part 252 2nd light emission part 26 Switch 27 Leg part 28 2nd charging terminal 29 Connector 31 Microphone 32 Speaker 33 Sensor 330 Sensor I / F
331 Human sensor 332 Touch sensor 333 Gyro sensor 34 Actuator 340 Actuator I / F
341 Magnet 342 Coil 343 Rotating shaft 344 Stopper 35 Chain hole 36 Charging stand 361 Recess 362 Leg insertion part 38 First charging terminal 39 Connector 40 Information processing device 41 CPU
42 Main storage device 43 Auxiliary storage device 44 Second communication unit 441 Second transmission unit 442 Second reception unit
45 touch panel 451 display unit 452 input unit 46 first communication unit 461 first transmission unit 462 first reception unit 47 display device 48 two-dimensional barcode 49 camera 50 server 51 CPU
52 Main storage device 53 Auxiliary storage device 54 4th communication part 541 Information transmission part 542 Information reception part 60 Smartphone 61 History DB
62 Conversation log DB
63 Bot DB
69 Coupon use button 71 Location column 721 Start column 722 End column 731 Age distribution graph column 732 Number of people graph column 733 Time graph column
DESCRIPTION OF SYMBOLS 90 Computer 95 Reading part 96 Portable recording medium 97 Program 98 Semiconductor memory

Claims (10)

In an information processing system comprising a robot and an information processing device,
The information processing apparatus includes:
A camera,
A first transmitter for transmitting data relating to an image captured by the camera to a network;
A first receiver for receiving response data created based on the data related to the image transmitted from the first transmitter;
A second transmitter for transmitting voice data to the robot based on the response data received by the first receiver;
The robot is
A third receiver for receiving the audio data transmitted by the second transmitter;
An information processing system comprising: a speaker that outputs sound based on the sound data received by the third receiving unit. The robot is
With a microphone,
A third transmitter that transmits the audio data acquired by the microphone to the information processing device;
The information processing apparatus includes:
A second receiver for receiving the audio data transmitted from the third transmitter;
The first transmitter is
The information processing system according to claim 1, wherein the data related to the audio data and the data related to the image received by the second receiving unit are transmitted to a network. The robot is
It has a sensor that detects the approach or contact of the user,
The third transmission unit transmits a detection result to the information processing device when the sensor detects an approach or contact of a user,
The information processing system according to claim 2, wherein the first transmission unit transmits the detection result to a network. A server connected to the information processing apparatus via a network;
The information processing apparatus or the server is
An attribute estimation unit that estimates user attributes based on the data related to the image;
The server
A bot selection unit that selects a chat bot based on the attribute estimated by the attribute estimation unit;
The information processing system according to claim 2, further comprising: an information transmission unit configured to transmit the response data created by the chat bot selected by the bot selection unit. The information processing system according to claim 4, further comprising: a log recording unit that records the attribute, the response data, and a user response to the voice output by the robot in association with each other. The information processing apparatus includes a display device that displays an image based on the response data,
The information processing system according to any one of claims 1 to 5, wherein the robot is smaller than the display device. The robot includes a columnar body part and a head disposed on one end surface of the body part,
The information processing system according to any one of claims 1 to 6, wherein the head is rotated about a central axis of the body part with respect to the body part. A recess corresponding to a part of the exterior of the robot;
A first charging terminal provided in the recess;
Equipped with a plate-shaped charging stand,
The information processing system according to any one of claims 1 to 7, wherein the robot includes a second charging terminal corresponding to the first charging terminal. The information processing system according to any one of claims 1 to 8, wherein the robot has a chain hole for an anti-theft chain. A tablet having a camera and a display unit and a robot having a height shorter than the length of the short side of the display unit of the tablet are arranged close to each other,
Photograph the user with the camera,
Send data about the captured image from the tablet to the network,
The response data created based on the transmitted data is received by the tablet,
Display the video based on the received response data on the display unit,
Transmitting voice data based on the received response data from the tablet to the robot;
The transmitted voice data is received by the robot,
An information processing method for outputting sound based on received sound data from a speaker built in the robot.

Images