JP2019105966A - Information processing method and information processing apparatus - Google Patents

Abstract

To provide an information processing method and an information processing apparatus capable of determining an optimal service in an in-vehicle without taking time after getting on the vehicle.SOLUTION: An information processing apparatus acquires a captured image of an interior of a vehicle from a camera for capturing an in-vehicle image, and estimates features of persons present in the in-vehicle based on the captured image of the in-vehicle, estimates a personnel composition of the in-vehicle in association with the number of persons in the in-vehicle and the features of the persons present in the in-vehicle, identifies services used in the in-vehicle, causes a storage device to store a usage history in the in-vehicle in association with the personnel composition of the in-vehicle and the services in the in-vehicle, and analyzes the stored usage history in the in-vehicle to identify the most suitable service for each staff in the in-vehicle.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing method and an information processing apparatus.

  Based on the voice collecting means for collecting the voice generated in the car, the seating position specifying means for identifying the seating position where the occupant is seated in the car as the seating position, the collected voice, and the identified seating position A speaker estimation means for estimating a speaker among the existing occupants, a conversation content estimation means for estimating a content of a conversation by the speaker based on collected voices, a specified seating position, an estimated speaker, Occupant configuration estimation means for estimating the configuration of an occupant present in the vehicle based on the estimated conversation content, and behavioral purpose estimation means for estimating an occupant's behavioral objective based on the estimated conversation content and the estimated occupant configuration And a recommended service determining means for determining a recommended service as a recommended service based on the estimated occupant configuration and the estimated action purpose. Obtaining vehicle apparatus is known (Patent Document 1).

JP, 2012-133530, A

  In the prior art, not only can the occupant's configuration be estimated from the occupant's seating position and conversation content, but it is not possible to determine the service recommended in the vehicle unless the occupant's action purpose is estimated from the occupant's conversation. There is a problem that it takes a long time to provide the recommended service in the car after that.

  The problem to be solved by the present invention is to provide an information processing method and an information processing apparatus capable of determining an optimal service in a vehicle without spending time after getting on the vehicle.

  The present invention acquires a captured image of the interior of a vehicle from a camera that captures an interior of the vehicle, estimates the characteristics of a person present in the vehicle based on the captured image of the interior of the vehicle, and associates the characteristics of the person in the vehicle with the number of people in the vehicle Estimate the personnel composition of the in-vehicle, identify the service used in the vehicle, store the usage history in the in-vehicle where the personnel composition of the in-vehicle and the in-vehicle service are associated, and store in the storage device. The above problem is solved by analyzing the usage history of the vehicle and identifying the optimal service for each personnel configuration in the vehicle.

  According to the present invention, it is possible to determine the optimum service in the vehicle without spending time after getting on the vehicle.

FIG. 1 is a block diagram of an information providing system according to the present embodiment. FIG. 2 is a diagram (A) for explaining the in-vehicle personnel configuration estimation process and a diagram (B) for explaining the out-of-vehicle personnel configuration estimation procedure. FIG. 3 is a diagram for explaining the service usage history stored in the database. FIG. 4 is a flowchart until the in-vehicle device transmits usage history information in the vehicle to the data center. FIG. 5 is a flowchart until the external device transmits usage history information outside the vehicle to the data center. FIG. 6 is a flow chart for explaining a process of identifying an optimal service for each personnel configuration. FIG. 7A is a flow chart for explaining a process from getting on the vehicle to providing a service optimum for the personnel configuration in the vehicle. FIG. 7B is a flow chart for describing a process from getting on the vehicle to providing a service optimum for the personnel configuration in the vehicle.

  Hereinafter, embodiments of the present invention will be described based on the drawings.

  In the present embodiment, the information processing apparatus according to the present invention is applied to an in-vehicle apparatus 100 mounted on a vehicle, an external apparatus 200 installed outside the vehicle, and an information providing processing system cooperating with the data center 300. The case will be described as an example.

  FIG. 1 is a block diagram of the information providing system 1. The information providing system 1 of the present embodiment includes an in-vehicle device 100, an external device 200, and a data center 300. The information providing system 1, the in-vehicle device 100, the external device 200, and devices included in these are computers that include arithmetic processing devices such as a CPU and execute arithmetic processing.

  First, the in-vehicle apparatus 100 will be described. The in-vehicle apparatus 100 according to the present embodiment includes an in-vehicle camera 110, an in-vehicle microphone 120, an in-vehicle operation device group 130, an in-vehicle display 140, an in-vehicle speaker 150, an in-vehicle communication device 160, and an in-vehicle control device 170. The respective devices constituting the in-vehicle device 100 are connected by a controller area network (CAN) or another in-vehicle LAN to exchange information with each other.

  The on-vehicle camera 110 is provided in the room of the vehicle and captures an image of the room. The on-vehicle camera 110 outputs the captured image to the on-vehicle control device 170. The installation position of the on-vehicle camera 110 and the number of the on-vehicle cameras 110 are not particularly limited. For example, the on-vehicle camera 110 is provided on an instrumental panel and captures an image of a person sitting in the front seat. Also, for example, the on-vehicle camera 110 is provided in the vicinity of the rearview mirror, and captures an image of a person sitting on the rear seat. Also, for example, the on-vehicle camera 110 is provided at the rear of the vehicle and captures an image displayed by the on-vehicle display 140 provided at the front of the vehicle. Details of the content displayed by the in-vehicle display 140 will be described later.

  The in-vehicle microphone 120 is provided in the room of the vehicle and collects the sound generated in the vehicle. The in-vehicle microphone 120 converts the collected voice into a digital signal, and outputs the digital signal to the in-vehicle control device 170 as audio information. The installation position of the in-vehicle microphone 120 and the number of the in-vehicle microphones 120 are not particularly limited. For example, in-vehicle microphone 120 is provided at a predetermined position of the seat for each seat, and collects conversations in the vehicle, music output from in-vehicle speaker 150, and the like. The detail of the content which the vehicle-mounted speaker 150 outputs is mentioned later.

  The in-vehicle operation device group 130 is provided in the vicinity of the in-vehicle display 140, and changes the content of the image displayed by the in-vehicle display 140 or adjusts the volume of the sound output from the in-vehicle speaker 150. The in-vehicle operation device group 130 includes, for example, a mechanical switch, a touch panel switch provided on the in-vehicle display 140, and the like. The in-vehicle operation device group 130 outputs, to the in-vehicle control device 170, operation information by a person in the vehicle.

  The on-vehicle display 140 is provided in the cabin of the vehicle, and displays, for example, images such as a television program, a movie, and a route guidance screen in a navigation system according to operation information from the on-vehicle operation device group 130. In addition, when the vehicle can be connected to the Internet, for example, a website is displayed on the in-vehicle display 140. As the on-vehicle display 140, a liquid crystal display may be mentioned as an example.

  The on-vehicle speaker 150 is provided in the room of the vehicle, and outputs, for example, audio based on music, a radio program, and other video data in accordance with operation information from the on-vehicle operation device group 130. When the vehicle can be connected to the Internet, the on-vehicle speaker 150 outputs a radio program of the Internet radio.

  Operation information is input to the in-vehicle display 140 and the in-vehicle speaker 150 from the in-vehicle operation device group 130 via the in-vehicle control device 170. The in-vehicle display 140 displays an image or a moving image according to the operation information, and the in-vehicle speaker 150 outputs an audio according to the operation information.

  For example, when a person in the vehicle operates the in-vehicle operation device group 130 to use the navigation system, a screen for route guidance and a screen for operation are displayed on the in-vehicle display 140. Are voices for providing route guidance and voices for conveying traffic information.

  Also, for example, when a person in the vehicle operates the in-vehicle operation device group 130 to view a television program, an image of the television program is displayed on the in-vehicle display 140, and an image of the television program is displayed from the in-vehicle speaker 150. The voice corresponding to is output. The television program includes a program for entertainment purpose, a program for conveying current affairs news, a program for conveying road information such as road construction information or traffic jam information, a program for conveying weather forecast, and the like.

  Also, for example, when a person in the vehicle operates the in-vehicle operation device group 130 to play music or a radio program, the in-vehicle speaker 150 outputs a music or a radio program. The radio program includes a program for entertainment purpose, a program for conveying current affairs news, a program for conveying road information such as road construction information or traffic congestion information, a program for conveying weather forecast, and the like, similarly to a television program. Also, music includes music of various genres such as classical music, popular music and ethnic music.

  Further, information (hereinafter referred to as service guidance information) for guiding a service optimum for the present personnel configuration in the vehicle is input to the on-vehicle display 140 from the data center 300 via the on-vehicle communication device 160 and the on-vehicle control device 170. . The in-vehicle display 140 presents service guidance information to a person in the vehicle. The details of the service guidance information transmitted from the data center 300 will be described later.

  The in-vehicle communication device 160 is a device capable of communicating with the server communication device 310 of the data center 300 via a telephone network, the Internet network, or the like. The in-vehicle communication device 160 receives information on the configuration of personnel in the vehicle and information on service usage history in the vehicle (hereinafter referred to as usage history information in the vehicle), which will be described later. , Transmits to the server communication device 310. The in-vehicle communication device 160 can include the information in the probe information, and may transmit the information as the probe information to the server communication device 310. Further, the in-vehicle communication device 160 receives the service guidance information from the server communication device 310, and outputs the received service guidance information to the in-vehicle control device 170. In addition, the details of the information on the personnel configuration in the vehicle and the service usage history information in the vehicle will be described later.

  Further, information on the configuration of personnel in the vehicle and usage history information in the vehicle are input to the in-vehicle communication device 160 at asynchronous timing. The on-vehicle control device 170 described later outputs information on the personnel configuration in the vehicle to the on-vehicle communication device 160 at the timing when the ignition is turned on and the timing when the vehicle starts. Thus, information on the personnel configuration in the vehicle is transmitted to the data center 300 before or immediately after the vehicle starts moving. On the other hand, the in-vehicle control device 170 outputs in-vehicle usage history information to the in-vehicle communication device 160 at a timing when the ignition is turned off or at a timing when a predetermined time has elapsed. That is, the in-vehicle communication device 160 transmits the information of the personnel configuration in the vehicle to the server communication device 310 of the data center 300 before transmitting the usage history information in the vehicle.

  The in-vehicle control device 170 is a device for estimating the personnel configuration in the vehicle and generating usage history information in the vehicle. Specifically, the on-vehicle control device 170 generates the in-vehicle service usage history by executing a ROM (Read Only Memory) storing a program for executing the in-vehicle service usage history generation processing and the program stored in the ROM. It is a computer provided with CPU (Central Processing Unit) as an operation circuit which performs a process, and RAM (Random Access Memory) which functions as an accessible storage device.

  Specifically, the in-vehicle control device 170 according to the present embodiment generates usage history information in the vehicle by executing the following processing. The in-vehicle control device 170 includes an in-vehicle person feature estimation process for estimating features of a person in the vehicle, an in-vehicle personnel structure estimation process for estimating a personnel configuration in the vehicle, and an in-vehicle service identification process for identifying a service used in the vehicle. And performing in-vehicle service usage history generation processing of generating usage history information in the vehicle based on the processing results.

  The on-vehicle control device 170 executes each function by the cooperation of the above-described hardware and software for executing the above-described functions or to execute each process.

  First, in-vehicle person feature estimation processing will be described. The on-vehicle control device 170 estimates features of a person present in the vehicle based on the captured image of the on-vehicle camera 110. The characteristics of the person in the car include gender and age. For example, the on-vehicle control device 170 performs image processing on the captured image of the on-vehicle camera 110 to extract a plurality of feature points (eyes, nose, mouth, hair, etc.) of a person shown in the captured image. Then, the on-vehicle control device 170 estimates the gender of the person in the vehicle by collating the extracted feature points of the person in the vehicle with the predetermined feature pattern and determining the gender. be able to. Similarly, the in-vehicle control device 170 estimates the age of the person in the vehicle by matching the feature points of the person in the vehicle with the predetermined feature pattern and the feature pattern for estimating the age. Can.

  The accuracy of the age estimated by the on-board controller 170 is not particularly limited. The in-vehicle control device 170 can appropriately change the accuracy of the age to be estimated according to the image processing capability and the imaging accuracy of the in-vehicle camera 110. For example, the on-vehicle control device 170 stores in advance in the storage device a feature pattern categorized in units of 10 years, such as "10's" in the range of 10 to 19 years and "20's" in the range of 20 to 29 years. By doing this, it is possible to estimate the age of the person in the car. The in-vehicle person feature estimation process is not limited to the method described above, and the in-vehicle control device 170 can appropriately use the in-vehicle person feature estimation process known at the time of filing the present application.

  Next, the in-vehicle personnel composition estimation process will be described. The in-vehicle control device 170 estimates the personnel configuration in the vehicle based on the captured image of the in-vehicle camera 110 and the processing result of the in-vehicle person feature estimation processing. For example, the in-vehicle control device 170 detects the number of people in the vehicle from the captured image of the in-vehicle camera 110. Then, the on-vehicle control device 170 estimates the personnel configuration in the vehicle in which the detected number of people in the vehicle and the processing result (sex and age) of the in-vehicle person feature estimation processing are associated.

  An example of the processing result of the in-vehicle personnel configuration estimation process will be described with reference to FIG. 2 (A). FIG. 2A is a diagram for explaining an in-vehicle personnel configuration estimation process. FIG. 2A shows the processing result of the on-vehicle control device 170 when four persons including the driver are on the vehicle. In this example, the in-vehicle control device 170 first detects from the captured image of the in-vehicle camera 110 that the number of people in the vehicle is four. Next, as shown in FIG. 2A, the in-vehicle control device 170 executes the above-described in-vehicle person feature estimation process for each of four people, and "20's male: 1 person, 30's woman" Estimated personnel composition in the car: 1 person, 50's male: 1 person, 60's male: 1 person, 4 people in total.

  Thus, in the present embodiment, the in-vehicle control device 170 can estimate the personnel configuration in the vehicle at the timing when the image captured by the in-vehicle camera 110 is input. Therefore, the personnel configuration of the vehicle can be estimated also at the time of getting on the vehicle and the timing before the vehicle starts such as the timing when the ignition is turned on. Further, since only the captured image of the on-vehicle camera 110 is used, the personnel configuration in the vehicle can be estimated even when conversation in the vehicle is not performed.

  When the in-vehicle control unit 170 estimates the in-vehicle personnel configuration, the in-vehicle control device 170 transmits the in-vehicle personnel configuration information to the server communication device 310 of the data center 300 via the in-vehicle communication device 160. The in-vehicle control device 170 can also include the captured image itself of the in-vehicle camera 110 in the information of the personnel configuration in the vehicle. Thus, the server processing device 330 of the data center 300, which will be described later, can determine whether or not the person shown in the captured image is a person having a boarding history in the past.

  Referring back to FIG. 1 again, the in-vehicle service identification process of the in-vehicle control device 170 will be described. The in-vehicle control device 170 specifies a service used in the vehicle. The service used in the vehicle in the present embodiment is a service using at least one of the on-vehicle display 140 and the on-vehicle speaker 150. Examples of services available in a car include video reproduction, music reproduction, road information provision, and weather information provision. The video reproduction includes the reproduction of the television program described above and the reproduction of the movie, and the music reproduction includes the reproduction of music of each genre described above. Also, provision of road information includes not only provision of road information by television programs but also provision of road information by radio programs. The provision of weather information also includes the provision of weather forecasts by radio programs as well as the provision of weather forecasts by television programs.

  The in-vehicle control device 170 only specifies the type of service (image reproduction, music reproduction, etc.) based on the captured image from the in-vehicle camera 110, the audio information from the in-vehicle microphone 120, and the operation information from the in-vehicle operation device group 130. It also specifies the specific content (program name, song name) of each service. For example, information of a program guide of a television program and a radio program is stored in advance in a storage device such as a ROM. Then, based on the operation information from the in-vehicle operation device group 130, the in-vehicle control device 170 identifies the television channel set by the person in the vehicle, and collates with the program information stored in the storage device. It is possible to specifically identify the contents of a television program that is being Also, for example, music information of any genre is stored in advance in a storage device such as a ROM. Then, the in-vehicle control device 170 can specify the genre and the song title of the music currently flowing in the vehicle by collating the audio information from the in-vehicle microphone 120 with the music information stored in the storage device.

  Next, the in-vehicle service usage history generation process will be described. The on-vehicle control device 170 generates a history of services used in the vehicle as usage history information in the vehicle based on the results of the above-described respective processes. The in-vehicle control device 170 uses information in which the estimated in-vehicle personnel configuration is associated with the identified service used in the vehicle as usage history information in the vehicle. The in-vehicle control device 170 transmits in-vehicle usage history information to the server communication device 310 of the data center 300 via the in-vehicle communication device 160.

  Next, the external device 200 will be described. The external apparatus 200 of the present embodiment is an apparatus for providing services such as video reproduction and music reproduction to a person existing outside the vehicle. The installation site of the external device 200 is not particularly limited, and the installation site is not limited to a commercial facility or a facility such as a public facility, and may be at home. The external device 200 can be exemplified by a smart speaker equipped with artificial intelligence (AI). The external device 200 includes an external camera 210, an external microphone 220, an external operation device group 230, an external display 240, an external speaker 250, an external communication device 260, and an external control device 270. The respective devices constituting the external device 200 are connected by a signal line such as a LAN or the like, or can communicate by wireless communication such as Wifi or Bluetooth (registered trademark) in order to mutually exchange information.

  In the present embodiment, the external apparatus 200 includes an apparatus having the same function as the above-described in-vehicle apparatus 100. Therefore, in the following, the in-vehicle apparatus 100 performs the contents overlapping with the in-vehicle apparatus 100. The part of the explanation is incorporated. Moreover, "the exterior of a vehicle" in which the external device 200 is provided is outside the vehicle.

  The external camera 210 is provided outside the vehicle and captures an image of the surroundings of the external device 200. The external camera 210 outputs the captured image to the external control device 270. The number of external cameras 201 is not particularly limited.

  The external microphone 220 is provided outside the vehicle and collects audio generated around the external device 200. The external microphone 220 converts the collected voice into a digital signal and outputs the digital signal to the external control device 270 as voice information.

  The external operation device group 230 is provided, for example, in the vicinity of the external display 240, and changes the content of the video displayed by the external display 240, and adjusts the volume of the audio output from the external speaker 250. The external operation device group 230 may be a touch panel provided on the external display 240, or may be a remote control capable of remotely operating the external device 200 as a device different from the other devices.

  The external display 240 is provided outside the vehicle, and displays, for example, videos such as television programs and movies, and moving images distributed via the Internet according to operation information from the external operation device group 230. The external speaker 250 is provided outside the vehicle, and outputs, for example, audio based on music, a radio program, and other video data in accordance with operation information from the external operation device group 230. When the external device 200 can connect to the Internet, the external display 240 can display a website or a screen of an application included in the external device 200. Also, the external speaker 250 can output a radio program of the Internet radio.

  Operation information is input to the external display 240 and the external speaker 250 from the external operation device group 230 via the external control device 270. The external display 240 displays an image or a moving image according to the operation information, and the external speaker 250 outputs an audio according to the operation information.

  For example, in the living room of the home, when one of the family operates the external operation device group 230 to play music or a radio program, a music or radio program corresponding to the operation of the external operation device group 230 from the external speaker 250 Is output. The music includes the music of each genre described above, and the radio program includes a program intended for entertainment, a program for conveying current affairs news, a program for conveying weather forecast, and the like.

  The external communication device 260 is a device capable of communicating with the server communication device 310 of the data center 300 via the telephone network, the Internet network, or the like, as with the in-vehicle communication device 160. The external communication device 260 receives information on service usage history outside the vehicle described later (hereinafter referred to as usage history information outside the vehicle), and the external communication device 260 can communicate with the server communication device 310 at a timing when it can communicate with the server communication device 310. Usage history information in the server communication device 310 is transmitted to the server communication device 310.

  The external control device 270 is a device for generating usage history information outside the vehicle. Specifically, the external control device 270 generates the out-of-vehicle service usage history by executing a ROM (Read Only Memory) storing a program for executing the out-of-vehicle service usage history generation processing and the program stored in the ROM. It is a computer provided with CPU (Central Processing Unit) as an operation circuit which performs a process, and RAM (Random Access Memory) which functions as an accessible storage device.

  Specifically, the external control device 270 of the present embodiment generates usage history information outside the vehicle by executing the following processing. The external control device 270 performs an outside person feature estimation process for estimating the feature of a person outside the vehicle, an outside person configuration estimation process for estimating the personnel configuration outside the vehicle, and an in-vehicle service identification process for identifying a service used in the vehicle. The external service usage history generation processing of generating external usage history information based on the processing results is executed.

  In the present embodiment, each process executed by the external control device 270 is the same as each process executed by the in-vehicle control device 170 except that the object of the process is changed from the inside of the vehicle to the outside of the vehicle. . In the following, in each process, the description given for the in-vehicle control device 170 is partially used for the content overlapping with the in-vehicle control device 170.

  First, the outside person characteristic feature estimation process will be described. The external control device 270 estimates the feature of the person existing outside the vehicle based on the captured image of the external camera 210. The characteristics of the person outside the vehicle include gender and age. The outside person characteristic feature estimation process may be the same processing method as the in-vehicle person feature estimation processing of the on-vehicle control device 170, or may be a different processing method.

  Next, the out-of-vehicle personnel configuration estimation process will be described. The external control device 270 estimates the personnel configuration outside the vehicle based on the outside-vehicle captured image captured by the external camera 210 and the processing result of the outside-vehicle person feature estimation processing. For example, the external control device 270 detects the number of people outside the vehicle existing around the external device 200 from the captured image of the external camera 210. Then, the external control device 270 estimates the personnel configuration outside the vehicle in which the detected number of people outside the vehicle and the processing result (sex and age) of the person characteristic estimation process outside the vehicle are associated.

  An example of the processing result of the out-of-vehicle personnel configuration estimation process will be described with reference to FIG. 2 (B). FIG. 2 (B) is a diagram for explaining the out-of-vehicle personnel configuration estimation process. FIG. 2 (B) shows the processing result of the external control device 270 when there are four people in the home. In this example, the external control device 270 first detects from the captured image of the external camera 210 that the number of people present around the external device 200 is four. Next, as shown in FIG. 2B, the external control device 270 executes the above-described outside person characteristic estimation process for each of the four persons, and as shown in FIG. Less than 1 male, 1 teenage female, 1 male, 50 male, 4 people in total, and estimate the personnel composition outside the car. As described above, in the present embodiment, the external control device 270 estimates the number of people outside the vehicle and the configuration of the number of people outside the vehicle in which the gender and age of the person outside the vehicle are associated.

  Referring back to FIG. 1 again, the external service identification processing of the external control device 270 will be described. The external control device 270 specifies a service used outside the vehicle. The service used outside the vehicle in the present embodiment is a service using at least one of the external display 240 and the external speaker 250. Examples of services available outside the vehicle include video reproduction, music reproduction, and provision of weather information.

  Similar to the on-vehicle control device 170, the external control device 270 determines the type of service (video reproduction, based on the captured image from the external camera 210, the audio information from the external microphone 220, and the operation information from the external operation device group 230). Not only specifying music reproduction etc.), but also specifying specific contents (program name, song name) of each service. Also, when the external device 200 can connect to the Internet, the external control device 270 acquires information on the website accessed by the external device 200, and identifies the specific content of the service, for example, a social network service etc. can do.

  Next, out-of-vehicle service usage history generation processing will be described. The external control device 270 generates a history of services used outside the vehicle as usage history information outside the vehicle based on the results of the above-described respective processes. The external control device 270 uses information obtained by associating the estimated personnel composition outside the vehicle and the identified service outside the vehicle as the usage history information outside the vehicle. The external control device 270 transmits usage history information outside the vehicle to the server communication device 310 of the data center 300 via the external communication device 260.

  Next, the data center 300 will be described. The data center 300 is a facility in which various computers and devices such as data communication are installed. As shown in FIG. 1, the data center 300 includes a server communication device 310, a database 320, and a server processing device 330.

  The server communication device 310 can communicate with the in-vehicle communication device 160 of the in-vehicle device 100 and the external communication device 260 of the external device 200 via a telephone network or the Internet. The server communication device 310 communicates with the plurality of in-vehicle communication devices 160 mounted in each of the plurality of vehicles, and receives the information of the personnel configuration in the vehicle and the usage history information in the vehicle from each vehicle. The server communication device 310 outputs, to the server processing device 330, the information on the personnel configuration in the vehicle received from each vehicle and the usage history information in the vehicle.

  In addition, the server communication device 310 communicates with a plurality of external communication devices 260 provided in each facility (commercial facility, public facility) and each home, and receives usage history information outside the vehicle from each facility or home Do. The server communication device 310 outputs, to the server processing device 330, usage history information outside the vehicle received from each facility or the like.

  The database 320 is a database storing usage history information in the vehicle acquired from a plurality of vehicles and usage history information outside the vehicle acquired from a plurality of facilities and the like. The usage history information in the car and the usage history information outside the vehicle are sequentially input from the server communication device 310 to the database 320 via the server processing device 330, so the two types of usage history information are time elapsed. Together with the database 320. Further, in the database 320, two types of usage history information are stored in a format that can be distinguished from each other.

  The server processing device 330 includes a ROM storing a program for specifying an optimal service for each personnel configuration in the vehicle, a CPU executing the program stored in the ROM, and a RAM functioning as an accessible storage device. It consists of The server processing device 330 stores the usage history information in the vehicle transmitted from the in-vehicle device 100 and the usage history information outside the vehicle transmitted from the external device 200 by executing the program stored in the ROM by the CPU. Based on usage history storage processing, usage history analysis processing for analyzing services tending to be used for each personnel configuration in the vehicle, and personnel configuration in the vehicle on which the in-vehicle apparatus 100 is mounted. And an information providing process provided to the apparatus 100.

  First, usage history storage processing will be described. The server processing device 330 stores in-vehicle usage history information and out-of-vehicle usage history information input from the server communication device 310 in the database 320. Further, the server processing device 330 can identify whether the transmission source of the two types of usage history information is the in-vehicle device 100 or the external device 200, and determines the two types of usage history information and stores it in the database 320. In addition, the server processing device 330 executes statistical processing such as sorting the usage history information accumulated in the database 320 for each gender, age, and number of people based on the personnel configuration included in the usage history information. Can.

  FIG. 3 is an example of in-vehicle usage history information stored in the database 320. As shown in FIG. 3, the database 320 stores the type of service used in the car and the specific content of the service in association with the number of people in the car, the gender of the person in the car, and the age of the person in the car. It is done. For example, for the column "service used in car" in FIG. 3, "play music" as the type of service and "song name A" as the specific content of the music are shown in the first line. . On the second line, "play TV program" and "program name A" are shown.

  Further, in FIG. 3, “number of people”, “sex”, and “age” are shown as information on the personnel configuration in the vehicle when each service is used. For example, the service of “playing back a television program of program name A” is associated with a total of two persons, one male in her 60s and one female in her 60s.

  Next, usage history analysis processing will be described. The server processing device 330 analyzes the usage history information stored in the database 320 to identify the optimal service for each personnel configuration in the vehicle. The server processing device 330 may analyze one of the usage history information in the car and the usage history information outside the vehicle, or may analyze data in which two types of usage history information are combined. Here, analysis of usage history information in a car will be described as an example.

  The server processing device 330 analyzes services that tend to be utilized for each personnel configuration in the vehicle. For example, the server processing device 330 acquires in-vehicle usage history information from the database 320, executes clustering processing for one service, and generates parameters for the personnel configuration in the vehicle (for each person, gender Group by age group) Then, the server processing device 330 executes statistical processing for each of the grouped items, and analyzes the tendency of the service to be used by the number of people, by sex, by age. The server processing device 330 calculates the combination of the analyzed number of persons, gender, and age tending to be used for one service as the feature value for one service. The server processing device 330 analyzes a service that tends to be used for each staff configuration in the vehicle by repeating the above-described processing for each service.

  In addition, the server processing device 330 repeats the above-described analysis process every predetermined cycle, that is, whenever the latest usage history information is accumulated in the database 320. This makes it possible to learn the relationship between the staffing in the car and the services that tend to be used. Then, when the distribution of the analysis result in each cycle falls within the predetermined range, the server processing device 330 determines that the fluctuation of the tendency of the service to be used has converged, and the server processor 330 determines the combination of the number, gender and age. To identify the best service.

  Further, the server processing apparatus 330 not only identifies one optimal service for each staffing configuration in the car, but also identifies a plurality of service candidates that may tend to be used for each staffing configuration in the car. For example, the server processing device 330 may be prioritized in the order of services in which the number of combinations of the number of people tending to be used, gender, and age among the services having been used, and may be used. Extract high service candidates. In addition, the analysis method of utilization historical information in a car is not limited to the method mentioned above, The server processing apparatus 330 can use the technique of the analysis method at the time of this application application suitably.

  The server processing device 330, when the analysis of the usage history information in the car is finished, the service optimum for each staffing configuration in the car, the plurality of service candidates that may be used for each staffing configuration in the car, and the features of each service Each of the quantities (combinations of people tending to be used, gender, and age) is stored in database 320. In addition, the server processing device 330 executes the same process as the process described above on the usage history information outside the vehicle, thereby utilizing the optimal service for each personnel configuration outside the vehicle and the personnel configuration outside the vehicle. The plurality of possible service candidates and feature quantities of each service are stored in the database 320.

  Next, information provision processing will be described. First, the server processing device 330 determines, from the information transmitted from the in-vehicle device 100, whether or not there is a person who has experienced riding in the past in the current vehicle. The server processing device 330 determines whether there is an experienced passenger from the captured image of the in-vehicle camera 110 among the personnel configuration in the vehicle, and even an experienced rider other than the driver is included as a companion. In this case, from the database 320, the service experienced by the rider in the past is extracted. Then, the server processing device 330 identifies the extracted service as the optimal service, and uses it as service guidance information.

  Next, the case where there is no passenger experienced in the current car will be described. In this case, the server processing device 330 identifies a service that is optimal for the current personnel configuration in the vehicle, and uses it as service guidance information. For example, when the information on the current personnel configuration in the vehicle is input from the in-vehicle device 100, the server processing device 330 refers to the analysis result of the usage history information in the vehicle stored in the database 320. Then, when the analysis result includes a service that is optimal for the current personnel configuration in the vehicle, the server processing device 330 identifies the service as the optimal service. On the other hand, the server processing device 330 refers to the analysis result of the usage history information outside the vehicle, when the analysis result of the usage history information in the vehicle does not include the optimum service. Then, when the analysis result includes a service that is optimal for the current personnel configuration in the vehicle, the server processing device 330 identifies the service as the optimal service.

  Further, when the analysis result of any usage history information does not include a service optimum for the present personnel configuration in the vehicle, the server processing device 330 determines the current personnel configuration from the analysis result of the usage history information in the vehicle. The plurality of service candidates that are likely to be used in the above, and that are given priority, are extracted as service guidance information.

  The server processing device 330 transmits the service guidance information obtained by each process described above to the on-vehicle device 100 via the server communication device 310. The on-vehicle apparatus 100 displays the received service guidance information on the on-vehicle display 140, and is used for the person in the vehicle in the information on the service most suitable for the current personnel configuration in the vehicle, and in the current personnel configuration in the vehicle. Information on a plurality of possible service candidates or information on services used in the past can be presented. The service guidance information includes the type of service such as video reproduction and music reproduction, and specific contents of the service such as a program name and a song name.

  Next, with reference to FIG. 4, an operation until the in-vehicle apparatus 100 transmits usage history information in the vehicle to the data center 300 will be described. FIG. 4 is a flowchart until the in-vehicle apparatus 100 transmits in-vehicle usage history information to the data center 300. The process of the flowchart shown in FIG. 4 is respectively executed by the on-vehicle apparatus 100 mounted on each vehicle.

  First, in step S101, the on-vehicle control device 170 acquires an in-vehicle captured image from the on-vehicle camera 110. When a plurality of in-vehicle cameras 110 are provided in the in-vehicle apparatus 100, the in-vehicle control device 170 acquires a captured image from each in-vehicle camera 110.

  In step S102, the on-vehicle control device 170 estimates features of a person present in the vehicle based on the in-vehicle captured image acquired in step S101. The characteristics of the person in the car include the gender of the person in the car and the age of the person in the car. The in-vehicle control device 170 estimates features one by one when there are a plurality of persons in the vehicle. Also, in this step, the in-vehicle control device 170 detects a person present in the vehicle from the captured image, and calculates the number of people in the vehicle.

  In step S103, the on-vehicle control device 170 estimates the personnel configuration in the vehicle. The number of people in the vehicle calculated in step S102 and the feature of the person in the vehicle estimated in the same step are associated with the personnel configuration in the vehicle (see FIG. 2A).

  In step S104, the in-vehicle control device 170 specifies a service used in the vehicle. For example, the in-vehicle control device 170 detects the screen of the in-vehicle display 140 from the captured image input from the in-vehicle camera 110, thereby specifying the program name of the television program being reproduced in the vehicle. In addition, for example, the on-vehicle control device 170 specifies, from the audio information input from the on-vehicle microphone 120, the title of the music being played in the vehicle.

  In step S105, the in-vehicle control device 170 creates in-vehicle usage history information as a history of services used in the vehicle. The in-vehicle control device 170 associates the in-vehicle personnel configuration estimated in step S103 with the service used in the vehicle identified in step S104, and uses it as in-vehicle usage history information. Thereby, the staff composition in the car and the service used in the staff composition are associated.

  In step S106, the on-vehicle control device 170 transmits the in-vehicle usage history information generated in step S105 to the data center 300. When step S106 ends, the process until the on-vehicle apparatus 100 transmits usage history information in the car to the data center 300 ends.

  Next, with reference to FIG. 5, an operation until the external device 200 transmits usage history information outside the vehicle to the data center 300 will be described. FIG. 5 is a flowchart until the external device 200 transmits usage history information outside the vehicle to the data center 300. The process of the flowchart shown in FIG. 5 is respectively executed by the external device 200 provided in each facility or each home.

  First, in step S201, the external control device 270 acquires a captured image outside the vehicle from the external camera 210. When a plurality of external cameras 210 are provided in the external device 200, the on-vehicle control device 170 acquires a captured image from each on-vehicle camera 110.

  In step S202, the external control device 270 estimates the characteristics of a person present outside the vehicle based on the captured image outside the vehicle acquired in step S201. The characteristics of the person outside the vehicle include the gender of the person outside the vehicle and the age of the person outside the vehicle. When a plurality of persons exist around the external device 200, the external control device 270 estimates features one by one. Also, in this step, the external control device 270 detects a person present around the external device 200 from the captured image, and calculates the number of people outside the vehicle.

  In step S203, the external control device 270 estimates the personnel configuration outside the vehicle. The number of people outside the vehicle calculated in step S202 and the characteristics of the person outside the vehicle estimated in the same step are associated with the personnel configuration outside the vehicle (see FIG. 2B).

  In step S204, the external control device 270 specifies a service used outside the vehicle. For example, the external control device 270 specifies, from the audio information input from the external microphone 220, the music title of the music being played in the home or in the facility.

  In step S205, the external control device 270 creates usage history information outside the vehicle as a history of the service used outside the vehicle. The external control device 270 associates the personnel configuration outside the vehicle estimated in step S203 with the service used outside the vehicle identified in step S204, and uses it as usage history information outside the vehicle. Thereby, the staff composition outside the vehicle and the service used in the staff composition are associated.

  In step S206, the external control device 270 transmits the usage history information outside the vehicle generated in step S205 to the data center 300. When step S206 ends, the process until the external device 200 transmits usage history information outside the vehicle to the data center 300 ends.

  Next, with reference to FIG. 6, an operation of identifying an optimal service for each personnel configuration will be described. FIG. 6 is a flow chart for explaining a process of identifying an optimal service for each personnel configuration. The processing of the flowchart shown in FIG. 6 is executed by the server processing device 330 of the data center 300.

  In step S301, the server processing device 330 collects in-vehicle usage history information (see FIG. 4) transmitted from the in-vehicle device 100 and usage history information outside the vehicle transmitted from the external device 200 (see FIG. 5). And store them in the database 320. The period in which the server processing device 330 collects the two types of usage history information is not particularly limited. Usage history information is accumulated in the database 320 in a format in which the used service, the number of persons, gender, and age are determined (see FIG. 3).

  In step S302, the server processing device 330 analyzes the configuration of personnel in the vehicle and the services tending to be used from the usage history information of each vehicle in the vehicle accumulated in the database 320 in step S301. For example, the server processing device 330 groups (by number, by gender, by age) each parameter constituting the information of the personnel configuration in the vehicle by clustering processing for one service used in the vehicle. Then, the server processing device 330 analyzes the tendency of one service to be used for each of the grouped, and can represent the one service by the combination of the number of persons, the gender, and the age, for one service. Calculate the feature amount. The server processing device 330 calculates the feature amount for each service by executing the above-described processing for each service.

  In addition, in this step, the server processing device 330 also analyzes the usage history information outside the vehicle, for services that tend to be used for each personnel configuration outside the vehicle, similarly to the usage history information inside the vehicle described above.

  In step S303, the server processing device 330 identifies the optimal service for each personnel configuration based on the analysis result obtained in step S302. Also, in this step, the server processing device 330 is a plurality of service candidates that may be used for each personnel configuration from the database 320 based on the analysis result obtained in step S302, and is used. The service candidates are prioritized in the descending order of possibility.

  In step S304, the server processing device 330 stores, in the database 320, the analysis result obtained in step 302 and the optimal service for each personnel configuration identified in step S303. When step S304 ends, the process in which the server processing device 330 specifies the optimal service for each personnel configuration ends.

  Next, with reference to FIGS. 7A and 7B, a process from getting on the vehicle to providing a service optimal for the personnel configuration in the vehicle will be described. FIG. 7A and FIG. 7B are flowcharts for explaining the process from getting on the vehicle to providing the service optimum for the personnel configuration in the vehicle. The processes of steps S501 to S505 and S508 shown in FIG. 7A and the processes of steps S512 to S514 shown in FIG. 7B are executed by the on-vehicle apparatus 100 mounted on a specific vehicle. Further, the processing of steps S506, S507, and S509 to S511 shown in FIG. 7A is executed by the server processing device 330 of the data center 300.

  In step S501, the on-vehicle control device 170 acquires a current captured image of the inside of the vehicle from the on-vehicle camera 110. The on-vehicle control device 170 acquires a captured image of the current in-vehicle from the on-vehicle camera 110 at, for example, timing when one or more persons get on the vehicle or when the vehicle starts moving. This step corresponds to step S101 in FIG.

  In step S502, the on-vehicle control device 170 estimates features of a person present in the current vehicle based on the current captured image in the vehicle acquired in step S501. This step corresponds to step S102 in FIG.

  In step S503, the on-vehicle control device 170 estimates the current personnel configuration in the vehicle. This step corresponds to step S103 in FIG.

  In step S504, the in-vehicle control device 170 determines whether or not a person who has the experience of getting in the past has got on the basis of the current in-vehicle captured image acquired in step S501. If a person who has a ride experience in the past is not detected in the current car, the process proceeds to step S505, and if a person having a ride experience in the past is detected, the process proceeds to step S508.

  In step S505, the on-vehicle control device 170 transmits, to the data center 300, the information on the current personnel configuration in the vehicle estimated in step S503.

  If it is determined in step S504 that a person who has experienced riding in the past is detected, the process proceeds to step S508. In step S <b> 508, the in-vehicle control device 170 transmits, to the data center 300, the information of the person who has the boarding experience detected in step S <b> 504.

  When step S505 or step S508 ends, the on-vehicle apparatus 100 stands by until a response from the data center 300 comes.

  In step S505, when the information of the current personnel configuration in the vehicle is transmitted from the in-vehicle apparatus 100 to the data center 300, the process proceeds to step S506. In step S506, the server processing device 330 receives, via the server communication device 310, the information on the current personnel configuration in the vehicle transmitted from the in-vehicle device 100 in step S505.

  In step S507, the server processing device 330 extracts an optimal service from the database 320 for the current personnel configuration in the vehicle received in step S506. Note that, in this step, the server processing device 330 may extract from the database 320 a plurality of service candidates that are likely to be used in the current personnel configuration in the vehicle. The server processing device 330 creates service guidance information including the optimal service and / or a plurality of service candidates.

  When the information of the person who has got a ride experience is transmitted from the in-vehicle apparatus 100 to the data center 300 in step S508, the process proceeds to step S509. In step S509, the server processing device 330 receives, via the server communication device 310, the information of the person with the boarding experience transmitted from the in-vehicle device 100 in step S508.

  In step S510, the server processing device 330 extracts, from the database 320, the service that a person with boarding experience has used in the past. When there are a plurality of corresponding services, the server processing device 330 can prioritize the plurality of services in descending order of usage frequency. The server processing device 330 creates service guidance information including a service that a person with boarding experience has used in the past.

  In step S511, the server processing device 330 transmits the service guidance information created in step S507 or step S510 to the in-vehicle device 100.

  In step S512, the in-vehicle control device 170 receives service guidance information from the data center 300 via the in-vehicle communication device 160.

  In step S513, the on-vehicle control device 170 causes the on-vehicle display 140 to display that the service guidance information has been received, and prompts the person in the vehicle to select whether to use the service included in the service guidance information. For example, the on-vehicle control device 170 causes the on-vehicle display 140 to display the service included in the service guidance information, and displays a selection screen as to whether or not to use the service. If the person in the car selects to use the service guidance information, the process proceeds to step S514. If the person in the car selects not to use the service guidance information, the information provision processing is ended.

  In step S514, the on-vehicle control device 170 outputs the service guidance information received in step S512. The on-vehicle control device 170 provides the optimum service included in the service guidance information in the vehicle using at least one of the on-vehicle display 140 and the on-vehicle speaker 150. For example, the on-vehicle control device 170 causes the on-vehicle speaker 150 to output the music of the song name A. Thus, for example, even in the scene where a couple with a child and a relative do not know the music of the child's age, even if the couple does not know the music of the child's age, music relative to the child of the relative can be played in the car it can.

  Further, in this step, the onboard control device 170 may list a plurality of service candidates included in the service guidance information, and cause the onboard display 140 to display a list of service candidates. Thus, the person in the vehicle can select the optimal service from among the plurality of service candidates in the vehicle.

  Further, in this step, the in-vehicle control device 170 may provide a service previously used by a person who has got in the past in the vehicle by using at least one of the in-vehicle display 140 and the in-vehicle speaker 150. When step S514 ends, the process from getting on the board to providing a service optimum for the personnel configuration in the car ends.

  As described above, in the present embodiment, the on-vehicle apparatus 100 acquires a captured image of the interior of the vehicle from the on-vehicle camera 110 that captures an interior of the vehicle, and estimates features of a person present in the vehicle based on the captured image of the interior of the vehicle. Estimate the staff composition in the car in which the features of the person in the car and the number of people in the car are associated, identify the service used in the car, and use in the car in which the staff configuration in the car and the service in the car are associated The history information is sent to the data center 300. In the data center 300, the server processing device 330 stores in-vehicle usage history information in the database 320, and analyzes the stored in-vehicle usage history information to identify the optimal service for each personnel configuration in the vehicle. . As a result, it is possible to provide services corresponding to various in-vehicle personnel configurations, and it is possible to determine the optimal service in the vehicle without spending time after getting on the vehicle.

  Further, in the present embodiment, the on-vehicle apparatus 100 estimates the sex and age of the person in the vehicle as the feature of the person in the vehicle. These features can be estimated by using only the captured image of the on-vehicle camera 110, and the parameters included in the personnel configuration can be easily estimated without using a complicated device.

  Furthermore, in the present embodiment, the external device 200 acquires a captured image of the outside of the vehicle from the external camera 210 that captures the outside of the vehicle, estimates a feature of a person outside the vehicle based on the captured image of the outside of the vehicle, Of the number of people and the characteristics of persons outside the vehicle are associated with each other to estimate the number of people outside the vehicle, identify the service used outside the vehicle, and the usage history information outside the vehicle with the number of personnel outside the vehicle and the service outside the vehicle It transmits to the data center 300. In the data center 300, the server processing device 330 stores the usage history information outside the vehicle in the database 320, and analyzes the usage history information inside the vehicle and the usage history information outside the vehicle to optimize the service for each personnel configuration in the vehicle Identify Since the usage history information outside the vehicle includes services used in facilities and homes, it is possible to provide, for example, even services adapted to the current trend.

  Further, in the present embodiment, the in-vehicle apparatus 100 estimates the feature of the person in the vehicle based on the plurality of in-vehicle captured images acquired from the plurality of in-vehicle cameras 110. Thereby, the feature of the person in the car can be accurately estimated.

  Furthermore, in the present embodiment, the in-vehicle device 100 acquires a current captured image of the in-vehicle from the in-vehicle camera 110, estimates a feature of a person existing in the current vehicle based on the captured image of the current in-vehicle, The present in-vehicle personnel composition in which the feature number of the in-vehicle and the current in-vehicle person is associated is estimated, and information on the current in-vehicle personnel composition is transmitted to the data center 300. In the data center 300, the server processing device 330 selects the service that is most suitable for the current personnel configuration in the vehicle from the optimal services that are stored in the database 320 and that are specified for each personnel configuration in the vehicle and selects the in-vehicle device 100. Send to In the in-vehicle apparatus 100, the in-vehicle control apparatus 170 provides the in-vehicle service that is optimal for the current personnel configuration received from the data center 300. Thereby, it is possible to provide the optimum service in the vehicle without spending time after getting on the vehicle.

  Further, in the present embodiment, the in-vehicle apparatus 100 urges a person in the vehicle to select whether to provide the optimal service received from the data center 300 in the vehicle. Thereby, the person in the car can use the service guidance information from the data center 300 as needed.

  Furthermore, in the present embodiment, the in-vehicle apparatus 100 transmits information of a person who has a ride experience to the data center 300 when a person who has a ride experience in the past is detected from the captured image in the vehicle captured by the in-vehicle camera 110. . The data center 300 extracts, from the database 320, a service that a person with boarding experience has used in the past, and transmits the service to the in-vehicle apparatus 100. In the in-vehicle device 100, the in-vehicle control device 170 provides the extracted service in the vehicle. In this way, it is possible to provide a service in the car that matches the preferences of each person who has a ride experience.

  Further, in the present embodiment, the server processing device 330 extracts a plurality of service candidates suitable for the current personnel configuration in the vehicle based on the analysis result stored in the database 320. In addition, the server processing device 330 prioritizes the plurality of extracted services in the descending order of the possibility of being used by the current personnel configuration in the vehicle. In the on-vehicle apparatus 100, a plurality of prioritized service candidates are presented in the vehicle. This makes it possible to propose a service close to the passenger's preference even when there is a first passenger.

  The embodiments described above are described to facilitate the understanding of the present invention, and are not described to limit the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents that fall within the technical scope of the present invention.

  For example, in the above-described embodiment, the service using at least one of the in-vehicle display 140 and the in-vehicle speaker 150 has been described as an example of the service used in the vehicle, but the service used in the vehicle is It is not limited. For example, the service used in the car may be a service using a terminal device owned by a person in the car. Examples of the terminal device include devices that can be connected to the Internet, such as a mobile phone, a smartphone, a notebook computer, and a tablet computer. Moreover, as a service using a terminal device, music reproduced using an application, a television program, a website displayed by an internet browser, etc. may be mentioned as an example.

  When the service using the terminal device is used in the car, the type of the service using the terminal device and the in-vehicle captured image captured by the in-vehicle camera 110 and the voice information collected by the in-vehicle microphone 120 Identify the specific content. Also, in this case, the transmission destination of the service guidance from the data center 300 may be a terminal device instead of the in-vehicle device 100. As described above, by targeting the service using the terminal device, the data center 300 can remove in advance the service that can not be used according to the specification of the terminal device from the analysis target. This makes it possible to prevent the presentation of service information that can not be used in practice, which does not conform to the specifications of the service output device, in the vehicle.

  Also, for example, in the embodiment described above, the target of the service specified by the external device 200 is the service provided by the external device 200 itself, but is not limited to this. For example, the external device 200 specifies a service provided by a device or device provided around the external device 200 from the captured image outside the vehicle captured by the external camera 210 and the audio information collected by the external microphone 220. Can. For example, if the external device 200 is provided in a commercial facility, the external device 200 can specify the music flowing into the commercial facility by the external microphone 220. Also, for example, if the external device 200 is provided in the home, the external device 200 can specify a program being broadcasted on a television by the external camera 210. A wide variety of services are stored in the database 320 by expanding the targets of the services specified by the external device 200. As a result, it is possible to provide the in-vehicle optimum service capable of dealing with various personnel configurations.

  Further, for example, in the above-described embodiment, the on-vehicle apparatus 100 side has described the configuration in which the feature of the person in the vehicle is estimated or the personnel configuration in the vehicle is estimated. Inferring the in-vehicle personnel configuration may be the data center 300. For example, the in-vehicle apparatus 100 may transmit an in-vehicle captured image captured by the in-vehicle camera 110 to the data center 300, and estimate the characteristics of the person in the vehicle or the personnel configuration in the vehicle on the server processing apparatus 330 side.

  Further, for example, in the embodiment described above, the information processing apparatus according to the present invention has been described by taking the configuration of the on-vehicle control device 170, the database 320, and the server processing device 330 that constitute the information providing system 1 as an example. The present invention is not limited to this. For example, the on-vehicle apparatus 100 may include the database 320 and the server processing device 330 provided in the data center 300.

  Further, for example, in the above-described embodiment, the controller according to the present invention has been described by taking the configurations of the on-vehicle control device 170 and the server processing device 330 as an example, but the present invention is not limited thereto.

DESCRIPTION OF SYMBOLS 1 ... Information provision system 100 ... Vehicle-mounted device 110 ... Vehicle-mounted camera 120 ... Vehicle-mounted microphone 130 ... Vehicle-mounted operating device group 140 ... Vehicle-mounted display 150 ... Vehicle-mounted speaker 160 ... Vehicle-mounted communication device 200 ... External device 210 ... External camera 220 ... External microphone 230 ... External operation equipment group 240 ... external display 250 ... external speaker 260 ... external communication device 300 ... data center 310 ... server communication device 320 ... database 330 ... server processing device

Claims (10)

An information processing method using a controller and a storage device, comprising
Acquire an in-vehicle captured image from a camera that captures the interior of the vehicle,
Based on the captured image in the car, the feature of the person present in the car is estimated;
Estimating the in-vehicle personnel configuration in which the number of people in the vehicle and the characteristics of the person in the vehicle are associated;
Identify the services used in the car,
Causing the storage device to store the usage history in the vehicle associated with the personnel configuration in the vehicle and the service in the vehicle;
An information processing method for specifying an optimal service for each personnel configuration in the vehicle by analyzing the accumulated usage history in the vehicle. The information processing method according to claim 1, wherein
The information processing method in which the feature of the person existing in the vehicle is gender and age. The information processing method according to claim 1 or 2,
The information processing method, wherein the service is a service available to a terminal device. The information processing method according to any one of claims 1 to 3, wherein
Acquiring an image taken outside the vehicle from a camera which images the outside of the vehicle,
Based on the captured image outside the vehicle, the feature of the person existing outside the vehicle is estimated;
Estimating the personnel composition outside the vehicle, wherein the number of people outside the vehicle and the characteristics of the person outside the vehicle are associated;
Identify the services used outside the vehicle,
Causing the storage device to store an external usage history associated with the personnel configuration outside the vehicle and the service outside the vehicle;
An information processing method for specifying an optimal service for each personnel configuration in the vehicle by analyzing the accumulated usage history in the vehicle and the usage history outside the vehicle. The information processing method according to any one of claims 1 to 4, wherein
The plurality of in-vehicle captured images are acquired from the plurality of cameras that capture the interior of the vehicle,
The information processing method which estimates the characteristic of the person who exists in the said in-vehicle based on the image pick-up picture of said several in-vehicle. The information processing method according to any one of claims 1 to 5, wherein
Obtain a current in-vehicle captured image from the camera in the vehicle;
Estimating features of a person present in the current vehicle based on the current captured image in the vehicle;
Estimating the current in-vehicle personnel configuration in which the current number of people in the vehicle and the characteristics of the current in-vehicle person are associated;
Choose the service that is most suitable for the current in-vehicle personnel configuration from the optimal services specified for each in-vehicle personnel configuration,
An information processing method for outputting the selected optimal service. The information processing method according to claim 6, wherein
An information processing method for switching whether or not to output the optimal service according to the selection of a person present in the vehicle. The information processing method according to claim 6 or 7,
The information processing method which makes the service used in the car in the past the above-mentioned optimal service, when the person who has the boarding experience in the past in the past is detected from the present captured image in the car. The information processing method according to any one of claims 6 to 8, wherein
From the information stored in the storage device, a plurality of service candidates suitable for the current personnel configuration in the vehicle are extracted;
The plurality of service candidates are prioritized in the order of high possibility of being used in the current in-vehicle personnel configuration,
An information processing method for outputting candidates for the plurality of services to which priority is given. A controller that executes information processing;
With storage
The controller
Acquire an in-vehicle captured image from a camera that captures the interior of the vehicle,
Based on the captured image in the car, the feature of the person present in the car is estimated;
Estimating the in-vehicle personnel configuration in which the number of people in the vehicle and the characteristics of the person in the vehicle are associated;
Identify the services used in the car,
Causing the storage device to store the usage history in the vehicle associated with the personnel configuration in the vehicle and the service in the vehicle;
An information processing apparatus that specifies an optimal service for each personnel configuration in the vehicle by analyzing the accumulated usage history in the vehicle.

Images