JP2021077953A - Information processing device and cooperative operation control method - Google Patents

Abstract

To provide an information processing device and cooperative operation control method capable of avoiding a state where a third person existing within a predetermined space can use a function based on a cooperative operation between the information processing device and a portable terminal even when a user goes out of the predetermined space while keeping an ON state of a power source in the information processing device.SOLUTION: An information processing device includes: an outing detection section 13 for detecting that an owner of a portable terminal 20 under wireless connection with an on-vehicle device 10 goes out of a vehicle; and a cooperative operation stop section 14 for stopping provision of a specific function (hands-free call function) based on a cooperative operation with the portable terminal 20 while keeping a wireless connection state with the portable terminal 20 when an outing of the owner is detected. When an owner using the portable terminal 20 under wireless connection with the on-vehicle device 10 inside the vehicle goes out of the vehicle, provision of a hands-free call function is automatically stopped.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing device and a cooperative operation control method, and more particularly to an information processing device that wirelessly connects to a mobile terminal and operates in cooperation with the information processing device, and a cooperative operation control method by the information processing device.

Conventionally, there has been provided a technique in which a mobile terminal such as a smartphone and an in-vehicle device are connected by Bluetooth (registered trademark), and the mobile terminal and the in-vehicle device cooperate with each other to provide a predetermined function. For example, a hands-free call function for avoiding one-handed driving during telephone operation is known. By using this hands-free call function, it is possible to perform a call operation to the other party and an answer operation when there is an incoming call from the other party through the screen of the in-vehicle device. During a call, the transmitted voice input from the in-vehicle microphone to the in-vehicle device is sent to the other party through the mobile terminal, while the received voice received from the other party in the mobile terminal is output from the in-vehicle speaker via the in-vehicle device. ..

In order to connect the mobile terminal to the in-vehicle device by Bluetooth, it is necessary to register the identification information of the mobile terminal in the in-vehicle device in advance by an operation called pairing. When pairing is performed, the mobile terminal is automatically wirelessly connected to the in-vehicle device while the mobile terminal is within the communicable range of the in-vehicle device, and communication is possible. Generally, the communicable distance of Bluetooth is several meters to several tens of meters (about 10 m according to the Class 2 standard adopted by many smartphones), and communication is possible between a mobile terminal outside the vehicle and an in-vehicle device inside the vehicle.

By the way, a passenger (driver or passenger) who is in a vehicle with a mobile terminal that has been paired with the in-vehicle device is using the hands-free call function by connecting the mobile terminal to the in-vehicle device via Bluetooth. Occasionally, you may need to get out of the car and stay around the car for a while. Cases of staying around the vehicle for a while include, for example, smoking outside the vehicle, putting in and out of luggage in the trunk room, and purchasing beverages from a vending machine.

In such a situation, even if the passenger who goes out of the car leaves his / her mobile terminal in the car, or even if he / she takes it out of the car, the Bluetooth connection between the in-vehicle device and the mobile terminal is not possible. Be maintained. For this reason, the following problems may occur when using the hands-free call function. For example, another passenger remaining in the vehicle can perform a telephone operation (outgoing operation, answering operation for an incoming call, etc.) through the screen of the in-vehicle device. In addition, when the mobile terminal taken out of the vehicle is put into a call state, the received voice from the other party is output from the in-vehicle speaker and is unintentionally heard by other passengers. In this case, if another passenger remaining in the vehicle speaks, it is possible to have a conversation with the other passenger.

During a hands-free call using the microphone and speaker installed in the hands-free device, the hands-free call is stopped when the occupant's disembarkation is detected depending on whether the ignition switch of the vehicle is turned off or not. After confirming whether the simultaneous call cancellation condition, which is presumed to be no problem, is satisfied, stop the hands-free call state and switch to the handset call state using the microphone and speaker equipped on the mobile phone. Such an in-vehicle telephone system is known (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2003-78607

When the technique described in Patent Document 1 is used, if the passenger turns off the ignition switch of the vehicle and goes out of the vehicle, the hands-free call state is automatically stopped and switched to the handset call state. There is no problem. However, if the passenger goes out of the vehicle with the ignition switch of the vehicle turned on, the above-mentioned problem cannot be avoided.

The present invention has been made to solve such a problem, and information is provided when a user who uses a mobile terminal connected to an information processing device in a predetermined space goes out of the predetermined space. To make it possible to avoid a situation in which other people in a predetermined space can use functions based on the cooperative operation between the information processing device and the mobile terminal even when the power of the processing device is turned on. With the goal.

In order to solve the above-mentioned problems, in the present invention, in an information processing device that connects to a mobile terminal and operates in cooperation with the mobile terminal, the mobile terminal that is wirelessly connected in a predetermined space or the mobile terminal that is wirelessly connected or wired is connected. When the owner detects that he / she has gone out of the predetermined space, he / she stops providing a specific function based on the cooperation operation with the mobile terminal while maintaining the state of connection with the mobile terminal.

According to the present invention configured as described above, the owner who uses the mobile terminal connected to the information processing device in the predetermined space goes out of the predetermined space with or without the mobile terminal. Then, it is detected that the mobile terminal or its owner has gone out of the predetermined space, and the provision of a specific function based on the cooperative operation is automatically stopped. As a result, even if the power of the information processing device is turned on, a specific function based on the cooperative operation between the information processing device and the mobile terminal can be used by another person in a predetermined space. It can be avoided.

It is a figure which shows the configuration example of the wireless communication system using the information processing apparatus by this embodiment. It is a figure for demonstrating the operation example of the in-vehicle device which is an example of the information processing apparatus by this Embodiment. It is a block diagram which shows the functional structure example of the in-vehicle device which is an example of the information processing apparatus by this Embodiment. It is a figure which shows an example of the display screen of an in-vehicle device in order to explain the operation example of the cooperation operation stop part by this Embodiment. It is a flowchart which shows the operation example of the in-vehicle device by this Embodiment. It is a block diagram which shows the functional structure example of the in-vehicle device which concerns on 1st modification. It is a block diagram which shows the functional structure example of the in-vehicle device which concerns on 2nd modification.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration example of a wireless communication system using the information processing device according to the present embodiment. As shown in FIG. 1, the wireless communication system of the present embodiment is composed of an in-vehicle device 10 which is an example of an information processing device and a mobile terminal 20. The mobile terminal 20 is, for example, a smartphone, and has a plurality of applications for providing various functions such as a telephone function, a music playback function, a navigation function using a map, an Internet connection function, an email function, and a chat function. ..

The in-vehicle device 10 and the mobile terminal 20 are wirelessly connected by Bluetooth, and are configured to cooperate with each other to provide a predetermined function. That is, the vehicle-mounted device 10 of the present embodiment is wirelessly connected to the mobile terminal 20 by Bluetooth and operates in cooperation with the mobile terminal 20.

In the present embodiment, one of the functions that can be provided by the cooperative operation is the hands-free call function. As described above, when the hands-free call function is used, it is possible to perform a call operation to the other party and a response operation when there is an incoming call from the other party through the screen of the vehicle-mounted device 10. During a call, the transmitted voice input from the in-vehicle microphone to the in-vehicle device 10 is sent to the other party through the mobile terminal 20, while the received voice received from the other party in the mobile terminal 20 is sent to the in-vehicle speaker via the in-vehicle device 10. Is output from.

Further, in the present embodiment, a music reproduction function is also provided as a function that can be provided by the cooperative operation between the in-vehicle device 10 and the mobile terminal 20. By using the music reproduction function based on the cooperative operation, it is possible to transmit the music data stored in the mobile terminal 20 to the vehicle-mounted device 10 and output the music sound reproduced by the vehicle-mounted device 10 from the vehicle-mounted speaker. In addition, the operation of transmitting the list data of the music (data including information such as the music name and the artist name) stored in the mobile terminal 20 to the in-vehicle device 10 to select the desired music, and the reproduction of the selected music. It is also possible to perform various operations such as pause and playback stop through the screen of the in-vehicle device 10.

In the present embodiment, it is assumed that the mobile terminal 20 implements the Bluetooth function of Class2. In this case, the wireless communication possible distance between the vehicle-mounted device 10 and the mobile terminal 20 is about 10 m. Therefore, as shown in FIG. 1, even when the mobile terminal 20 is outside the vehicle, the wireless connection between the vehicle-mounted device 10 and the mobile terminal 20 can be performed as long as it is within the wireless communication range within about 10 m from the vehicle-mounted device 10. It is maintained, and it is possible to communicate and perform cooperative operation.

Here, an operation example of the present embodiment will be schematically described with reference to FIG. FIG. 2A shows a state in which the driver and other passengers are in the vehicle and the driver wirelessly connects his / her own mobile terminal 20 to the in-vehicle device 10 to use the hands-free call function. Shown. FIG. 2B shows a state in which the driver using the hands-free call function is required to go out of the vehicle. At this time, the driver is out of the vehicle with the mobile terminal 20, but since he is within the wireless communication range from the in-vehicle device 10, the Bluetooth wireless connection is maintained.

As shown in FIG. 2B, the vehicle-mounted device 10 detects that the mobile terminal 20 or the driver who owns the mobile terminal 20 has gone out of the vehicle, and stops providing the hands-free call function. As a result, the passenger remaining in the vehicle cannot perform operations related to the hands-free call function through the operation screen displayed on the display of the vehicle-mounted device 10. Further, the received voice received by the mobile terminal 20 from the other party is not output from the vehicle-mounted speaker via the vehicle-mounted device 10.

FIG. 3 is a block diagram showing a functional configuration example of the vehicle-mounted device 10 according to the present embodiment. As shown in FIG. 3, the vehicle-mounted device 10 of the present embodiment has a wireless connection unit 11, an existence detection unit 12, an outing detection unit 13, a cooperative operation stop unit 14, a return detection unit 15, and a cooperative operation start unit as functional configurations. 16. A hands-free call control unit 17 and a music reproduction control unit 18 are provided.

Each of the above functional blocks 11 to 18 can be configured by any of hardware, DSP (Digital Signal Processor), and software. For example, when configured by software, each of the above functional blocks 11 to 18 is actually configured to include a computer CPU, RAM, ROM, etc., and is a program stored in a recording medium such as RAM, ROM, hard disk, or semiconductor memory. Is realized by the operation of.

The wireless connection unit 11 is an example of a “connection unit” within the scope of claims, and wirelessly connects to a mobile terminal 20 within a wireless communication range. In the present embodiment, the wireless connection unit 11 wirelessly connects to the mobile terminal 20 that has been paired with Bluetooth and is within the wireless communication range. The in-vehicle device 10 stores the terminal identification information of the mobile terminal 20 that has been paired, and the wireless connection unit 11 confirms whether or not the communication partner is the paired mobile terminal 20 by the terminal identification information. After that, connect wirelessly.

The hands-free call control unit 17 provides a hands-free call function in cooperation with the mobile terminal 20 wirelessly connected by the wireless connection unit 11. That is, the hands-free call control unit 17 displays an operation screen for performing an operation for making a call to the other party and an operation for answering an incoming call from the other party on the display of the vehicle-mounted device 10. Then, when an operation is performed through the operation screen, a process corresponding to the operation content is executed. Further, the hands-free call control unit 17 transmits the transmitted voice input from the in-vehicle microphone to the in-vehicle device 10 to the other party through the mobile terminal 20 during a call with the other party, while the other party transmits the transmitted voice to the other party through the mobile terminal 20. The received received voice is output from the in-vehicle speaker.

The music reproduction control unit 18 provides a music reproduction function in cooperation with the mobile terminal 20 wirelessly connected by the wireless connection unit 11. That is, the music playback control unit 18 displays an operation screen for performing various operations such as music selection, playback, pause, and playback stop based on the music list data sent from the mobile terminal 20 on the vehicle-mounted device 10. Display on the display of. Then, when an operation is performed through the operation screen, a process corresponding to the operation content is executed. Further, the music reproduction control unit 18 reproduces the music data sent from the mobile terminal 20 and outputs the reproduced sound from the vehicle-mounted speaker.

The presence detection unit 12 detects the presence of a person in a predetermined place in a predetermined space which is narrower than the wireless communication range from the vehicle-mounted device 10. In the present embodiment, it is assumed that the inside of the predetermined space is inside the vehicle, the outside of the predetermined space is outside the vehicle, and the predetermined place in the predetermined space is the driver's seat. That is, the presence detection unit 12 detects the presence of a person in the driver's seat. For example, the presence detection unit 12 detects the presence of a person in the driver's seat by determining whether or not a person is seated in the driver's seat based on the output value of the weight sensor installed in the driver's seat.

Alternatively, the presence detection unit 12 analyzes an image taken by a camera installed so as to take a picture of the driver's seat, and determines whether or not a person is in the driver's seat, thereby determining the presence of a person in the driver's seat. May be detected. As another example, the presence detection unit 12 detects the presence of a person in the driver's seat by determining whether or not the steering wheel is gripped based on the output value of the contact sensor installed on the steering wheel. May be good. In addition, the presence of a person in the driver's seat may be detected based on the output values of the infrared sensor and the motion sensor. The detection methods listed here are examples, and are not limited thereto.

The outing detection unit 13 detects that the mobile terminal 20 or the owner of the mobile terminal 20 that is wirelessly connected by the wireless connection unit 11 has gone out of the vehicle in a vehicle that is narrower than the wireless communication range from the in-vehicle device 10. To do. In the present embodiment, the outing detection unit 13 changes from a state in which a person is detected in the driver's seat to a state in which a person is not detected by the presence detection unit 12, and thus the mobile terminal 20 or the mobile terminal 20 wirelessly connected in the vehicle. Detects that the owner of the car has gone out of the car.

In the present embodiment, it is assumed that the driver basically uses the hands-free call function, and by determining whether the driver is seated or away from the driver's seat, the wireless communication is performed. It is designed to detect that the connected mobile terminal 20 or the driver who owns the terminal 20 has gone out of the vehicle from the inside of the vehicle. What the presence detection unit 12 can actually detect is that the driver has gone out of the vehicle, but the driver is in the driver's seat on the assumption that there is a high possibility that he / she will go out of the vehicle with the mobile terminal 20. When not seated, the mobile terminal 20 is regarded as having gone out of the vehicle. The driver may leave the mobile terminal 20 in the car and go out of the car, but even in that case, the hands-free call function may be stopped, so that when the driver is away from the driver's seat. Even if it is considered that the mobile terminal 20 has gone out of the vehicle, there is no practical problem.

When the outing detection unit 13 detects the outing of the mobile terminal 20 or the driver who owns the mobile terminal 20, the cooperative operation stop unit 14 maintains the state of the wireless connection with the mobile terminal 20 by the wireless connection unit 11. The hands-free call control unit 17 will stop providing the hands-free call function (corresponding to "a specific function based on cooperative operation" in the claims).

Specifically, the cooperative operation stop unit 14 stops providing the function of outputting the received voice from the vehicle-mounted speaker into the vehicle based on the voice information transmitted from the mobile terminal 20. In this case, the cooperation operation stop unit 14 may stop receiving the voice information of the received voice from the mobile terminal 20. Alternatively, although the cooperative operation stop unit 14 receives the voice information of the received voice from the mobile terminal 20, the voice reproduction process may be stopped based on the voice information.

Further, the cooperation operation stop unit 14 stops providing a function of displaying the user interface for operating the cooperation operation related to the hands-free call function on the display screen in an operable state. FIG. 4 is a diagram simply showing an example of the display screen in this case. FIG. 4A shows an example of a display screen when the driver who is the owner of the mobile terminal 20 is in the vehicle and the provision of the hands-free call function is not stopped. On this display screen, an operation icon 41 for activating the hands-free call function is displayed. When the operation icon 41 is operated, an operation screen for performing an operation for making a call to the other party, an operation for answering an incoming call from the other party, and the like is displayed on the display.

On the other hand, FIG. 4B shows an example of a display screen when the driver who is the owner of the mobile terminal 20 is out of the vehicle and the provision of the hands-free call function is stopped. On this display screen, the operation icon 41 for activating the hands-free call function is not displayed. If the driver goes out of the vehicle while the operation screen for performing the outgoing call operation or the answering operation is displayed, the button for performing the outgoing call operation or the answering operation is hidden on the operation screen. Therefore, the passengers remaining in the vehicle cannot perform operations related to the hands-free call function.

Here, an example of hiding the operation icon 41 of the hands-free call function or the button for performing a outgoing call operation, an answering operation, or the like has been described, but the present invention is not limited to this. For example, while displaying these user interfaces in a grayed out state, operations on the user interface may not be accepted.

The cooperative operation stop unit 14 stops the hands-free call control unit 17 from providing the hands-free call function, but the music reproduction control unit 18 does not stop the provision of the music reproduction function. Therefore, when the mobile terminal 20 is connected to the in-vehicle device 10 when the driver is in the vehicle and the music of the mobile terminal 20 is output from the in-vehicle speaker by using the music reproduction function by the cooperative operation, the driver Even if you go out of the car, the music playback function by the cooperative operation is continuously provided, and the music sound is continuously output from the in-vehicle speaker.

In the return detection unit 15, after the provision of the hands-free call function is stopped by the cooperative operation stop unit 14, the mobile terminal 20 or its owner whose outing is detected by the outing detection unit 13 has returned from the outside of the vehicle to the inside of the vehicle. Is detected. Specifically, the return detection unit 15 causes the mobile terminal 20 or its owner to return from the outside of the vehicle to the inside of the vehicle when the presence detection unit 12 changes from a state in which no person is detected in the driver's seat to a state in which a person is detected. Detect that it has come.

When the return detection unit 15 detects the return of the mobile terminal 20 or the driver who owns the mobile terminal 20, the cooperation operation start unit 16 releases the stop state of the cooperation operation stop unit 14, and the hands-free call control unit 17 Resumes the provision of hands-free calling function by. Specifically, the cooperative operation start unit 16 resumes providing the function of outputting the received voice from the vehicle-mounted speaker into the vehicle based on the voice information transmitted from the mobile terminal 20. Further, the cooperation operation start unit 16 resumes providing a function of displaying the user interface for operating the cooperation operation related to the hands-free call function on the display screen in an operable state.

FIG. 5 is a flowchart showing an operation example of the vehicle-mounted device 10 configured as described above. The flowchart shown in FIG. 5 starts when the driver having the paired mobile terminal 20 gets on the vehicle and turns on the power of the on-board unit 10.

First, the wireless connection unit 11 wirelessly connects to the mobile terminal 20 in the vehicle when the power of the vehicle-mounted device 10 is turned on (step S1). Next, the presence detection unit 12 detects that a person (driver) is seated in the driver's seat (step S2). After that, the outing detection unit 13 determines whether or not the driver has gone out of the vehicle by determining whether or not the presence detection unit 12 has changed from a state in which a person is detected in the driver's seat to a state in which a person is not detected. (Step S3).

Here, if the outing detection unit 13 does not detect that the driver has gone out of the vehicle, the process proceeds to step S7. On the other hand, when the outing detection unit 13 detects that the driver has gone out of the vehicle, the cooperative operation stop unit 14 makes a hands-free call while maintaining the state of the wireless connection with the mobile terminal 20 by the wireless connection unit 11. The control unit 17 stops providing the hands-free call function (step S4).

After that, the return detection unit 15 determines whether or not the presence detection unit 12 has changed from a state in which no person is detected in the driver's seat to a state in which a person is detected, and thus whether or not the driver has returned from the outside of the vehicle to the inside of the vehicle. (Step S5). The determination in step S5 is executed until it is detected that the driver has returned to the vehicle. Then, when the return detection unit 15 detects that the driver has returned to the vehicle, the cooperation operation start unit 16 resumes providing the hands-free call function by the hands-free call control unit 17 (step S6).

After that, the vehicle-mounted device 10 determines whether or not the power is turned off (step S7). If the power of the vehicle-mounted device 10 is not turned off, the process returns to step S3. On the other hand, when the power of the vehicle-mounted device 10 is turned off, the processing of the flowchart shown in FIG. 5 ends.

As described in detail above, in the present embodiment, in the in-vehicle device 10 that wirelessly connects to the mobile terminal 20 and operates in cooperation with the mobile terminal 20, the mobile terminal 20 or its owner is wirelessly connected in the vehicle, which is narrower than the wireless communication range. When it is detected that the driver has gone out of the vehicle, a specific function (hands-free call function) based on the cooperative operation with the mobile terminal 20 is provided while maintaining the state of wireless connection with the mobile terminal 20. I try to stop it.

According to the present embodiment configured in this way, when a driver who is using the mobile terminal 20 wirelessly connected to the in-vehicle device 10 in the vehicle goes out of the vehicle with or without the mobile terminal 20, the cooperation is established. The provision of the hands-free call function based on the operation is automatically stopped. As a result, even if the power of the in-vehicle device 10 is left on, the passengers in the vehicle can use the hands-free call function based on the cooperative operation between the in-vehicle device 10 and the mobile terminal 20. It can be avoided.

That is, it is possible to prevent the passenger remaining in the vehicle from performing a telephone operation (outgoing operation, answering operation for incoming call, etc.) through the screen of the vehicle-mounted device 10. Further, when the driver starts a call by the mobile terminal 20 taken out of the vehicle, the received voice from the other party is not output from the in-vehicle speaker, so that the passenger remaining in the vehicle hears the call. Can be avoided. It is also possible to prevent the passenger from speaking and having a conversation with the other party.

Further, in the present embodiment, when it is detected that the driver has returned from the outside of the vehicle to the inside of the vehicle, the provision of the hands-free call function is resumed. Even when the driver goes out of the vehicle and the provision of the hands-free call function is stopped, the Bluetooth connection between the in-vehicle device 10 and the mobile terminal 20 is maintained, so that the driver returns from the outside of the vehicle to the inside of the vehicle. There is no need to make a new Bluetooth connection at that time. Therefore, the hands-free call function can be restarted smoothly. For example, it is possible to make a call on the mobile terminal 20 until just before the driver sits in the driver's seat, and switch to hands-free at the same time as sitting down to make a call.

<First modification>
In the above embodiment, it is assumed that the driver basically uses the hands-free call function, and depending on whether or not the driver is seated in the driver's seat, the mobile terminal 20 or the wirelessly connected mobile terminal 20 or Although an example of detecting that the driver who is the owner of the vehicle has gone out of the vehicle from the inside of the vehicle has been described, the present invention is not limited to this. For example, assuming that the driver does not always use the hands-free call function, a user who is using the hands-free call function is specified, and the specified user goes out of the car from the inside of the car. May be detected.

FIG. 6 is a block diagram showing a functional configuration example of the vehicle-mounted device 10'related to such a first modification. In FIG. 6, those having the same reference numerals as those shown in FIG. 3 have the same functions, and therefore, duplicate description will be omitted here. As shown in FIG. 6, the vehicle-mounted device 10'according to the first modification further includes an owner information identification unit 61 and an existence location detection unit 62 as a functional configuration, and also includes an existence detection unit 12 instead of the existence detection unit 12. It has 12'. Further, the vehicle-mounted device 10'according to the first modification includes a user information storage unit 60 as a storage medium. Further, the camera 30 is connected to the vehicle-mounted device 10'according to the first modification via wiring.

The user information storage unit 60 associates the terminal identification information of the mobile terminal 20 used when the wireless connection unit 11 wirelessly connects with the mobile terminal 20 with the user-related information that can identify the owner of the mobile terminal 20. Stores user information consisting of. The terminal identification information is information registered when the mobile terminal 20 is paired with the in-vehicle device 10, and is unique identification information stored in the mobile terminal 20. The user-related information that can identify the owner of the mobile terminal 20 is, for example, a face image of the owner or a feature amount obtained by analyzing the face image. Hereinafter, it will be described assuming that the face image is used as the user-related information. After pairing the in-vehicle device 10 and the mobile terminal 20, it is possible to transmit the data of the owner's face image stored in the mobile terminal 20 to the in-vehicle device 10 and store it in the user information storage unit 60. Is.

The camera 30 is installed so as to photograph the entire area inside the vehicle. As a result, the camera 30 can capture facial images of all passengers (drivers and passengers) in the vehicle. The image taken by the camera 30 is supplied to the location detection unit 62.

The owner information identification unit 61 is based on the terminal identification information used for wireless connection with the mobile terminal 20 by the wireless connection unit 11 and the user information stored in the user information storage unit 60, and the wireless connection unit 11 Identifyes user-related information of the owner of the mobile terminal 20 that is wirelessly connected. That is, the owner information identification unit 61 refers to the user information stored in the user information storage unit 60 based on the terminal identification information used for wireless connection with the mobile terminal 20 by the wireless connection unit 11. The user-related information (face image of the owner of the mobile terminal 20 during wireless connection) associated with the terminal identification information is specified.

The presence location detection unit 62 acquires a face image of one or more passengers existing in the vehicle by analyzing the captured image in the vehicle supplied from the camera 30, and also acquires the face image of each of the passengers in the vehicle. To identify. Then, the location detection unit 62 uses the user-related information (face image of the owner of the mobile terminal 20 wirelessly connected) specified by the owner information identification unit 61 to wirelessly connect the mobile terminal 20 in the vehicle. Detects where the owner of the is located. That is, the location detection unit 62 captures a face image that is determined to match the face image specified by the owner information identification unit 61 among the locations of one or more passengers identified by the analysis of the captured image. The existing location is specified as the existing location of the owner of the mobile terminal 20 being wirelessly connected.

The presence detection unit 12'detects the presence of a person at the place detected by the presence location detection unit 62. For example, the presence detection unit 12'detects the presence of a person in the place detected by the presence location detection unit 62 by analyzing the image taken by the camera 30 installed so as to photograph the entire interior of the vehicle. Alternatively, the presence of a person at the location is determined by determining whether or not a person is seated at the location detected by the location detection unit 62 based on the output value of the weight sensor installed in each seat of the vehicle. It may be detected. In addition, the presence of a person at the location detected by the location detection unit 62 may be detected based on the output values of the infrared sensor and the motion sensor.

<Second modification>
In the first modification, the configuration is shown in which the owner of the mobile terminal 20 wirelessly connected to the vehicle-mounted device 10 is detected and the presence of a person is detected at that location, but the location is not detected. It is also possible to apply the method. FIG. 7 is a block diagram showing a functional configuration example of the vehicle-mounted device 10 ”corresponding to the second modification in that case. In FIG. 7, those having the same reference numerals as those shown in FIG. 6 are the same. Since it has a function, a duplicate description will be omitted here.

As shown in FIG. 7, the on-board unit 10 ”corresponding to the second modification includes an owner detection unit 72 instead of the location detection unit 62, and goes out detection instead of the outing detection unit 13 and the return detection unit 15. The vehicle-mounted device 10 "according to the second modification does not include the presence detection unit 12'shown in FIG.

The owner detection unit 72 owns the mobile terminal 20 wirelessly connected in the vehicle by using the user-related information (face image of the owner of the mobile terminal 20 wirelessly connected) specified by the owner information identification unit 61. Detect a person. That is, the owner detection unit 72 is identified by the owner information identification unit 61 from the facial images of one or more passengers existing in the vehicle by analyzing the captured image in the vehicle supplied from the camera 30. A face image that is determined to match the face image is detected, and the passenger of the face image is detected as the owner of the mobile terminal 20 that is wirelessly connected.

The outing detection unit 13 ”has changed from the state in which the owner of the mobile terminal 20 wirelessly connected is detected by the owner detection unit 72 to the state in which the owner is not detected, so that the mobile terminal 20 or its wirelessly connected in the vehicle is connected. It detects that the owner has gone out of the vehicle. That is, the outing detection unit 13 "changes the state in which the owner's face image detected by the owner detection unit 72 is not shown in the captured image. With the change, it is detected that the mobile terminal 20 or its owner, which is wirelessly connected in the vehicle, has gone out of the vehicle.

The return detection unit 15 ”has changed from a state in which the owner of the mobile terminal 20 being wirelessly connected has not been detected by the owner detection unit 72 to a state in which the owner has detected the mobile terminal 20 or its owner from outside the vehicle. Detecting that the vehicle has returned to the vehicle, that is, the return detection unit 15 ”changes from a state in which the owner's face image detected by the owner detection unit 72 is not shown in the captured image to a state in which the owner's face image is shown. This detects that the mobile terminal 20 or its owner has returned from the outside of the vehicle to the inside of the vehicle.

When configured as in the first modification or the second modification, the same applies not only when the driver who connects the mobile terminal 20 to the in-vehicle device 10 and uses the hands-free call function goes out of the vehicle. Even when a passenger using the hands-free call function goes out of the vehicle, it is possible to avoid a situation in which the passenger in the vehicle can use the hands-free call function based on the cooperative operation. In addition, when the passenger (driver or passenger) who is using the cooperative operation returns from the outside of the vehicle to the inside of the vehicle, the hands-free call function can be smoothly restarted.

In the above embodiment, an example of detecting whether or not the owner of the mobile terminal 20 being wirelessly connected has gone out of the vehicle or returned from the outside of the vehicle by detecting the presence of a passenger in the vehicle will be described. However, the present invention is not limited to this. For example, the presence detection unit 12 shown in FIG. 3 is provided with a radio wave intensity detection unit, and the wireless connection unit 11 periodically detects the intensity of the radio wave received from the mobile terminal 20 being wirelessly connected. Then, when the radio wave intensity detected by the radio wave intensity detection unit becomes smaller by a difference of a predetermined value or more, it is detected that the mobile terminal 20 or its owner who is wirelessly connected in the vehicle has gone out of the vehicle. You may. In addition, when the radio field intensity detected by the radio wave strength detection unit increases with a difference of a predetermined value or more, it is detected that the mobile terminal 20 or its owner in wireless connection has returned from the outside of the vehicle to the inside of the vehicle. You may.

Further, in the above embodiment, when it is detected that the mobile terminal 20 or its owner has gone out of the vehicle, the provision of the hands-free call function is stopped, but the provision of the music playback function is not stopped. However, the provision of the music playback function may be stopped. In this case, the cooperative operation stop unit 14 stops providing the function of outputting the music sound from the vehicle-mounted speaker into the vehicle based on the voice information transmitted from the mobile terminal 20, for example. At this time, a fade-out process may be performed in which the volume of the music sound being output from the vehicle-mounted speaker is gradually reduced. Similarly, when resuming the provision of the music playback function that has been stopped, a fade-in process may be performed in which the volume of the music sound output from the vehicle-mounted speaker is gradually increased. Further, the cooperation operation stop unit 14 may stop providing the function of displaying the user interface for operating the cooperation operation on the display screen regarding the music reproduction function.

Further, in the above embodiment, a hands-free call function and a music playback function are examples of functions based on the cooperative operation between the vehicle-mounted device 10 (10', 10 ") (hereinafter, simply referred to as the vehicle-mounted device 10) and the mobile terminal 20. However, the above two are not limited to this. The in-vehicle device 10 and the mobile terminal 20 should be linked with each other for the navigation function using the map, the Internet connection function, the mail function, the chat function, and the like. It is also possible to control the stop / restart of these functions when configured in. In this case, the cooperative operation stop unit 14 is a mobile terminal 20 as a specific function based on the cooperative operation. The provision of the function of displaying an image on the display screen based on the image information transmitted from the user may be stopped.

Further, in the above embodiment, the vehicle interior is cited as an example of the predetermined space, and the vehicle-mounted device 10 is cited as an example of the information processing device, but the present invention is not limited thereto. For example, a predetermined space may be one room in a house, and the information processing device may be a television, a home audio device, an image reproduction device, or the like installed in the room.

Further, in the above embodiment, an example of wirelessly connecting the vehicle-mounted device 10 and the mobile terminal 20 by Bluetooth has been described, but the present invention is not limited to this. For example, a wireless LAN such as Wi-Fi may be used for wireless connection. Alternatively, the vehicle-mounted device 10 and the mobile terminal 20 may be connected by wire with a cable or the like.

When the in-vehicle device 10 and the mobile terminal 20 are connected by wire, a wired connection unit that connects the mobile terminal 20 by wire in a predetermined space is used instead of the wireless connection unit 11. The outing detection unit 13, 13 ”has changed from a state in which a person is detected in a predetermined space by the presence detection units 12, 62 to a state in which a person is not detected, or the owner of the mobile terminal 20 is the owner detection unit 72. It is detected that the owner of the mobile terminal 20 being wiredly connected by the wired connection unit has gone out of the predetermined space in the predetermined space by changing from the state detected by When the owner's outing is detected by the outing detection units 13 and 13 ”, the cooperation operation stop unit 14 has a specific function based on the cooperation operation with the mobile terminal 20 while maintaining the state of the wired connection by the wired connection unit. Stop offering.

Further, the return detection units 15 and 15 "have changed from a state in which a person is not detected in a predetermined space to a state in which a person is detected by the presence detection units 12 and 62, or the owner of the mobile terminal 20 is the owner. The owner of the mobile terminal 20 being wiredly connected by the wired connection unit in the predetermined space has returned from the outside to the inside of the predetermined space due to the change from the state not detected by the detection unit 72 to the detected state. Further, when the return of the owner is detected by the return detection units 15 and 15 ”, the cooperation operation start unit 16 releases the stop state by the cooperation operation stop unit 14 and connects with the mobile terminal 20. Resume providing specific functions based on linked operations.

In the case of wireless connection, regardless of whether the owner of the mobile terminal 20 takes the mobile terminal 20 out of the predetermined space or does not carry it, the state of the wireless connection is within the wireless communication range. Can be maintained. On the other hand, in the case of the wired connection, when the owner of the mobile terminal 20 takes the mobile terminal 20 out of the predetermined space, the vehicle-mounted device 10 and the mobile terminal 20 are not connected to each other. The cooperation operation between the mobile terminal 20 and the mobile terminal 20 is naturally stopped. Therefore, in the case of a wired connection, the processing of the cooperation operation stop unit 14 and the cooperation operation start unit 16 is performed only when the owner of the mobile terminal 20 goes out of the predetermined space without holding the mobile terminal 20.

In addition, the above embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from its gist or its main features.

10,10', 10 "on-board unit (information processing device)
11 Wireless connection (connection)
12, 12'Existence detection unit 13, 13 "Outing detection unit 14 Coordination operation stop unit 15, 15" Return detection unit 16 Coordination operation start unit 17 Hands-free call control unit 18 Music playback control unit 20 Mobile terminal 60 User information storage unit 61 Owner information identification unit 62 Location detection unit 72 Owner detection unit

Claims (17)

An information processing device that connects to a mobile terminal and operates in cooperation with the mobile terminal.
The connection part that connects to the above mobile terminal and
An outing detection unit that detects that the owner of a mobile terminal that is wirelessly connected by the connection unit in a predetermined space or a mobile terminal that is wirelessly or wired is out of the predetermined space.
When the mobile terminal or the owner's outing is detected by the outing detection unit, the cooperation for stopping the provision of a specific function based on the cooperation operation with the mobile terminal while maintaining the connection state by the connection unit. An information processing device characterized by having an operation stop unit. The connection unit is a wireless connection unit that wirelessly connects to the mobile terminal within the wireless communication range.
In the outing detection unit, a mobile terminal that is wirelessly connected by the wireless connection unit or the owner of the mobile terminal has gone out of the predetermined space in the predetermined space that is narrower than the wireless communication range. Detect that
The cooperation operation stop unit is based on the cooperation operation with the mobile terminal while maintaining the state of wireless connection by the wireless connection unit when the mobile terminal or the owner goes out is detected by the outing detection unit. The information processing apparatus according to claim 1, wherein the provision of a specific function is stopped. The connection unit is a wired connection unit that connects to the mobile terminal by wire in the predetermined space.
The outing detection unit detects that the owner of the mobile terminal, which is connected by the wired connection unit in the predetermined space, has gone out of the predetermined space.
When the owner's outing is detected by the outing detection unit, the cooperation operation stop unit has a specific function based on the cooperation operation with the mobile terminal while maintaining the state of the wired connection by the wired connection unit. The information processing apparatus according to claim 1, wherein the provision is stopped. After the provision of the specific function is stopped by the cooperation operation stop unit, the mobile terminal or the owner whose outing is detected by the outing detection unit has returned from the outside to the inside of the predetermined space. The return detector to detect and
The information according to claim 1, further comprising a cooperative operation start unit that resumes the provision of the specific function when the return of the mobile terminal or the owner is detected by the return detection unit. Processing equipment. A presence detection unit for detecting the presence of a person in a predetermined place in the predetermined space is further provided.
The outing detection unit detects that the mobile terminal or the owner has gone out of the predetermined space by changing from a state in which a person is detected by the presence detection unit to a state in which a person is not detected. The information processing apparatus according to claim 1. User information storage that stores user information that is associated with the terminal identification information of the mobile terminal used when the connection unit connects to the mobile terminal and the user-related information that can identify the owner of the mobile terminal. Department and
Based on the terminal identification information used to connect to the mobile terminal by the connection unit and the user information stored in the user information storage unit, the user-related information of the owner of the mobile terminal connected by the connection unit. Owner information identification department that identifies information, and
An existence location detection unit that detects a location where the owner exists in the predetermined space by using the user-related information of the owner specified by the owner information identification unit.
It is further provided with an existence detection unit that detects the presence of a person at the location detected by the existence location detection unit.
The outing detection unit detects that the mobile terminal or the owner has gone out of the predetermined space by changing from a state in which a person is detected by the presence detection unit to a state in which a person is not detected. The information processing apparatus according to claim 1. User information storage that stores user information that is associated with the terminal identification information of the mobile terminal used when the connection unit connects to the mobile terminal and the user-related information that can identify the owner of the mobile terminal. Department and
Based on the terminal identification information used to connect to the mobile terminal by the connection unit and the user information stored in the user information storage unit, the user-related information of the owner of the mobile terminal connected by the connection unit. Owner information identification department that identifies information, and
An owner detection unit that detects the owner in the predetermined space by using the user-related information of the owner specified by the owner information identification unit is further provided.
The outing detection unit detects that the mobile terminal or the owner has gone out of the predetermined space by changing from a state in which the owner is detected by the owner detection unit to a state in which the owner is not detected. The information processing apparatus according to claim 1, wherein the information processing apparatus is used. A presence detection unit for detecting the presence of a person in a predetermined place in the predetermined space is further provided.
The return detection unit indicates that the mobile terminal or the owner has returned from the outside to the inside of the predetermined space due to the change from the state in which the person is not detected by the presence detection unit to the state in which the person is detected. The information processing apparatus according to claim 4, wherein the information processing apparatus is to be detected. User information storage that stores user information that is associated with the terminal identification information of the mobile terminal used when the connection unit connects to the mobile terminal and the user-related information that can identify the owner of the mobile terminal. Department and
Based on the terminal identification information used to connect to the mobile terminal by the connection unit and the user information stored in the user information storage unit, the user-related information of the owner of the mobile terminal connected by the connection unit. Owner information identification department that identifies information, and
An existence location detection unit that detects a location where the owner exists in the predetermined space by using the user-related information of the owner specified by the owner information identification unit.
It is further provided with an existence detection unit that detects the presence of a person at the location detected by the existence location detection unit.
The return detection unit indicates that the mobile terminal or the owner has returned from the outside to the inside of the predetermined space due to the change from the state in which the person is not detected by the presence detection unit to the state in which the person is detected. The information processing apparatus according to claim 4, wherein the information processing apparatus is to be detected. User information storage that stores user information that is associated with the terminal identification information of the mobile terminal used when the connection unit connects to the mobile terminal and the user-related information that can identify the owner of the mobile terminal. Department and
Based on the terminal identification information used to connect to the mobile terminal by the connection unit and the user information stored in the user information storage unit, the user-related information of the owner of the mobile terminal connected by the connection unit. Owner information identification department that identifies information, and
An owner detection unit that detects the owner in the predetermined space by using the user-related information of the owner specified by the owner information identification unit is further provided.
The return detection unit changes from a state in which the owner is not detected by the owner detection unit to a state in which the owner is detected, so that the mobile terminal or the owner returns from the outside to the inside of the predetermined space. The information processing apparatus according to claim 4, wherein the information processing device is characterized in that it detects that. The claim is characterized in that the cooperation operation stop unit stops providing a function of outputting voice to the predetermined space based on voice information transmitted from the mobile terminal as a specific function based on the cooperation operation. The information processing apparatus according to 1. As a specific function based on the linked operation, the linked operation stop unit is characterized in that it stops providing a function of displaying a user interface for operating the linked operation on a display screen in an operable state. Item 1. The information processing apparatus according to item 1. Claim 1 is characterized in that the cooperation operation stop unit stops providing a function of displaying an image on a display screen based on image information transmitted from the mobile terminal as a specific function based on the cooperation operation. The information processing device described in. The information processing device according to claim 1, wherein the inside of the predetermined space is inside the vehicle, and the outside of the predetermined space is outside the vehicle. The inside of the predetermined space is inside the vehicle, and the outside of the predetermined space is outside the vehicle.
The information processing device according to claim 5, wherein the predetermined place in the predetermined space is a driver's seat. It is a method of controlling the above-mentioned cooperative operation in a system in which an information processing device and a mobile terminal are connected and cooperated with each other.
The outing detection unit of the information processing device detects that the owner of the mobile terminal wirelessly connected in the predetermined space or the mobile terminal connected wirelessly or by wire has gone out of the predetermined space. Step 1 and
When the cooperation operation stop unit of the information processing device detects the outing of the mobile terminal or the owner by the outing detection unit, the cooperation with the mobile terminal is maintained while maintaining the connection state with the mobile terminal. A cooperative operation control method comprising a second step of stopping the provision of a specific function based on the operation. After the provision of the specific function is stopped by the cooperative operation stop unit, the return detection unit of the information processing device is used by the mobile terminal or the owner whose outing is detected by the outing detection unit in the predetermined space. The third step to detect that you have returned from the outside to the inside of
The cooperation operation start unit of the information processing device further has a fourth step of resuming the provision of the specific function when the return of the mobile terminal or the owner is detected by the return detection unit. The cooperative operation control method according to claim 16, which is characterized.

Images