JP6862629B2 - Vehicle management server, in-vehicle terminal, watching method, and program in the watching transfer system - Google Patents

Description

The present invention relates to a watching system for children, the elderly, and the like.

Conventionally, a watching system for watching over children, the elderly, etc. using a vehicle has been known. For example, Patent Document 1 discloses a notification system capable of performing watching over a person to be watched using on-demand traffic. According to this system, the transmission / reception unit of the operation management server receives boarding / alighting information consisting of the boarding / alighting date / time, the stop point, and the passenger name or the getting-off person name transmitted from the information terminal device mounted on the passenger vehicle. Then, the notification information generation unit acquires the notification destination associated with the received passenger name or disembarkation name from the notification destination storage unit, generates notification information to the acquired notification destination, and the transmission / reception unit generates the notification information. , Send the generated notification information.

Further, Patent Document 2 describes a method of watching over an object by a system mounted on an automobile, such as detecting a wireless device, determining the position of the wireless device, and recognizing the position of the wireless device with respect to the vehicle. A system is disclosed that includes analyzing the position of a wireless device with respect to a preset state declaration and activating an alarm in response to the satisfaction of the preset state declaration.

In addition to the monitoring function, a monitoring system with another function is also known. For example, Patent Document 3 includes a monitoring system that allows parents to confirm the existence of a child with an image including the face of their own child and to collect the image as a commemorative image such as a growth record of the child. It is disclosed. According to this system, the mobile terminal includes a wireless tag identifier determination unit, a face image acquisition unit, and a face image transmission unit, and the wireless tag identifier determination unit manages the wireless tag identifier included in the received radio wave. The face image acquisition unit determines whether or not it is the wireless tag identifier of the target wireless tag, and determines that the wireless tag identifier included in the received radio wave is the wireless tag identifier of the wireless tag to be managed. When the determination unit determines, the image including the face of the monitoring target person is acquired, and the face image transmission unit wirelessly manages the image including the face of the monitoring target person acquired by the face image acquisition unit. Send to the communication device associated with the tag identifier.

Japanese Unexamined Patent Publication No. 2016-21526 Special Table 2016-507792 Japanese Unexamined Patent Publication No. 2016-110471

By the way, as the government has presented in the public-private ITS concept / roadmap 2016, efforts toward the realization of "unmanned autonomous driving and moving services in limited areas" are accelerating. This is to provide a service with an image similar to local public transportation within a restricted area (limited area), and basically it is assumed that it is a means of transportation that connects the limited area to the local public transportation. However, the possibility of getting on and off from the front door of the house is also being considered. Universities and companies are already demonstrating autonomous driving technology on public roads, and it is expected that if this trend is followed, the service may begin to spread in 2020, as shown by the government roadmap. .. However, there are still many difficult issues for fully unmanned autonomous driving, including general roads, and it is expected that humans will continue to try to automate in limited areas.

The inventors are mainly aware of the following three social issues, and are aiming for the early launch of an unmanned autonomous driving mobile service that will be incorporated into the real world.
(1) A serious social issue of the arrival of an era of shortage of bus and taxi drivers
(2) Due to the decrease in users, the labor cost of drivers will be a burden, and the number of areas where public transportation cannot be maintained will increase at an accelerating rate.
(3) It is not possible to support the rapidly aging era without new means of transportation that are closely related to daily life. Especially for the social issues of (3), we will conduct empirical studies on the aging community. We are proceeding with specific activities.

In the demonstration study, as one of the specific issues of the unmanned autonomous driving service, a monitoring system for children, elderly people living alone, etc. using the transportation system in the limited area as described above was introduced. It came to the recognition that it was important. In particular, when the vehicle is autonomous driving, no matter how advanced the autonomous driving technology is, it is uneasy to let only children and the elderly ride (this is the same even if it is manned driving). In addition, the guardians such as parents and family members are watching not only the place and time when the target person got on and off the vehicle, but also the surrounding environment at the time of getting on and off, the state at the time of getting on and off, the state during boarding (complexion, conversation with the surroundings). I would like to know the situation inside and outside the car, that is, the situation before getting on and after getting off. This is because there is danger before getting in the vehicle or after getting off the vehicle.

However, although the system described in Patent Document 1 described above is a monitoring system using a vehicle, it only notifies the date and time of getting on and off, the place of stop, and the name of the person getting on and off, and also notifies the surrounding environment at the time of getting on and off and the state during boarding. It's not something to do. Further, the system described in Patent Document 2 observes the position of a target (personal property in the vehicle, children, pets, etc.) by a system mounted on the vehicle and monitors the surrounding environment of the target. It is not intended for self-driving vehicles. In addition, the system described in Patent Document 3 is really for one's own child only by identifying by a school bag, a hat, and a nickname indicating the characteristics of the child when the parent is notified that the child is in a predetermined position. I am not sure if there is any, and I may feel uneasy, so I try to confirm the child using a face image, but it has nothing to do with the vehicle.

Therefore, in view of the above problems, in the present invention, the autonomous driving vehicle (including the automatic driving with a human being on board) is not only the boarding / alighting place and the boarding / alighting time of the watched person, but also the surrounding environment before and after boarding / alighting. The purpose is to provide a monitoring system that can widely monitor the inside and outside of the vehicle, such as when getting on and off and while riding.

In order to solve the above problems, the present invention provides the following solutions.

(1) A vehicle management server in a watching transfer system in which a vehicle watches over a person to be watched at the time of pick-up, and an image of the surroundings when the person to be watched gets on and off from the vehicle, an image of the person to be watched, and the said A vehicle image receiving means for receiving an image captured by an in-vehicle camera mounted on the vehicle, a recording means for recording the received image, and the recording means for the person to be watched. It is characterized by including a notification information extracting means for extracting information to be notified to a watcher according to a set condition, and a watcher terminal transmitting means for transmitting the extracted notification information to the watcher's terminal.

(2) In the configuration of (1) above, the vehicle is characterized by being an unmanned autonomous driving vehicle or a semi-autonomous driving vehicle with a human being on board.

(3) In the configuration of (1) or (2) above, the vehicle-mounted camera is provided with a face image authentication means that receives a face image of a passenger's face from the vehicle and performs face image authentication. ..

(4) In any of the configurations (1) to (3) above, the watcher message receiving means for receiving the message from the watcher and the watching target person are on board or are scheduled to board the message. It is characterized by including an in-vehicle terminal transmitting means for transmitting to the in-vehicle terminal of the vehicle.

(5) An in-vehicle terminal in a watching transfer system in which a vehicle watches over a person to be watched during transportation, and an image of the surroundings, an image of the person to be watched, and a target to be watched when the person to be watched gets on and off the vehicle. Notification of extracting information to be notified to the watcher of the watch target person from the recording means and a recording means for recording the video captured by the in-vehicle camera mounted on the vehicle according to the set conditions. The information extraction means and the watcher terminal transmission means for transmitting the extracted notification information to the watcher's terminal are provided.

(6) In the configuration of (5) above, the vehicle-mounted communication device for having a conversation with a passenger in the vehicle is provided.

(7) In the configuration of (5) or (6) above, a face image authentication means that images the face of a passenger in the vehicle and authenticates the face image, and the passenger when the face image is not registered. It is characterized in that it includes a user registration means for performing user registration.

(8) This is a method of watching over the person to be watched when the vehicle is picked up, and when the person to be watched gets on and off the vehicle, the surrounding image, the image of the person to be watched, and the inside of the vehicle of the person to be watched. The stage of recording the image captured by the in-vehicle camera mounted on the vehicle, the stage of extracting the information to be notified to the observer of the person to be watched from the recorded image according to the set conditions, and the above. It is characterized by including a step of transmitting the extracted notification information to the terminal of the watcher.

(9) A program characterized in that a computer executes each step of the monitoring method described in (8) above.

According to the present invention, an autonomous driving vehicle (including automatic driving with a human being on board) has not only the boarding / alighting place and boarding / alighting time of the watched person, but also the surrounding environment before and after boarding / alighting, and the state during boarding / alighting. , It is possible to provide a monitoring system that can widely monitor the situation inside and outside the vehicle.

It is a figure which shows the image of the watching transfer service which concerns on embodiment of this invention. It is a figure which shows the structure of the vehicle which performs the watching and pick-up service which concerns on embodiment of this invention. It is a figure which shows the functional structure of the vehicle and the vehicle management server in the monitoring transfer system which concerns on 1st Embodiment of this invention. It is a figure which shows the functional structure of an in-vehicle communication device. It is a figure which shows the state of the conversation in the car recorded in the communication content DB. It is a figure which shows the example of the notification screen for a watcher. It is a figure which showed the screen example at the time of user registration by face image authentication. It is a figure which shows the functional structure of the vehicle and the vehicle management server in the monitoring transfer system which concerns on 2nd Embodiment of this invention.

Hereinafter, embodiments for carrying out the present invention (hereinafter, embodiments) will be described in detail with reference to the accompanying drawings. In the following figures, the same elements are numbered or coded throughout the description of the embodiments. Further, in the functional configuration diagram, the arrows between the functional blocks indicate the data flow direction or the processing flow direction.

(Image of watching and pick-up service)
FIG. 1 is a diagram showing an image of a watching and pick-up service according to an embodiment of the present invention. The watching and pick-up service (hereinafter referred to as this service) is a pick-up service that uses an autonomous vehicle or a semi-autonomous vehicle (automatic driving with a human being) of a community-based transportation system. It is a service that monitors the inside and outside of the vehicle of the person to be watched (watcher) such as the elderly, and notifies the watcher if necessary. On the contrary, the service also provides a service in which the watcher sends a message to the vehicle and conveys the message to the watcher when the watcher gets on and off the vehicle.

As shown in the figure, the watcher terminal 20 (20a, 20b) which is a terminal possessed by the watcher (X, Y) of the vehicle 10 and the watch target person (children A, B, C, elderly D in this example). ) Is connected to the vehicle management server 100 by a wireless communication network. Child A is already in the vehicle 10, and children B and C are about to get in. In addition, the elderly person D is on the way to the vehicle 10 for boarding.

The vehicle 10 is provided with a plurality of in-vehicle cameras 12 and various sensors for automatic driving, and can capture images (including audio) inside and outside the vehicle. In this service, these in-vehicle cameras 12 and the like are used to capture and record images of children, the elderly, etc. before and after getting on and off, when getting on and off, and during getting on, and send them to the watcher when necessary. be able to. The images before and after getting on and off are images taken from a camera mounted on the vehicle body, an external camera 30 such as a security camera, or another vehicle 10'. The external camera 30 and the other vehicle 10'can transmit the image taken by themselves to the nearby vehicle 10 by short-range wireless communication. Further, the image of the passengers in the vehicle 10 is captured by a plurality of in-vehicle cameras 12 in the vehicle. The image of the vehicle-mounted camera 12 is transmitted and recorded to the vehicle management server 100 at regular time intervals or at a predetermined operation. The recorded video can be viewed from the watcher terminal 20.

The vehicle management server 100 transmits the video transmitted and stored from the vehicle 10 to the watcher terminal 20 according to the conditions set in advance by the watcher. For example, the setting conditions are limited to images showing the person being watched over by oneself, and even if a specific place (for example, school, cram school, hospital, geriatric center, etc.) or time zone is specified. Good. However, from the viewpoint of personal privacy, the captured image may be blurred or the image of a person other than the person to be watched by the watcher may be obscured or may not be viewed.

FIG. 2 is a diagram showing a configuration of a vehicle 10 used in this service. The vehicle 10 is provided with a plurality of in-vehicle cameras, and the in-vehicle cameras are mounted on the front, rear, left, and right sides of the body of the vehicle 10 and mainly image an object outside the vehicle (in the example of the figure, ■ There are 12a, 12b, 12c, 12d) represented by, and in-vehicle cameras (12e, 12f, 12g represented by ● in the example of the figure) installed in the vehicle. The external camera captures and records surrounding objects and people when the vehicle is running and stopped. The external camera also functions as a sensor that detects objects and humans outside the vehicle. In addition, the in-vehicle camera captures and records the state of passengers in the vehicle and the surrounding conditions as seen from the inside of the vehicle. Infrared sensors, ultrasonic sensors, and the like are provided at the four corners of the vehicle body (not shown).

Further, the vehicle 10 is equipped with an in-vehicle terminal 11. The in-vehicle terminal 11 is a control device that controls the in-vehicle camera 12 of the vehicle 10, collects various information, and controls communication with the outside. The in-vehicle terminal 11 may be incorporated in the car navigation system.

Further, the vehicle 10 may be provided with the in-vehicle communication device 13. The in-vehicle communication device 13 is a device that controls communication in the vehicle, such as a conversation between passengers, a conversation between a vehicle and a passenger, and a remote conversation between a passenger and a watcher when the passenger is a watched person. Further, the in-vehicle communication device 13 is provided with speakers (directional speaker 136 and general speaker 137, which will be described later), and controls the utterance to all passengers and the utterance for a specific individual. The in-vehicle terminal 11 and the in-vehicle communication device 13 will be described in detail later.

(Functional configuration of the watching and pick-up system)
FIG. 3 is a diagram showing the functional configurations of the vehicle 10 and the vehicle management server 100 in the monitoring transfer system (hereinafter referred to as the present system) according to the first embodiment of the present invention.

In this system, a plurality of vehicles 10 constituting a community-based transportation system, a plurality of watcher terminals 20, and a vehicle management server 100 are mainly connected by wireless communication via a network 40.

(vehicle)
In addition to the above-mentioned in-vehicle terminal 11, a plurality of in-vehicle cameras 12, and an in-vehicle communication device 13, the vehicle 10 includes a user ID acquisition means 14, a short-range wireless communication means 15, and a watching status display means 16, which will be described later. ..

As described above, the in-vehicle terminal 11 is a control device that manages and controls various information of the vehicle, can communicate with the vehicle management server 100 (hereinafter, simply a server), and includes various images captured inside and outside the vehicle. Send information to the server. It also receives various information from the server.

As described with reference to FIG. 2, the in-vehicle camera 12 is a general term for cameras mounted on the vehicle 10, and is mounted on the front, rear, left and right of the body of the vehicle 10, the ceiling of the driver's seat, the ceiling of the rear seat, and the like.

The in-vehicle communication device 13 (installed at a position behind the rear seat in the vehicle in FIG. 2) is a device that enables the passengers of the vehicle 10 and the vehicle to communicate with each other as described in FIG. .. When the in-vehicle communication device 13 detects a passenger, the vehicle-mounted communication device 13 is provided with artificial intelligence capable of speaking to the passenger, responding to a voice from the passenger, and having a conversation. Further, as will be described in detail later, for a plurality of passengers, each position is detected, the directional speaker 136 speaks to each passenger, and the passenger responds to the utterance. And you can talk to the passengers individually. Of course, it is also possible to speak to all passengers with the omnidirectional general speaker 137.

The user ID acquisition means 14 (user identification means) is a means for acquiring the user ID of a passenger registered in this system, and is, for example, a membership card, a transportation IC card, or a user terminal (not shown) at the time of boarding. It is a reader that reads and acquires a user ID. This reader may be installed at the entrance, or may be installed at each seat in a large vehicle such as a bus. Of course, when getting off, the boarding fee is automatically paid from the billing destination of the user ID. Even an unregistered user can get on the vehicle 10 of this system. In that case, the in-vehicle camera 12 is used to capture a passenger's face image, and if the captured face image is unregistered, the passenger's face image is captured. You can also register as a user on the spot. The user registration using this face image will be described in detail later.

The short-range wireless communication means 15 can communicate with the nearby external camera 30 and another vehicle 10', receive the images taken by them, and transmit them to the server as if they were taken by the vehicle 10. it can. That is, the external camera 30 and the in-vehicle camera of the other vehicle 10'may also be considered as a part of the in-vehicle camera 12.

The watching status display means 16 displays who is currently watching so that it can be recognized from outside the vehicle. A safer watching system can be provided by allowing a person outside the vehicle to grasp the watching status of the vehicle. Instead of watching individually, for example, when children are returning home from elementary school all at once, by displaying "Child watching" on the display outside the car, parents and others watching by their side can feel at ease. ..

(Vehicle management server)
The vehicle management server 100 includes a vehicle image receiving means 101, a face image authentication means 102, a user registration means 103, a watcher message receiving means 104, an in-vehicle terminal transmitting means 105, a notification information extracting means 106, a watching person terminal transmitting means 107, and a vehicle. It includes a video recording DB 110 and a user registration DB 120.

The vehicle image receiving means 101 periodically receives various vehicle images captured by the vehicle-mounted camera 12 from the vehicle 10 and records them in the vehicle image recording DB 110, which is a vehicle image recording means, for a certain period of time. The vehicle image recorded here includes an image (including audio) and additional information of the image (recording start / end time of the image, recording time, passenger information at that time, etc.). The data recorded in the vehicle video recording DB 110 is deleted after a certain period of time.

In the face image authentication means 102, the vehicle-mounted camera 12 takes an image of the passenger's face, receives the face image from the vehicle 10, and authenticates the passenger's face image. If the face image is not registered in the user registration DB 120, the in-vehicle terminal 11 is notified to that effect, and the in-vehicle terminal 11 uses the in-vehicle communication device 13 to call out to the passenger and whether or not to newly register the user. Ask. If the passenger wishes to newly register, the user registration means 103 transmits the necessary data for registration to the vehicle 10 and registers the data in the user registration DB 120. All this registration can also be done by voice. If the passenger is a child or elderly person to be watched over, ask for the contact information of the watcher such as a parent or family member, notify the contact information, and ask for confirmation of the watcher. New user registration may be performed during boarding or after disembarking. By doing so, even a child or an elderly person who is likely to be a person to be watched over and does not have a user terminal can register as a user in the car.

Further, the face image authentication means 102 may be set to accept registration of reservation information including a face image, and in some cases, to refuse boarding if this registration is not performed. By doing so, it is possible to prevent suspicious boys and adults from riding in children's cars.

The watcher message receiving means 104 receives the message from the watcher and instructs the vehicle-mounted terminal transmitting means 105 to transmit the message to the vehicle-mounted terminal 11. If the person to be watched over by the destination has not yet boarded the vehicle, the in-vehicle terminal 11 stores the message for a predetermined time. If the person to be watched has already disembarked, a message to that effect is transmitted to the watcher terminal 20.

The in-vehicle terminal transmitting means 105 transmits a message from the watcher terminal 20 to the in-vehicle terminal 11, and also transmits information from the server side to the in-vehicle terminal 11 as needed.

The notification information extracting means 106 extracts the notification information to be notified to the watcher from the information recorded in the vehicle video recording DB 110 according to the preset notification conditions, and extracts the notification destination of the watcher from the user registration DB 120. Then, the notification information is instructed to the watcher terminal transmitting means 107 to be transmitted to the watcher terminal 20. The notification conditions include, for example, the notification interval, the necessity of the video, and if the video is required, the timing and the required video (for example, the video for only one minute before and after getting on and off is required, and the video inside the car is not required, etc. ) Etc. are included. Further, if privacy protection is required, the notification information extracting means 106 also performs a processing process of adding a blur or the like to the face of a person other than the person to be watched over the extracted video. When the video is unnecessary, only the text information including the boarding place, the boarding time, the getting-off place, and the getting-off time is notified. However, when the vehicle 10 detects an accident or other abnormality, the image may be notified regardless of the notification conditions. In addition, the notification information includes the ID of the watcher, the ID of the watcher, the email address or telephone number of the watcher, the boarding time of the watcher, the getting-off time, and the images and conversations inside and outside the car including before and after the ride. Records are included.

(In-vehicle communication device)
FIG. 4 is a diagram showing a functional configuration of the in-vehicle communication device 13. The in-vehicle communication device 13 is a device for mainly performing voice communication between the vehicle and the passenger. It is equipped with conversational means, speakers (including earphones), microphones, etc. for that purpose. A display device for image communication may be provided. Bone conduction earphones may be provided for the hearing impaired.

As shown in FIG. 4, the in-vehicle communication device 13 includes a detection means 131, a communication content determination means 132, a personal information transmission means 133, a general information transmission means 114, a communication content analysis means 135, and a directional speaker. It includes 136, a general speaker 137, a directional microphone 138, and a communication content DB 130. The in-vehicle communication device 13 may communicate with the passenger by communicating with the user terminal possessed by the passenger.

The detection means 131 detects the position of a passenger by means for detecting a person who has boarded the vehicle and a determination means for determining the position of the passenger based on the detected information. At this time, the detection means 131 may detect the position of the passenger from the image captured by the in-vehicle camera 12. When the user ID acquisition means 14 is an ID reader connected to each seat of the vehicle, the seat associated with the ID reader that has read the user ID may be regarded as the passenger's position. Good.

The communication content determining means 132 records the conversation in the vehicle, refers to the communication content DB 130, and determines the information of the communication content to be transmitted to the passenger. Further, the communication content determining means 132 may receive and store a message from the watcher terminal 20 to the watcher and notify the message when the watcher gets on board.

In addition, the communication content determination means 132 is used when the time of getting on and off or the place of getting on and off is different from the usual one (for example, since the swimming school ends at 17:00 every Monday, the person who gets on at 17:30 gets on at a different time than usual. Notification information may be sent when the passenger gets on and off at a time or place different from the schedule, even though the boarding / alighting date / time and the boarding / alighting place are scheduled in advance.

The in-vehicle communication device 13 may include personal information transmission means 133 for passengers who do not want others to hear personal information in the vehicle. The personal information transmitting means 133 transmits personal information, which is a word including personal content determined by the communication content determining means 132, only to the passengers in the direction detected by the detecting means 131. For example, a message from a watcher may correspond to personal information. Specifically, the personal information transmitting means 133 is determined by the communication content determining means 132 by the directional speaker 136 that emits a sound only in a predetermined range (a range including only passengers in the direction detected by the detecting means 131). Pronounce the words that your personal information indicates.

In addition, the personal information transmitting means 133 can also transmit personal information for each passenger when there are a plurality of passengers. The personal information transmitting means 133 may transmit personal information only to the user terminal possessed by the passenger in the direction detected by the detecting means 131. In this case, the user terminal may pronounce, for example, a word indicated by personal information received from the personal information transmitting means 133 on a bone conduction earphone or the like connected to the user terminal. By doing so, other passengers in the vehicle will not be asked for personal information.

The general information transmitting means 134 transmits conversation contents that may be heard by other passengers, and transmits general information, which is a word other than personal information, determined by the communication content determining means 132 to all passengers. .. Specifically, the general information transmitting means 134 pronounces the words indicated by the general information determined by the communication content determining means 132 by the general speaker 137 that emits a sound in a range wider than a predetermined range (a range in which all passengers can hear). The general information transmitting means 134 may transmit general information to the user terminals of all passengers. In this case, the user terminal may use a speaker or the like to pronounce the words indicated by the general information received from the general information transmitting means 134.

The communication content analysis means 135 uses the directional microphone 138 to transmit the user's voice according to the personal information transmitted by the personal information transmitting means 133 and the user's voice according to the general information transmitted by the general information transmitting means 134. The sound is collected, the content of the voice of the user who has collected the sound and the presence / absence of the voice are analyzed, and the communication content DB 130 is updated according to the analysis result.

The directional speaker 136 emits a sound only in a predetermined range, and is provided for each seat of the vehicle, and emits a sound that can be heard only by the passengers sitting in each seat.

The general speaker 137 produces a sound in a range wider than a predetermined range produced by the directional speaker 136, and produces a sound that can be heard by all passengers.

The directional microphone 138 collects sounds generated from a predetermined range, and is provided for each seat of the vehicle, and collects sounds emitted by passengers sitting in each seat.

The above-mentioned functional configuration of this system is merely an example, and one functional block (database and functional processing unit) may be divided, or a plurality of functional blocks may be collectively configured as one functional block. In each function processing unit, the CPU (Central Processing Unit) built into the device reads computer programs stored in storage devices such as ROM (Read Only Memory), flash memory, SSD (Solid State Drive), and hard disk. , Realized by a computer program executed by the CPU. That is, each function processing unit reads and writes necessary data such as a table from a database (DB; Data Base) stored in a storage device or a storage area in a memory by this computer program, and is related in some cases. It is realized by controlling the hardware (for example, input / output device, display device, communication interface device). Further, the database (DB) in the embodiment of the present invention may be a commercial database, but it also means a mere collection of tables and files, and the internal structure of the database itself does not matter.

(Communication content in the car)
FIG. 5 is a diagram showing a state of passenger conversation in the vehicle recorded by the in-vehicle communication device 13. Although detailed explanation is omitted, in this example, only child A is on board at the beginning, child B and child C who are leaving school on the way are on board, and then elderly person D is on board. Until the person gets off, the content of the conversation in the vehicle is recorded in the communication content DB 130. Vehicles also participate in the conversation with an automatic response. The conversation also includes the message of the watcher. In the conversation content, the elderly D receives a message from his family, the elderly D listens to the message in the car, the child B receives a message from his mother, and the child B and the elderly D talk. It is recorded that child B gets off. Of course, not only the conversation content but also the video captured by the vehicle-mounted camera 12 at that time is recorded. Therefore, for example, the mother of the child B can confirm that the child has arrived at the front door of the house without detouring after getting out of the car.

Since such conversation contents and video records in the vehicle can be viewed from the watcher terminal 20, not only the boarding / alighting place and boarding / alighting time of the watched person, but also the surrounding environment before and after boarding / alighting, and during boarding / alighting. It is possible to widely monitor the situation inside and outside the car, such as the situation of the car. In addition, even if the child or the elderly does not have a means of communication such as a smartphone or mobile phone, the message from the watcher can be heard in the car. Of course, you can also respond to the message on the fly. Since the message is recorded on the vehicle side, the contents can be confirmed at any time by both the watcher and the watch target person. That is, the vehicle serves as a means of communication between the watcher and the watcher.

(Notification screen for watchers)
FIG. 6 is a diagram showing an example of a notification screen for the watcher. As already described, the vehicle can record a series of situations before, during, during, during, and after the disembarkation of the person to be watched. For example, FIG. 6 (a) shows a state when child B is on board, FIG. 6 (b) is a state while children A, B, and C are on board, and FIG. 6 (c) is a state in which child B gets off. It is certainly displayed that he has returned to his home entrance. The watcher can select the scene to be viewed from the thumbnail images 2001 to 203 at the bottom of the screen and check the state of the child with a moving image or a still image. A message can also be sent to the vehicle from this screen if the child knows which vehicle he or she is riding or will be riding. Further, it is possible to specify the timing of playing the message in the vehicle, such as when there are no other passengers, and the content that the other passengers do not want to hear.

(User registration screen)
FIG. 7 is a diagram showing an example of a screen when a passenger newly registers a user by face image authentication. Since the vehicle is equipped with an in-vehicle camera 12, it is possible to take an image of a passenger's face using these cameras, send the image to a server, and register the user in the vehicle. As the face image, those taken by a plurality of in-vehicle cameras 12 may be used. This will improve the accuracy of face recognition.

The server determines whether or not the face image sent from the vehicle is registered in the user registration DB 120, and if it is not registered, recommends user registration.

The screen of FIG. 7 shows the state of user registration at this time on an interactive screen. If the passenger has a user terminal, it may be displayed on the screen, and if the passenger does not have the user terminal, it may be displayed on the display device of the in-vehicle terminal 11. Here, all input from passengers can be made by voice, and the utterance may be ordinary spoken language.

User registration requires information such as the address and name of the person to be watched over, the date of birth, the address and name of the watcher such as a family member, the date of birth, a telephone number, an email address, etc. Many elderly people are not good at operating terminals or cannot operate the terminal at all, so if you can find out the name and phone number of a watcher such as a family member, the server will contact the watcher and give you a face image. If you can confirm that you are who you say you are, you may ask the watcher to enter the necessary information.

At this time, if another passenger is in the vehicle, it is detected, and the in-vehicle communication device 13 detects it, and in order to protect privacy, an earphone or a directional speaker 136, a directional microphone 138. You can also use such as to prevent others from asking for your personal information at the time of registration.

(Second Embodiment)
FIG. 8 is a diagram showing a functional configuration of a watching transfer system according to a second embodiment of the present invention. In the first embodiment, the main functions are provided on the server side, but in the second embodiment, the main functions are provided on the vehicle side. Specifically, the in-vehicle terminal 11A of the second embodiment includes a face image authentication means 102, a vehicle image recording DB 110, a notification information extraction means 106, a watcher terminal transmission means 107, and a control unit 108. The in-vehicle communication device 13, the user ID acquisition means 14, and the short-range wireless communication means 15 may be considered to be included in the in-vehicle terminal 11A. Since the functions of the above means are the same as those of the functional blocks of FIG. 3, the description thereof will be omitted.

Further, in the vehicle management server 100A of the second embodiment, the user registration means 103 and the user registration DB 120 for registering the user in response to the face authentication request from the in-vehicle terminal 11A remain. Further, on the vehicle management server 100A, after confirming the watcher message receiving means 104 for receiving the message from the watcher and the received message from the watcher, the ID and name of the watcher and the watcher are confirmed, and then the vehicle is mounted on the vehicle. The vehicle-mounted terminal transmitting means 105 for transmitting to the terminal 11A also remains.

As described above, in the second embodiment, since the main functions are provided on the vehicle side, the load on the server side is significantly reduced as compared with the first embodiment.

(Effect of embodiment)
According to this system, using a community-based transportation vehicle, not only the boarding / alighting place and boarding / alighting time of the watched person, but also the surrounding environment before and after boarding / alighting, the situation during boarding / alighting, and the situation inside and outside the car. It is possible to provide a monitoring system that can widely monitor. Further, the vehicle may be an autonomous driving vehicle (including semi-automatic driving with a human being on board). In addition, since face recognition can be performed using an in-vehicle camera, passengers can be identified at the time of boarding, and new user registration can be performed in the car. At this time, when another person is on board, it is possible to control so that the personal information at the time of registration is not heard. In addition, since the vehicle can receive a message from the watcher to the watcher, even if the passenger does not have a user terminal such as a smartphone, he / she can listen to the message in the car and send the reply by voice. can do. In addition, since the vehicle is equipped with a communication device, it is possible to notify the terminal of the watcher at the time of getting on and off or during the conversation or video.

In the above embodiment, the vehicle is assumed to be an unmanned autonomous vehicle or a semi-autonomous vehicle, but this system can be applied to a vehicle that is not automatically driven. Even in that case, since the in-vehicle terminal controls all the control of the in-vehicle camera and the like, the communication with the outside of the vehicle, and the communication with the passengers, the driver is hardly burdened with the person to be watched over.

Although the present invention has been described above using the embodiments, it goes without saying that the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. Further, it is clear from the description of the scope of claims that the form to which such a modification or improvement is added may be included in the technical scope of the present invention.

In the above embodiment, the present invention has been mainly described as a monitoring vehicle management server and an in-vehicle terminal in a monitoring transfer system, but the present invention is an invention of a method (watching method) or a monitoring method, respectively. It can also be regarded as the invention of a program that causes a computer to execute this step.

A, B, C, D Watchers (watchers)
X, Y Watcher 10 Vehicle 10'Other vehicle 11 In-vehicle terminal (first embodiment)
11A in-vehicle terminal (second embodiment)
12 In-vehicle camera 12a, 12b, 12c, 12d External camera 12e, 12f, 12g In-vehicle camera 13 In-vehicle communication device 14 User ID acquisition means (user identification means)
15 Short-range wireless communication means 16 Watching Sumikyo display means 20 (20a, 20b) Watching person terminal 30 External camera 40 Network 100 Vehicle management server (first embodiment)
100A vehicle management server (second embodiment)
101 Vehicle image receiving means 102 Face image authentication means 103 User registration means 104 Watcher message receiving means 105 In-vehicle terminal transmitting means 106 Notification information extracting means 107 Watching person terminal transmitting means 108 Control unit 110 Vehicle image recording DB
120 User registration DB
130 Communication content DB
131 Detection means 132 Communication content determination means 133 Personal information transmission means 134 General information transmission means 135 Communication content analysis means 136 Directional speaker 137 General speaker 138 Directional microphone 201, 202, 203 Thumbnail image

Claims (9)

It is a vehicle management server in the watching and picking system where the vehicle watches over the target person at the time of picking up.
From the vehicle, an image of the surroundings, an image of the person being watched, and an image of the inside of the vehicle of the person being watched when the person being watched gets on and off is received by an in-vehicle camera mounted on the vehicle. Vehicle image receiving means and
A recording means for recording the received video and
A notification information extraction means for extracting from the video recorded in the recording means the video to be notified to the watcher of the watching target and the information accompanying the video according to the set conditions.
A watcher terminal transmitting means for transmitting the extracted notification information to the watcher's terminal,
A watcher message receiving means for receiving a message from the watcher,
An in-vehicle terminal transmitting means for transmitting the message to the in-vehicle terminal of the vehicle on which the watching target person is riding or is planning to board.
A vehicle management server characterized by being equipped with. The vehicle management server according to claim 1, wherein the vehicle is an unmanned autonomous vehicle or a semi-autonomous vehicle on which a human is riding. The vehicle management server according to claim 1 or 2, wherein the vehicle-mounted camera includes a face image authentication means that receives a face image of a passenger's face from the vehicle and performs face image authentication. Any one of claims 1 to 3, wherein an image of the watching target person returning to the home entrance, which is captured by the vehicle-mounted camera, is received from the vehicle and transmitted to the terminal of the watching target person. The vehicle management server described in the section. It is an in-vehicle terminal in the watching transfer system where the vehicle watches over the target person at the time of transportation.
A record of recording an image of the surroundings, an image of the person being watched, and an image of the inside of the person being watched when the person being watched gets on and off the vehicle, captured by an in-vehicle camera mounted on the vehicle. Means and
A notification information extraction means for extracting from the video recorded in the recording means the video to be notified to the watcher of the watching target and the information accompanying the video according to the set conditions.
A watcher terminal transmitting means for transmitting the extracted notification information to the watcher's terminal,
A watcher message receiving means for receiving a message from the watcher,
An in-vehicle terminal characterized by being equipped with. The vehicle-mounted terminal according to claim 5, further comprising an vehicle-mounted communication device for having a conversation with a passenger in the vehicle. A face image authentication means that captures the face of a passenger in the vehicle and authenticates the face image.
A user registration means for registering a passenger as a user when the face image is not registered, and
The in-vehicle terminal according to claim 5 or 6, wherein the in-vehicle terminal is provided. It is a watching method that watches over the target person when the vehicle is picked up.
An image of the surroundings when the person to be watched gets on and off the vehicle, an image of the person to be watched,
And the stage of recording the image of the inside of the vehicle of the person to be watched by the in-vehicle camera mounted on the vehicle.
From the recorded video, the stage of extracting the information to be notified to the watcher of the watch target person according to the set conditions, and
At the stage of transmitting the extracted notification information to the watcher's terminal, and
At the stage of receiving the message from the watcher,
The stage of transmitting the message to the in-vehicle terminal of the vehicle being or will be boarded by the watching target person, and
A watching method characterized by a computer performing. A program comprising causing a computer to execute each step of the monitoring method according to claim 8.

Images