JP7339636B1 - Left-in-vehicle detection system and server - Google Patents

Abstract

An object of the present invention is to detect an abandoned state in which a user who has boarded a vehicle has not gotten off the vehicle. [Solution] A face authentication image taken by a predetermined photographing means associated with the vehicle at the timing of getting into the vehicle is received from a vehicle terminal, and the face authentication is executed by the face authentication means. A first face recognition processing unit that identifies the user corresponding to the image and creates a list of passengers who have boarded the vehicle, and a face recognition unit that identifies the user corresponding to the image and creates a list of passengers who have boarded the vehicle, and face recognition that is photographed by a predetermined photographing means associated with the vehicle when the vehicle arrives at its destination and exits the vehicle. a second face authentication processing unit that receives an image from a vehicle terminal and executes face authentication by a face authentication means to identify a user corresponding to the face authentication image and create an alighted list, and a predetermined condition. When the boarding list is satisfied, the boarded list and the disembarked list are compared, and if there is a discrepancy in the comparison results, it is determined that there is a possibility of an abandoned state, and the notification target terminal that has been registered in advance is notified of the abandoned state. The server is equipped with a notification processing unit that notifies the possibility of occurrence. [Selection diagram] Figure 1

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a left-in-vehicle detection system and a server for detecting a left-in-vehicle state in which a user who gets into a vehicle does not get out of the vehicle.

2. Description of the Related Art Conventionally, there are apparatuses and systems for managing entrance/exit to facilities and attendance management. By entrusting entrance/exit management and attendance management to the system, it is possible to eliminate the element of human error and realize reliable management.

For example, in Patent Document 1, a personal authentication device that records a unique ID possessed by each student is read by a dedicated reader to automatically collect entry/exit information of students to and from schools and cram schools. An entry/exit information management system is disclosed which is provided with a configuration for physically acquiring information and notifying it to a parent's portable terminal. Further, Patent Document 2 discloses a configuration in which a terminal having a face authentication function is used for attendance management of children in kindergartens and elementary schools.

JP 2005-352621 A Utility Model Registration No. 3179821

By the way, in recent years, there have been tragic incidents in which kindergarten and kindergarten children are left unattended on the shuttle bus and die. Since the children have insufficient knowledge and strength, it is difficult for them to escape from the vehicle on their own once they are trapped. .

If management based on Patent Document 1 and Patent Document 2 is performed, it is possible to manage attendance at nursery school and kindergarten facilities. There is a problem that the system cannot detect that it is left unattended.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a vehicle left-in-vehicle detection system and a server for detecting a left-in-vehicle state in which a user has not gotten out of the vehicle.

A left-in-vehicle detection system according to the present invention includes a server and a vehicle terminal installed in a vehicle, and is a left-in-vehicle detection system for detecting a left-in-vehicle state in which a user who has boarded the vehicle has not exited the vehicle, The vehicle terminal includes a vehicle-side transmission unit that transmits to the server an image for face authentication of the user captured at a predetermined timing based on a predetermined image capturing unit associated with the vehicle; a vehicle-side receiving unit for receiving information, wherein the server specifies a registration image including a face obtained by photographing a user who is scheduled to use the vehicle by a predetermined photographing means, and identifies the user. face authentication pre-registration for performing face authentication pre-registration of the user by acquiring user specifying information for performing the registration and performing learning for a predetermined face authentication means using the registration image and the user specifying information and executing face recognition by the face recognition means on the image for face recognition photographed at the timing of boarding the vehicle, thereby specifying a user corresponding to the image for face recognition, and creating a boarding list. and a first face authentication processing unit that creates a face authentication image by performing face authentication by the face authentication means on the face authentication image taken when the vehicle gets off after arriving at the destination A second face authentication processing unit that identifies the user corresponding to the above and creates a list of people who have exited the vehicle, and when a predetermined condition is satisfied, compares the list of people who have boarded the vehicle and the list of people who have exited the vehicle, and a deviation occurs in the comparison result. and a notification processing unit that determines that there is a possibility of occurrence of the idle state, and notifies a previously registered notification target terminal of the possibility of occurrence of the idle state.

A left-in-vehicle detection system according to the present invention includes a server, a vehicle terminal installed in a vehicle, and a facility terminal installed in a facility that is a destination of the vehicle, and a user who has gotten into the vehicle does not get off the vehicle. A left-in-vehicle detection system for detecting a left-in-vehicle state, wherein the vehicle terminal captures a user's face authentication image at a predetermined timing based on a predetermined image capturing means associated with the vehicle. The facility terminal includes a vehicle-side transmission unit that transmits to a server, and a vehicle-side reception unit that receives various information from the server. A facility-side transmission unit that transmits a user's face authentication image to the server, and a facility-side reception unit that receives various information from the server, and the server is scheduled to use the vehicle by a predetermined photographing means. acquiring a registration image showing a face obtained by photographing a user and user identification information for identifying the user, and performing predetermined face authentication using the registration image and the user identification information a face authentication pre-registration unit that performs face authentication pre-registration of the user by learning the means; a first face authentication processing unit that specifies a user corresponding to the face authentication image by performing authentication and creates a boarding list; a second face authentication processing unit that identifies a user corresponding to the image for face authentication by performing face authentication by face authentication means and creates a list of people who have exited the vehicle; The list of people who have exited the vehicle is collated, and if there is a discrepancy in the collation result, it is determined that there is a possibility that the state of being left unattended, and the possibility of the state of being left unattended is notified to a pre-registered notification target terminal. and a notification processing unit that performs the notification.

Further, in the system for detecting being left in a vehicle according to the present invention, the terminal to be notified includes a terminal registered in advance in association with each user, and the notification processing unit detects the user who may be left in a state of being left alone. The possibility of occurrence of the idle state is notified to the notification target terminal registered in association with the user who performed the notification.

Further, in the left-in-vehicle detection system according to the present invention, further, when the notification processing unit executes face authentication in the first face authentication processing unit and/or the second face authentication processing unit to specify the user. 2, it notifies that the face authentication has been performed to the notification target terminal registered in association with the user.

In the left-in-vehicle detection system according to the present invention, at least one or more human sensors are installed inside the vehicle, and the human sensors trigger detection when a predetermined detection start condition is satisfied. The detection is terminated when a predetermined detection termination condition is satisfied as a trigger, and when some motion is detected, the motion detection is notified to the server via the vehicle terminal or directly, and the notification is performed. The processing unit is characterized in that, upon receiving a motion detection notification from the motion sensor, the processing unit notifies a pre-registered notification target terminal of the motion detection in the vehicle.

A server according to the present invention includes a server and a vehicle terminal installed in a vehicle, and is a server in a vehicle-inside-abandoned detection system for detecting a vehicle-idle state in which a user who has boarded the vehicle has not exited the vehicle. acquires a registration image in which the face of the user who plans to use the vehicle is photographed by the photographing means of and user identification information for identifying the user, and acquires the registration image and the user a face authentication pre-registration unit that executes pre-registration of the user's face authentication by learning a predetermined face authentication means using specific information; receiving from the vehicle terminal the image for face authentication photographed at the timing of boarding the vehicle, and executing face authentication by the face authentication means to identify the user corresponding to the image for face authentication, and list the user who has boarded the vehicle and receiving from the vehicle terminal the image for face authentication photographed by a predetermined photographing means associated with the vehicle when the vehicle arrives at the destination and then gets off the vehicle. a second face authentication processing unit that specifies a user corresponding to the image for face authentication by executing face authentication by the face authentication means and creates a list of people who have exited the vehicle; The list of exited vehicles is collated, and if there is a discrepancy in the collation result, it is determined that there is a possibility of occurrence of the idle state, and the possibility of occurrence of the idle state is determined for the pre-registered terminal to be notified. and a notification processing unit that notifies.

A server according to the present invention includes a server, a vehicle terminal installed in a vehicle, and a facility terminal installed in a facility that is a destination of the vehicle, and is configured to detect a state in which a user who has boarded the vehicle has not exited the vehicle. A server in a vehicle left-behind detection system for detecting an image for registration in which a user scheduled to use the vehicle is photographed by a predetermined photographing means, and in which the user's face is captured, and for specifying the user a face authentication pre-registration unit that acquires the user identification information and the user identification information and performs the face authentication pre-registration of the user by performing learning for a predetermined face authentication means using the registration image and the user identification information receiving from the vehicle terminal the image for face authentication photographed at the timing of getting into the vehicle by a predetermined photographing means associated with the vehicle, and performing face authentication by the face authentication means, thereby A first face authentication processing unit that specifies a user corresponding to the authentication image and creates a boarding list, and the face authentication image photographed in the facility by a predetermined photographing means provided in the facility is received from the facility terminal and face authentication is performed by the face authentication means to identify the user corresponding to the image for face authentication and create a list of people who have exited the vehicle; At the time of sufficiency, the boarding list and the alighting list are collated, and if there is a discrepancy in the collation result, it is determined that there is a possibility that the neglected state may occur. and a notification processing unit that notifies the possibility of occurrence of the idle state.

According to the present invention, by collating a boarded list created based on face recognition when boarding and an alighted list created based on face recognition when getting off, a person left in the car who has boarded but not alighted It is possible to detect and notify the possibility of

1 is a block diagram showing an example of a configuration of a vehicle left-in-vehicle detection system 100 according to the present invention; FIG. 1 is a block diagram showing an example of a configuration of a server 10 corresponding to at least one embodiment of the invention; FIG. FIG. 4 is a flow chart diagram showing the flow of face authentication pre-registration processing in the server 10 according to the present invention. FIG. 4 is a flow chart diagram showing the flow of left-in-vehicle detection processing in the server 10 according to the present invention. FIG. 4 is a flow chart diagram showing the flow of in-vehicle motion detection processing in the server 10 according to the present invention.

Hereinafter, an example of a left-in-vehicle detection system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of a left-in-vehicle detection system 100 according to the present invention. As shown in FIG. 1, the left-in-vehicle detection system 100 includes at least a server 10, a vehicle terminal 20, a facility terminal 30, and notification target terminals 401 to 40n. The server 10, the vehicle terminal 20, the facility terminal 30, and the notification target terminals 401 to 40n are connected to each other via a communication network 50 such as the Internet.

The server 10 is a device that executes processing necessary to detect leaving in the vehicle and appropriately communicates with the vehicle terminal 20, the facility terminal 30, and the notification target terminals 401 to 40n. Details of the functional configuration of the server 10 will be described later.

The vehicle terminal 20 is a terminal mounted on a vehicle and capable of communicating with the server 10 . Specifically, the vehicle terminal 20 has a function of transmitting to the server an image for face authentication of the user captured at a predetermined timing based on a predetermined photographing means associated with the vehicle. In addition, the vehicle terminal 20 has a function of receiving various information from the server. The photographing means may be a camera device provided in the vehicle terminal 20, or may be configured to acquire an image for face authentication from a camera device installed independently of the vehicle terminal 20. FIG. In addition, it is preferable that the photographing means for photographing inside the vehicle is provided near the entrance.

Here, the vehicle may be any vehicle as long as it can be used for pick-up and drop-off, and an example is a pick-up bus in a kindergarten, nursery school, or the like. Also, the method of installing the vehicle terminal 20 in the vehicle is not particularly limited, and the terminal may be detachably installed in the vehicle, or may be a terminal fixedly installed in the vehicle. Also, the term user is used as a term indicating a person who gets into a vehicle, and an example thereof includes kindergarten children, nursery school children, and the like.

Also, the vehicle 20 may include a human sensor 60 inside the vehicle. The human sensor 60 can adopt various methods such as an infrared method, an ultrasonic method, a laser method, an image processing method, and the like.

The facility terminal 30 is a terminal installed in a facility and capable of communicating with the server 10 . Specifically, the facility terminal 30 has a function of transmitting to the server an image for face authentication of the user captured at a predetermined timing based on a predetermined photographing means associated with the vehicle. The facility terminal 30 also has a function of receiving various information from the server 10 .

Here, the facility is not limited as long as it is a facility serving as the destination of the vehicle, but examples include kindergartens and nursery schools.

The notification target terminals 401 to 40n (hereinafter, the notification target terminals 401 to 40n may be collectively referred to as the notification target terminal 40) are terminals registered as notification targets in the vehicle left-in-vehicle detection system 100. FIG. One example is a terminal owned by a guardian of a child attending a kindergarten, nursery school, or the like. Further, it is assumed that the vehicle terminal 20 and the facility terminal 30 may be subject to notification as the notification target terminal 40 .

In addition, the server 10 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a memory, a hard disk drive, etc., which are normally provided in a general computer or server device. It is assumed that a storage is provided (not shown). Needless to say, various processes are executed by programs in order to cause these general computers and server devices to function as the server 10, the vehicle terminal 20, and the facility terminal 30 of this example.

The server 10, the vehicle terminal 20, and the facility terminal 30 use the data read into the memory as appropriate to execute information processing necessary for various processes in the CPU, and store the obtained processing results in the storage device as necessary. . A storage device has a function as a storage medium that stores various kinds of information. Although the configuration of the storage device is not particularly limited, examples thereof include HDD and SSD.

In the example shown in FIG. 1, the left-in-vehicle detection system 100 includes the server 10, the vehicle terminal 20, the facility terminal 30, the notification target terminals 401 to 40n, and the human sensor 60, but it does not necessarily include all of these. It is not necessary, and it is possible to implement it by appropriately combining necessary devices according to a specific embodiment of the left-in-vehicle detection process.

FIG. 2 is a block diagram showing an example of the configuration of the server 10 corresponding to at least one embodiment of the invention. As shown in FIG. 2, the server 10 includes at least a face authentication pre-registration unit 11, a first face authentication processing unit 12, a second face authentication processing unit 13, a notification processing unit 14, and a storage unit 15. .

The face authentication pre-registration unit 11 obtains an image for registration in which the face of the user who is planning to use the vehicle is photographed by a predetermined photographing means, and user identification information for identifying the user. It has a function of carrying out pre-registration of the user's face for authentication by learning a predetermined face authentication means using the registration image and the user identification information.

Here, the predetermined photographing means is not particularly limited as long as it is a means capable of photographing the user's face. Examples of the predetermined photographing means include a smartphone owned by the user and a camera provided at a predetermined location. An example of a camera provided at a predetermined location is a camera provided in a terminal for face authentication pre-registration installed in a facility. Note that the photographing means is realized by, for example, a two-dimensional photographing camera or a three-dimensional photographing camera capable of acquiring depth (depth) information. When the photographing means is implemented by a two-dimensional photographing camera, the photographing means may further include a sensor capable of acquiring depth information.

Also, the user identification information is not particularly limited as long as it is information that can identify a user, but information that has no possibility of duplication of content among a plurality of users is preferable. Examples of such information include an identification ID assigned to each user for identifying the user. Another example of user identification information is user's personal information. Examples of the user's personal information include the user's name, age, sex, and address. When a kindergarten or nursery school child is registered as a user, information related to the kindergarten, such as the child's class, may also be registered.

Also, the configuration for acquiring the registration image and the user identification information is not particularly limited, and at least one of the registration image and the user identification information may be received from an external device. The specific information may be acquired from a storage device included in the server 10 .

Further, the face authentication means means means for inputting a photographed image showing a person's face into the learning model and outputting the authentication result of the person whose face is photographed in the photographed image. The face authentication means may be any means including existing technology as long as it is a means for realizing so-called "face authentication", which is a technology that can uniquely identify the same person even from different facial photographs of the same person. may be The face authentication means can be implemented using a learning model learned by deep learning technology such as a neural network, for example. Note that the face authentication means may be implemented by a face authentication program stored in a storage device included in the server 10, or may be implemented by a face authentication program stored in a storage device of an external device other than the server 10. .

In addition, learning for the face authentication means using the registration image and the user identification information means that the learning model is updated by machine learning using the registration image and the user identification information as teacher data. means. For example, the learning here is supervised learning. User-specific information plays a role as a label in supervised learning. By performing learning on the face authentication means, even if a photographed image different from the image for registration is input to the learning model from the next time onward, the user who appears in the image for registration can be uniquely identified. it becomes possible to It is preferable that the number of teacher data (combinations of images for registration and user identification information) used to update the learning model is sufficient to ensure the accuracy of face recognition.

Further, pre-registration of a user's face for authentication is pre-registration for performing face authentication of the user, and means that the user is registered as a result of having the face authentication means perform learning. Since the user can be identified from the photographed image showing the user's face by making the face authentication means perform learning in advance, the user's identification information in the learning model and the user's user identification information can be combined. By correlating and recording, it is possible to acquire the associated user identification information when the user is determined to be the user the next time the face authentication is performed on the user. Specifically, the pre-registration of the user's face authentication means inputting a photographed image showing the user's face that is to be recognized by the learning model for face authentication and making the learning model learn it.

The first face authentication processing unit 12 identifies the user corresponding to the face authentication image by executing face authentication by the face authentication means on the face authentication image taken at the timing of getting into the vehicle, It has a function to create a boarded list. An image for face authentication is taken by a camera device as a photographing means for each user who gets on the vehicle, and the image for face authentication is received from the vehicle terminal 20 . As an example of acquisition of an image for face authentication in a vehicle, by displaying an image captured by a camera device included in a tablet terminal as the vehicle terminal 20 on the display of the tablet terminal, the user who takes the image can see the face displayed on the display. You may enable it to perform imaging|photography, looking at . In addition, as guidance to guide the user's face to a predetermined position in the shooting range, for example, a sentence such as "Put your face inside the circle!" , and when it is detected that the user's face is generally within the circle, the shutter is automatically released to acquire the image for face authentication. Then, face authentication is performed by the face authentication means using the acquired image for face authentication. As a result of face authentication, when the user appearing in the image for face authentication can be uniquely specified, the user specifying information is extracted. Using the extracted user identification information, the user is added to the boarding list and recorded. Such face authentication processing by the first face authentication processing unit 12 is executed each time a user gets on the vehicle so that all the users who get on the vehicle are included in the boarding list.

The second face authentication processing unit 13 performs face authentication using the face authentication means on an image for face authentication captured when the vehicle gets off the vehicle after arriving at the destination (after getting off the vehicle). It has a function of specifying the user corresponding to the image and creating a list of people who have exited the vehicle. In the second face authentication processing section 13, similarly to the first face authentication processing section 12, face authentication is performed by face authentication means on the obtained image for face authentication. As a result of face recognition, when the user in the image for face recognition can be uniquely identified, the user identification information is extracted, and the extracted user identification information is used to add the user to the exited list. Record. Such face authentication processing by the second face authentication processing unit 13 is executed each time a user gets off the vehicle, so that all the users who get off the vehicle are included in the getting off list.

The image for face authentication acquired by the second face authentication processing unit 13 may be configured to receive from the vehicle terminal 20 an image for face authentication photographed by a predetermined photographing means associated with the vehicle. Alternatively, an image for face authentication photographed by a predetermined photographing means provided in the facility, which is the destination of the vehicle, may be received from the facility terminal 30 . That is, the vehicle terminal 20 or the facility terminal 30 may be used as long as the exit list can be created after the arrival at the destination facility.

In addition, in order to determine whether the image for face authentication received from the vehicle terminal 20 is an image for creating a list of people who have boarded or an image for creating a list of people who have exited the vehicle, the switching timing is determined by the server. need to grasp. The vehicle terminal 20 may be configured to switch manually, but it is preferable to have a means for automatically grasping the timing of arrival at the facility, which is the destination, so that switching can be performed automatically. As an example, the position information of the vehicle is grasped based on the GPS information, the stopping position of the vehicle upon arrival at the destination is registered in advance, and when the vehicle reaches the stop position registered in advance, the position information of the vehicle is registered. It is conceivable to determine arrival at the facility and switch from creation of a boarding list to creation of alighting list.

The notification processing unit 14 collates the boarding completed list and the alighting completed list, and determines that there is a possibility of occurrence of an idle state when there is a discrepancy in the collation result, and notifies the pre-registered notification target terminal. It has a function to notify the possibility of occurrence of an idle state. If all the users who boarded the vehicle have gotten off the vehicle without any problems, the already-boarded list and the already-exited list will match perfectly. A situation arises where a name exists but is not in the exited list. In this case, there is a possibility that the vehicle is locked up and left in the vehicle. Therefore, if such a list deviation is detected, it is determined that there is a possibility that the vehicle is left unattended. When determining that there is a possibility that the idle state will occur, the notification processing unit 14 notifies the previously registered notification target terminal 40 of the possibility of the occurrence of the idle state. The pre-registered notification target terminal may be the facility terminal 30 or a terminal possessed by a guardian in the case where the user is a kindergarten child. By sending a notification to the facility terminal 30, it is possible to notify the person in charge of the facility of the possibility of the neglected state, and register the terminal of the guardian of the child in the kindergarten, nursery school, etc. as the notification target terminal 40 By doing so, the parent informed of the possibility of the neglected state will contact the kindergarten, and as a result, it will be possible to notify the person in charge of the kindergarten of the possibility of the neglected state.

It is preferable that the collation processing in the notification processing unit 14 is executed after a predetermined condition for executing collation is satisfied. This is because if the verification process is started immediately after the vehicle arrives at its destination, there is a high possibility that all passengers have not yet gotten off the vehicle. There is a risk that the notification will be made even though the possibility of the state has not occurred, and that the recipient of the notification will be unnecessarily disturbed. Therefore, it is possible to avoid the verification process before the completion of getting off the vehicle by executing the verification process after a predetermined condition is satisfied. As an example of the predetermined condition, it is conceivable that a predetermined time, such as 15 minutes, elapses from the timing when the vehicle arrives at the destination facility (automatically determined based on the GPS information of the vehicle).

Further, when the user is specified by performing face authentication in the first face authentication processing unit 12 and/or the second face authentication processing unit 13, the notification processing unit 14 performs the notification target registered in association with the user. You may make it notify with respect to the terminal 40 that face authentication was performed. With this configuration, for example, every time face authentication processing is executed for a child who attends a kindergarten or nursery school, the parent of the child is notified that the child has been recognized by face authentication. This has the effect of providing a sense of security, and from the point of view of a guardian, it is possible to suspect the possibility of leaving the child in the car due to the lack of notification. In addition, by transmitting the image for face authentication used when performing face authentication together with the notification, the guardian can check the face photograph each time.

The storage unit 15 has a function of storing data necessary for various processes performed in the server 10 and data obtained as a result of the processes.

Note that some of the functions realized by the configuration of the server 10 may be realized in the vehicle terminal 20 or the facility terminal 30 instead of the server 10 .

FIG. 3 is a flow chart showing the flow of face authentication pre-registration processing in the server 10 according to the present invention. As shown in FIG. 3, the face authentication pre-registration process acquires an image for registration in which the user's face is photographed by a predetermined photographing means, and user identification information for identifying the user. is started (step S101). As an example, the face authentication pre-registration process is started when the server 10 receives a registration image showing the user's face and user identification information of the user from the smartphone owned by the user.

Next, the server 10 performs learning for predetermined face authentication means using the registration image and the user identification information (step S102). As an example, the server 10 learns the user's face authentication by deep learning using a neural network, so that the user can be uniquely identified when the user's face photograph is input from the next time onward.

Next, the server 10 registers the learning result by associating and registering the identification information for identifying the person by the predetermined face authentication means and the user identification information (step S103), and ends the process. As an example, the server 10 causes the external device to store the update result of the learning model in association with the user, and terminates the process.

FIG. 4 is a flow chart showing the process of detecting left in the vehicle in the server 10 according to the present invention. As shown in FIG. 4, the left-in-vehicle detection process is started when the server 10 receives a face authentication image for boarding confirmation from the vehicle terminal 20 (step S201). Next, the server 10 identifies the user by performing face authentication using the face authentication image for boarding confirmation, and adds the user to the boarding list (step S202). These steps S201 and S202 are executed each time the user gets into the vehicle.

Next, the server 10 automatically determines the timing at which the vehicle arrives at the destination facility by, for example, the GPS information of the vehicle, thereby confirming the transition to the step of getting off the vehicle (step S203). Next, the server 10 receives the face authentication image for confirming getting off the vehicle (step S204). Next, the server 10 identifies the user by performing face authentication using the face authentication image for confirming getting off the vehicle, and adds the user to the exited list (step S205). These steps S204 and S205 are executed for all the users on board the vehicle.

Next, the server 10 determines that a predetermined condition for performing collation is satisfied (step S206). Next, the server 10 collates the boarding list and the alighting list, and determines that there is a possibility of occurrence of the neglected state when there is a discrepancy in the collation result (step S207). Then, the server 10 notifies the pre-registered notification target terminal of the possibility of occurrence of the left state (step S208), and terminates the left-in-vehicle detection process.

FIG. 5 is a flow chart showing the flow of motion detection processing in the server 10 according to the present invention. As shown in FIG. 5, the motion detection process is started by confirming whether or not a predetermined detection start condition is satisfied in the vehicle terminal 20 (step S301). The detection start conditions include, for example, a condition that a predetermined time has passed since the arrival of the vehicle at the destination was detected, a condition that the detection start operation is performed manually, a condition that a predetermined time has passed since the engine was stopped, and the like. can be considered. When the predetermined detection start condition is satisfied (S301-Y), the vehicle terminal 20 starts motion detection by the human sensor 60 (step S302). Next, the vehicle terminal 20 determines whether or not any action is detected (step S303). When motion is detected (S303-Y), the vehicle terminal 20 notifies the server 10 of the motion detection (step S304), and proceeds to step S305. If motion is not detected (S303-N), the vehicle terminal 20 proceeds to step S305. Next, the vehicle terminal 20 confirms whether or not a predetermined detection end condition is satisfied (step S305). If the detection end condition is not satisfied, the vehicle terminal 20 returns to step S303 and continues motion detection. Then, when the detection end condition is satisfied, the vehicle terminal 20 ends motion detection (step S306).

As described above, the left-in-vehicle detection system 100 according to the present embodiment includes a server and a vehicle terminal installed in a vehicle, and is a left-in-vehicle detection system for detecting a left-in-vehicle state in which a user who gets into the vehicle does not get out of the vehicle. The vehicle terminal includes a vehicle-side transmission unit that transmits to the server an image for face authentication of the user captured at a predetermined timing based on a predetermined image capturing unit associated with the vehicle; a vehicle-side receiving unit for receiving information, and the server is provided with a registration image including a face obtained by photographing a user who is scheduled to use the vehicle by a predetermined photographing means, and a registration image for identifying the user. a face authentication pre-registration unit that acquires the user identification information and the user identification information and performs the face authentication pre-registration of the user by performing learning for a predetermined face authentication means using the registration image and the user identification information; A first face for creating a boarding list by specifying a user corresponding to the face authentication image by performing face authentication by the face authentication means on the face authentication image taken at the timing of getting into the vehicle. The user corresponding to the face authentication image is specified by performing face authentication by the authentication processing unit and the face authentication means on the face authentication image taken when the vehicle arrives at the destination and when the vehicle gets off the vehicle, A second face authentication processing unit that creates a list of people who have exited the vehicle and a list of people who have exited the vehicle when a predetermined condition is satisfied. A notification processing unit is provided for judging and notifying the possibility of occurrence of an idle state to a terminal to be notified which has been registered in advance. By matching with a list of people who have exited the vehicle, which is created based on face recognition that is sometimes performed, it is possible to detect and notify the possibility of being left in the vehicle without getting off the vehicle.

In addition, in the system 100 for detecting left in the vehicle, the notification target terminal includes a terminal registered in advance in association with each user, and the notification processing unit registers in association with the user detected as a possibly left user. Since the possibility of occurrence of the neglected state is notified to the notification target terminal that has been registered, the notification target terminal registered in association with the user who may be left in the neglected state, for example, Since the notification is sent to the terminal, the parent can quickly recognize the possibility of the neglected state, and can create an opportunity to make an inquiry to the person in charge of the kindergarten or nursery school. By adopting this mechanism, notifications will be sent to people other than the responsible person on the facility side, so the possibility of human error, such as detecting the possibility of being left unattended but no one noticing the notification, is further reduced. becomes possible.

Further, in the left-in-vehicle detection system 100, the notification processing unit further performs face authentication in the first face authentication processing unit and/or the second face authentication processing unit and, when the user is identified, associates the user with the user. For example, every time a child who attends a kindergarten or a nursery school undergoes face recognition processing, the face of the child's guardian is notified. Since the recognition of the child is notified by the authentication, it is possible to obtain the effect of leading to a sense of security for the guardian. can be suspected. In this case, by contacting the facility side, the guardian will share the situation with the person in charge of the facility, so that the risk of delay in dealing with the trouble can be reduced.

Further, in the left-in-vehicle detection system 100, at least one or more human sensors are installed inside the vehicle. The detection shall be terminated when the detection end condition is satisfied as a trigger, and when some motion is detected, the motion detection shall be notified to the server via the vehicle terminal or directly. When a motion detection notification is received, the motion detection in the vehicle is notified to the pre-registered notification target terminal. Even if the liquid leaks for some reason and is left inside the vehicle, the human sensor can detect that the water is left inside the vehicle, so that safety can be ensured.

In the above, when the user is picked up and boarded and moved to the destination, the user is dropped off at the destination facility, that is, when the user goes to kindergarten by the courtesy bus, the situation in which the user is left in the car is prevented. Although the description has been made mainly based on the above, the present invention is not limited to this. In other words, the present invention may be similarly applied to a case where a user is carried in a facility and the user is dropped off at a desired drop-off position, that is, when a kindergarten is dropped off by a courtesy bus. In that case, in the vehicle terminal 20 or the facility terminal 30 when getting on the vehicle from the facility, face authentication processing similar to that of the first face authentication processing unit 12 is executed to create a boarding list, and when getting off the vehicle The vehicle terminal 20 executes face authentication processing similar to that of the second face authentication processing unit 13 to create a list of exited vehicles, and the notification processing unit 14 compares the list of completed boarding with the completed list of exited vehicles. good too. When leaving the kindergarten, it is assumed that the guardian will be there to pick up the child, so the possibility of the child being left in the car is low. becomes possible. In addition, by notifying the notification target terminal 40 that the face authentication process has been performed every time each face authentication process is performed when leaving the kindergarten, the guardian can feel secure even when leaving the kindergarten. From the point of view of the guardian, it is possible to suspect trouble such as the possibility that the child is not in the vehicle due to the fact that the notification is not received.

It should be noted that the present invention is not limited to the embodiments described above, and various configurations or embodiments can be adopted without departing from the gist of the present invention.

100 Vehicle Left-In Detection System 10 Server 11 Face Authentication Pre-Registration Unit 12 First Face Authentication Processing Unit 13 Second Face Authentication Processing Unit 14 Notification Processing Unit 15 Storage Unit 20 Vehicle Terminal 30 Facility Terminal 40, 401 to 40n Notification Target Terminal 50 Communication Network 60 Human sensor

Claims (5)

Vehicle left-in-vehicle detection for detecting a left-in-vehicle state in which a user who got into the vehicle does not get off, comprising a server, a vehicle terminal installed in the vehicle, and a facility terminal installed in a facility that is the destination of the vehicle. a system,
The vehicle terminal
a vehicle-side transmission unit configured to transmit, to the server, an image for face authentication of a user captured at a predetermined timing based on a predetermined capturing unit associated with the vehicle;
A vehicle-side receiving unit that receives various information from the server,
The facility terminal
a facility-side transmission unit configured to transmit to the server an image for face authentication of the user captured by a predetermined imaging unit provided in the facility;
a facility-side receiving unit that receives various information from the server;
The server is
An image for registration in which the face of a user scheduled to use the vehicle is photographed by a predetermined photographing means and user identification information for identifying the user are acquired, and the image for registration and the user identification information are obtained. a face authentication pre-registration unit that performs face authentication pre-registration of the user by learning a predetermined face authentication means using the user identification information;
By performing face authentication by the face authentication means on the image for face authentication taken at the timing of getting into the vehicle, a user corresponding to the image for face authentication is specified, and a boarding list is created. a first face authentication processing unit;
A second face that identifies a user corresponding to the image for face authentication by performing face authentication by the face authentication means on the image for face authentication photographed in the facility, and creates a list of people who have exited the vehicle. an authentication processing unit;
When a predetermined condition is satisfied, the list of boarded vehicles and the list of alighted vehicles are collated, and if there is a discrepancy in the comparison result, it is determined that there is a possibility that the state of being left unattended may occur, and a notification target terminal registered in advance is notified. and a notification processing unit for notifying the possibility of occurrence of the left-in-vehicle detection system. The notification target terminal includes a terminal registered in advance in association with each user,
2. The notification processing unit according to claim 1 , wherein the notification processing unit notifies the possibility of occurrence of the idle state to the notification target terminal registered in association with the user detected as the possibly idle state user. In-vehicle detection system. When the notification processing unit executes face authentication in the first face authentication processing unit and/or the second face authentication processing unit and identifies the user, the notification target terminal registered in association with the user 3. The left-in-vehicle detection system according to claim 2 , which notifies that the face authentication has been performed for. At least one or more motion sensors are installed inside the vehicle,
The motion sensor starts detection when a predetermined detection start condition is satisfied as a trigger, and terminates detection when a predetermined detection end condition is satisfied as a trigger. Shall directly notify the server of operation detection,
2. The left-in-vehicle detection system according to claim 1, wherein, when receiving a motion detection notification from said human sensor, said notification processing unit notifies a pre-registered notification target terminal of motion detection in the vehicle. Vehicle left-in-vehicle detection for detecting a left-in-vehicle state in which a user who got into the vehicle does not get off, comprising a server, a vehicle terminal installed in the vehicle, and a facility terminal installed in a facility that is the destination of the vehicle. A server in the system,
An image for registration in which the face of a user scheduled to use the vehicle is photographed by a predetermined photographing means and user identification information for identifying the user are acquired, and the image for registration and the user identification information are obtained. a face authentication pre-registration unit that performs face authentication pre-registration of the user by learning a predetermined face authentication means using the user identification information;
An image for face authentication photographed at the timing of getting into the vehicle by a predetermined photographing means associated with the vehicle is received from the vehicle terminal, and face authentication is performed by the face authentication means. a first face authentication processing unit that identifies the user corresponding to the image and creates a boarding list;
receiving from the facility terminal the image for face authentication photographed in the facility by a predetermined photographing means provided in the facility, and performing face authentication by the face authentication means to obtain the image for face authentication; a second face authentication processing unit that identifies the relevant user and creates a list of people who have exited the vehicle;
When a predetermined condition is satisfied, the list of boarded vehicles and the list of alighted vehicles are collated, and if there is a discrepancy in the comparison result, it is determined that there is a possibility that the state of being left unattended may occur, and a notification target terminal registered in advance is notified. A server, comprising: a notification processing unit that notifies the user of the possibility of occurrence of an idle state.

Images