JP6956687B2 - Abandonment detection device, abandonment detection method and abandonment detection program - Google Patents

Description

The present invention relates to a technique for detecting the occupant's leaving outside the vehicle by using an image inside the vehicle.

Patent Document 1 discloses a passenger management system that controls the entry and exit of passengers on a large bus. The passenger management system of Patent Document 1 controls the entry and exit of the occupant by receiving the identification signal of the weak radio wave transmitted from the tag carried by the occupant by the antenna installed near the door of the bus, and is not in the vehicle. Easily identify the occupants.

Japanese Unexamined Patent Publication No. 2004-139459

However, the technique of Patent Document 1 is based on the premise that each occupant carries a tag, and does not use an image of the vehicle interior. The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a technique for detecting the occupant's leaving outside the vehicle by using an image inside the vehicle.

The abandonment detection device of the present invention sets at least one of the parking / stopping points of the vehicle as the abandonment detection point based on the attribute or condition of the rear road, which is the road on which the vehicle may travel after the parking / stopping point. The first image is the image taken from the decision unit to decide and the camera that shoots the interior of the vehicle from the start of running of the vehicle to the time when the occupant gets off from the vehicle at the abandonment detection point, and the vehicle is from the abandonment detection point. The image taken at the time of departure is used as the second image, and the image acquisition unit to be acquired and the occupant shown in the first image but not in the second image are left behind and boarded after getting off the vehicle at the detection point. It is provided with a detection unit that detects an unattended occupant and an output control unit that outputs information to the effect that there is an abandoned occupant to an output device mounted on the vehicle.

The abandonment detection method of the present invention sets at least one of the parking / stopping points of a vehicle as an abandonment detection point based on the attribute or state of the rear road, which is a road on which the vehicle may travel after the parking / stopping point. The first image is the image taken from the camera that takes a picture of the interior of the vehicle after the vehicle starts running and before the occupant gets off the vehicle at the abandonment detection point. The images taken in the above are acquired as the second image, and the occupants who are reflected in the first image but not in the second image are detected as the abandoned occupants who have not boarded after getting off the vehicle at the abandonment detection point. Then, the information that there is an abandoned occupant is output to the output device mounted on the vehicle.

The abandonment detection program of the present invention displaces a computer at at least one of the parking / stopping points of a vehicle based on the attributes or conditions of the rear road, which is the road on which the vehicle may travel after the parking / stopping point. The first image is the image taken from the decision unit that determines the detection point and the camera that captures the interior of the vehicle, after the vehicle starts running and before the occupant disembarks at the abandonment detection point. The image taken when the vehicle departs from the vehicle is used as the second image, and the image acquisition unit to be acquired and the occupant appearing in the first image but not in the second image are left behind and disembarked from the vehicle at the detection point. This is an abandonment detection program for operating as a detection unit that detects an abandoned occupant who is not on board and an output control unit that outputs information to the effect that there is an abandoned occupant to an output device mounted on the vehicle.

According to the present invention, it is possible to detect the leaving of an occupant by comparing the first image and the second image, which are images of the vehicle interior.

It is a block diagram which shows the structure of the abandonment detection apparatus of Embodiment 1. FIG. It is a flowchart which shows the acquisition process of the 1st image by the abandonment detection apparatus of Embodiment 1. FIG. It is a flowchart which shows the abandonment detection process by the abandonment detection device of Embodiment 1. It is a figure which shows the abandonment warning screen. It is a block diagram which shows the structure of the abandonment detection device of Embodiment 2. It is a block diagram which shows the structure of the abandonment detection device of Embodiment 3. It is a flowchart which shows the acquisition process of the 1st image by the abandonment detection apparatus of Embodiment 3. It is a flowchart which shows the abandonment detection process by the abandonment detection device of Embodiment 3. It is a figure which shows the abandonment warning screen. It is a figure which shows the hardware configuration of the abandonment detection device of Embodiment 3. It is a figure which shows the hardware configuration of the abandonment detection device of Embodiment 3. It is a figure which shows the configuration example of the abandonment detection device by a vehicle and a server.

<A. Embodiment 1>
<A-1. Configuration>
FIG. 1 is a block diagram showing the configuration of the abandonment detection device 101 of the first embodiment. The abandonment detection device 101 is a device that detects the occupant's abandonment outside the vehicle and warns the user who is the driver of the vehicle by using the output device 22 mounted on the vehicle. In the present specification, the vehicle on which the output device 22 is mounted and is the target of the occupant's abandonment detection is simply referred to as a "vehicle". The abandonment detection device 101 is connected to the camera 21 and the output device 22 so that they can be used. The abandonment detection device 101 includes an image acquisition unit 11, a position acquisition unit 12, a determination unit 13, a detection unit 14, an output control unit 15, and a data storage unit 16.

The image acquisition unit 11 acquires the captured image of the camera 21. The camera 21 is mounted on the vehicle and photographs the interior of the vehicle. It is desirable that the camera 21 covers the entire interior of the vehicle so that all the passengers of the vehicle can be photographed.

The position acquisition unit 12 acquires the position of the vehicle. For example, the position acquisition unit 12 acquires a GPS signal from a GPS (Global Positioning System) receiver mounted on the vehicle, and identifies the position of the vehicle based on this and map data.

The determination unit 13 acquires the position of the vehicle from the position acquisition unit 12 and acquires the map data from the data storage unit 16. Then, based on these, at least one point from the parking / stopping points of the vehicle is left behind and determined as a determination point. The abandonment determination point is a point where the detection unit 14 determines whether or not the occupant has been left outside the vehicle. The determination unit 13 refers to the map data to determine the road on which the vehicle may travel after the parking / stopping point (hereinafter referred to as “rear road”), and once based on the attribute or state of the rear road. Once the vehicle departs, it is difficult to return to the original parking / stopping point, and the parking / stopping point is left behind and determined as the judgment point.

The detection unit 14 acquires the abandonment determination point from the determination unit 13 and determines whether or not the occupant has been left outside the vehicle at the abandonment determination point. When the detection unit 14 determines that the occupant has been left outside the vehicle at the abandonment determination point, that is, detects the abandonment, the determination result is output to the output control unit 15.

When the output control unit 15 acquires the determination result from the detection unit 14, the output control unit 15 causes the output device 22 to output warning information indicating that the occupant has been left outside the vehicle. In FIG. 1, the output device 22 includes a display device 221 and a speaker 222 mounted in the vehicle interior. The output control unit 15 causes the display device 221 to display a warning screen and the speaker 222 to output a warning sound. However, the output device 22 may include at least one of the display device 221 and the speaker 222. The display device 221 is, for example, a liquid crystal display (LCD) or a HUD (Head-Up Display).

The data storage unit 16 stores various types of data necessary for processing of the abandonment detection device 101. The data stored in the data storage unit 16 includes map data and images taken by the camera acquired by the image acquisition unit 11.

<A-2. Operation>
The abandonment detection device 101 acquires, as a first image, an image taken by the camera 21 with all the original occupants on board before the abandonment occurs at the parking / stopping point. Then, when the vehicle parks and stops at the abandonment determination point and departs, the captured image of the camera 21 is acquired as the second image, and the first image and the second image are compared to detect the abandonment of the occupant.

FIG. 2 is a flowchart showing a shooting process of the first image by the abandonment detection device 101. Hereinafter, the shooting process of the first image will be described with reference to FIG. The flow of FIG. 2 starts, for example, when the vehicle is parked / stopped at the parking / stopping point. The determination unit 13 grasps that the vehicle is parked or stopped based on the information of various sensors of the vehicle such as a speed sensor or a brake sensor (not shown in FIG. 1).

When the vehicle is parked / stopped, the determination unit 13 determines whether the current parking / stopping point of the vehicle is the abandoned determination point (step S101). Specifically, the determination unit 13 acquires the position information of the vehicle from the position acquisition unit 12, and acquires the map data from the data storage unit 16. Then, based on this information, the determination unit 13 determines whether the road behind the current parking / stopping point of the vehicle is a road on which it is difficult to turn back. When the road behind the current parking / stopping point of the vehicle is a road on which it is difficult to turn back, the determination unit 13 leaves the current parking / stopping point and determines it as the determination point.

A road that is difficult to turn back is a road that is difficult to return to the original parking / stopping point when the vehicle departs from the parking / stopping point and travels on the back road. The determination unit 13 determines the abandonment determination point by referring to the information on the attributes of the road on the map included in the map data. Here, the attribute of a road is a universal property, that is, a property of a road that does not change with time. The attributes of a road include, for example, a road type indicating whether the road is a highway or a general road. For example, if the parking point is a parking area, the rear road is a highway. When a vehicle departs from a parking area and travels on the freeway, it must exit and re-enter the freeway to return to the original parking area. Therefore, when the determination unit 13 identifies that the rear road is an expressway from the information on the attributes of the rear road, the determination unit 13 leaves the parking area, which is the parking / stopping point, and determines it as the determination point.

In step S101, when the determination unit 13 leaves the current parking / stopping point and determines that it is not the determination point, the shooting process of the first image ends.

In step S101, when the determination unit 13 determines that the parking / stopping point of the vehicle is left behind and is the determination point, the camera 21 takes an image of the vehicle interior as the first image (step S102), and the image acquisition unit 11 takes the camera 21. The first image is acquired from. Here, the first image is taken from the time when the vehicle is parked and stopped until the occupant gets off the vehicle, specifically, until the door is opened. The first image is stored in the data storage unit 16 (step S103), and the shooting process of the first image is completed.

In the above, the shooting timing of the first image is set from the time when the vehicle is left behind and arrives at the determination point until the occupant gets off the vehicle. However, since it is sufficient that all the original occupants are photographed in the first image, the photographing timing may be before the vehicle arrives at the abandoned judgment point, at least after the vehicle starts traveling. It suffices as long as it is between the vehicle and the occupant getting off at the abandonment judgment point.

Other examples of the attributes of the road behind the abandonment judgment point are shown below. When the width of the rear road is narrow, for example, less than 5 m, the determination unit 13 may leave the parking / stopping point as a determination point. This is because narrow roads are difficult to make U-turns and difficult to turn back. Further, when the rear road is a one-way road, the determination unit 13 may leave the parking / stopping point as a determination point. This is because one-way roads cannot make U-turns and it is difficult to turn back. Further, the determination unit 13 may leave the parking / stopping point as a determination point when there is a median strip on the rear road. This is because a road with a median strip cannot make a U-turn and it is difficult to turn back. Further, when the slope of the rear road is steep, for example, 10% or more, the determination unit 13 may leave the parking / stopping point as a determination point. This is because steep roads are difficult to make U-turns and difficult to turn back. Further, the determination unit 13 may set the number of roads in the area to which the rear road belongs, such as an area within 500 m from the rear road, as the road density, and may leave the parking / stopping point as a determination point when the road density is less than the threshold value. .. This is because the roads that run in areas with low road density are rural roads, and these roads are difficult to make U-turns and difficult to turn back. Further, the determination unit 13 may leave the parking / stopping point as a determination point when the number of traffic lights per unit length (for example, 5000 m) of the rear road is less than the threshold value. This is because roads with a small number of traffic lights are rural roads, and these roads are difficult to make U-turns and difficult to turn back. In these examples, the width of the road, whether it is a one-way road, the presence or absence of a median strip, the slope, the road density of the area to which the rear road belongs, the number of traffic lights per unit length, etc. are the attributes of the road and are maps. Included in the data.

Here, although the attributes of the rear road are considered, the determination unit 13 may determine the abandonment determination point based on the condition of the rear road, or determine the abandonment determination point based on the combination of the attribute and the condition of the rear road. You may. The state of the road is a property of the road that changes over time, for example, the traffic volume, the degree of congestion, the presence or absence of construction, etc., and affects the difficulty of turning back. When the map data does not include information on the road condition, the determination unit 13 can acquire the road condition from, for example, the Vehicle Information and Communication System (VICS (registered trademark)). .. The determination unit 13 may make it a condition of the abandonment determination point that the traffic volume on the rear road is large, that is, it is equal to or higher than the threshold value. This is because roads with more traffic are less likely to make U-turns and more difficult to turn back. Further, the determination unit 13 may set that the degree of congestion on the rear road is equal to or higher than the threshold value as a condition of the abandonment determination point. This is because the more congested the road is, the harder it is to make a U-turn and the more difficult it is to turn back. Further, the determination unit 13 may make it a condition of the abandonment determination point that the construction is being carried out on the rear road. This is because the roads under construction are prone to traffic jams, and in that case it is difficult to turn back.

FIG. 3 is a flowchart showing the abandonment detection process by the abandonment detection device 101. Hereinafter, the abandonment detection process will be described with reference to FIG. The flow of FIG. 3 starts at the timing when the engine of the vehicle starts, for example, at the parking / stopping point.

First, the determination unit 13 determines whether or not the current vehicle point is the abandonment determination point (step S201). This determination method is the same as in step S101 of FIG. Therefore, when the determination unit 13 has already performed the process of step S101 of FIG. 2 for the same parking / stopping point, the determination result of step S101 may be diverted to step S201.

In step S201, if the current vehicle point is not the abandonment determination point, the abandonment detection process ends.

In step S201, if the current vehicle point is the abandonment determination point, the camera 21 takes an image of the vehicle interior as a second image (step S202), and the image acquisition unit 11 acquires the second image from the camera 21. .. Next, the detection unit 14 acquires the second image from the image acquisition unit 11 and the first image from the data storage unit 16, and compares the second image with the first image to determine whether or not the second image is left behind. Is determined (step S203). Specifically, the detection unit 14 performs image analysis on the first image and the second image, and identifies the occupants shown in the respective images. Then, the detection unit 14 compares the occupant shown in the first image with the occupant shown in the second image, and when there is an occupant shown in the first image but not shown in the second image, It is determined that the occupant has been left behind. In the following description, the occupant determined by the detection unit 14 to be left behind is also referred to as a “left-behind occupant”.

If the detection unit 14 determines in step S203 that the abandonment has not occurred, the abandonment detection process ends. On the other hand, in step S203, when the detection unit 14 determines that the abandonment has occurred, the output control unit 15 causes the output device 22 to issue a abandonment warning (step S204).

FIG. 4 shows a left-behind warning screen displayed on the display device 221 in step S204 when the output device 22 includes the display device 221. On the abandonment warning screen, the warning wording "The occupant has been left behind has been detected." Is displayed. As a result, the user can recognize that there is an occupant who has not returned to the vehicle while getting off at the parking / stopping point, and can decide whether to wait or depart depending on the situation.

In FIG. 4, the abandonment warning screen for displaying the warning wording is illustrated, but the abandonment warning screen may be in another mode as long as it is a display indicating that the abandonment has occurred. For example, by displaying the abandoned occupant shown in the first image, it may be displayed to indicate which occupant is left behind.

Further, in the above, the acquisition timing of the second image is the timing at which the engine of the vehicle starts at the parking / stopping point, but another timing may be used as long as the vehicle departs from the parking / stopping point. For example, the acquisition timing of the second image may be the timing when the gear changes from parking to another gear at the parking / stopping point, or the timing when the vehicle starts traveling from the parking / stopping point.

<A-3. Effect>
The abandonment detection device 101 according to the first embodiment is based on the attribute or state of the rear road, which is a road on which the vehicle may travel after the parking / stopping point at least one of the parking / stopping points of the vehicle. The first image is an image taken from the determination unit 13 that determines the abandonment determination point and the camera 21 that photographs the interior of the vehicle from the start of the vehicle running to the time when the occupant disembarks from the vehicle at the abandonment determination point. The image taken when the vehicle departs from the abandonment judgment point is used as the second image, and the image acquisition unit 11 to be acquired respectively and the occupant who is shown in the first image but not in the second image are left behind. It is provided with a detection unit 14 that detects an abandoned occupant who has not boarded the vehicle after getting off the vehicle, and an output control unit 15 that outputs information to the effect that there is an abandoned occupant to an output device 22 mounted on the vehicle. Therefore, according to the abandonment detection device 101, it is possible to prevent the abandonment by detecting the abandonment of the occupant by using the image in the vehicle interior and notifying the user of this.

The abandonment detection method according to the first embodiment leaves at least one of the parking / stopping points of the vehicle based on the attribute or condition of the rear road, which is a road on which the vehicle may travel after the parking / stopping point. The first image is the image taken from the camera that captures the interior of the vehicle as the detection point and after the vehicle starts running until the occupant disembarks at the abandonment detection point. The images taken at the time of departure are acquired as the second images, and the occupants who are shown in the first image but not in the second image are left behind after getting off the vehicle at the detection point. It detects that it is an occupant and outputs information that there is an abandoned occupant to the output device mounted on the vehicle. Therefore, according to the abandonment detection method according to the first embodiment, it is possible to prevent the abandonment by detecting the abandonment of the occupant using the image in the vehicle interior and notifying the user of this.

The abandonment detection program according to the first embodiment sets the computer at at least one of the parking / stopping points of the vehicle to the attribute or state of the rear road, which is the road on which the vehicle may travel after the parking / stopping point. Based on this, the first image taken from the determination unit 13 that determines the abandonment detection point and the camera 21 that photographs the interior of the vehicle from the start of the vehicle running to the time when the occupant disembarks from the vehicle at the abandonment detection point. The image and the image taken when the vehicle departs from the abandonment detection point are used as the second image, and the image acquisition unit 11 to be acquired and the occupant appearing in the first image but not in the second image are left behind. To operate as a detection unit 14 that detects an abandoned occupant who has not boarded after getting off the vehicle at a detection point, and an output control unit 15 that outputs information to the effect that there is an abandoned occupant to an output device 22 mounted on the vehicle. , It is a left-behind detection program. Therefore, according to the abandonment detection program according to the first embodiment, it is possible to prevent the abandonment by detecting the abandonment of the occupant using the image in the vehicle interior and notifying the user of this.

<B. Embodiment 2>
<B-1. Configuration>
FIG. 5 is a block diagram showing the configuration of the abandonment detection device 102 of the second embodiment. The abandonment detection device 102 is connected to the camera 21 and the output device 22 so that they can be used. The abandonment detection device 102 includes an input reception unit 17 and a route search unit 18 in addition to the configuration of the abandonment detection device 101 of the first embodiment. Since the configuration of the abandonment detection device 102 other than the input reception unit 17 and the route search unit 18 is the same as that of the abandonment detection device 101, the description thereof will be omitted here.

The input receiving unit 17 has a function of receiving a user's input to the abandoned detection device 102. The user's input to the abandonment detection device 102 includes, for example, information on the waypoint or destination of the vehicle. That is, the input reception unit 17 functions as an information acquisition unit that acquires information on the waypoints and destinations of the vehicle. When the input receiving unit 17 is composed of a touch pad, the input receiving unit 17 and the display device 221 are integrally configured as a touch panel.

The route search unit 18 acquires the vehicle position information from the position acquisition unit 12, the vehicle waypoint and destination information from the input reception unit 17, and the map data from the data storage unit 16, and based on these, the vehicle Search for a travel route from your current location to your destination via a stopover. The route search unit 18 outputs the searched travel route to the determination unit 13 and the output control unit 15. The output control unit 15 causes the output device 22 to output the guide information of the traveling route.

In the second embodiment, since the determination unit 13 acquires the travel route of the vehicle from the route search unit 18, the rear road and the abandonment determination point can be determined using this travel route. The determination unit 13 sets the road on which the vehicle travels after the parking / stopping point along the travel route as the rear road. As a result, the determination unit 13 can accurately determine the rear road. Further, when the waypoint and the destination are set on the traveling route, the determination unit 13 may leave the parking / stopping point far from both the waypoint and the destination as a determination point. That is, the determination unit 13 may use a parking / stopping point whose distance to the nearest point along the traveling route among the waypoint and the destination is equal to or greater than the threshold value as the abandonment determination point. The operation of the determination unit 13 other than this is the same as that of the first embodiment.

<B-2. Effect>
In addition to the configuration of the abandonment detection device 101 of the first embodiment, the abandonment detection device 102 of the second embodiment includes an input reception unit 17 which is an information acquisition unit for acquiring information on the destination of the vehicle, and a destination of the vehicle. A route search unit 18 for searching a travel route up to is provided. The rear road is a road on which the vehicle travels along the travel route after the parking / stopping point. Therefore, according to the abandonment detection device 102, the abandonment determination can be made after accurately determining the rear road.

Further, in the abandonment detection device 102 of the second embodiment, the input reception unit 17 acquires the information on the waypoint of the vehicle, and the route search unit 18 is the travel route of the vehicle reaching the destination via the waypoint. To explore. The abandonment determination point is either a waypoint or a destination, and is a parking / stopping point where the distance to the nearest point along the traveling route is equal to or greater than the threshold value. Therefore, according to the abandonment detection device 102, the abandonment determination can be made when the parking / stopping point is some distance away from the next parking / stopping point.

<C. Embodiment 3>
<C-1. Configuration>
FIG. 6 is a block diagram showing the configuration of the abandonment detection device 103 of the third embodiment. The abandonment detection device 103 is connected to the camera 21, the output device 22, and the mobile terminal 23, and is configured to be available. The abandonment detection device 103 includes an occupant information registration unit 19 and a communication unit 20 in addition to the configuration of the abandonment detection device 102 of the second embodiment. Since the configuration of the abandonment detection device 103 other than the occupant information registration unit 19 and the communication unit 20 is the same as that of the abandonment detection device 102, the description thereof will be omitted here.

The input reception unit 17 acquires the face image of the occupant of the vehicle and the contact information such as the e-mail address of the mobile terminal 23 carried by the occupant by user operation, and outputs the contact information to the occupant information registration unit 19.

The occupant information registration unit 19 links the occupant's face image with contact information such as the e-mail address of the mobile terminal 23 carried by the occupant, and stores the occupant information in the data storage unit 16.

When the detection unit 14 detects the abandonment and the user does not approve the abandonment, the communication unit 20 refers to the occupant information of the data storage unit 16 and notifies the mobile terminal of the abandoned occupant.

<C-2. Operation>
FIG. 7 is a flowchart showing a shooting process of the first image by the abandonment detection device 103. Hereinafter, the shooting process of the first image will be described with reference to FIG. 7. The flow of FIG. 7 starts, for example, between the time when the vehicle starts traveling and the time when the vehicle stops at the first parking / stopping point.

First, the occupant information registration unit 19 determines whether or not the occupant information is registered, that is, whether or not it is stored in the data storage unit 16 (step S301). If the occupant information is not registered in step S301, the occupant information registration unit 19 registers the occupant information (step S302). Specifically, the output control unit 15 causes the display device 221 to display the occupant information input screen, and the input reception unit 17 accepts the input of the occupant information from the input screen. The user inputs the contact information of the mobile terminal 23 from the input screen. The face image of each occupant may be input by the user, or the photographed image of the camera 21 may be used. The occupant information registration unit 19 associates the contact information of the mobile terminal 23 with the face image of the occupant and stores it in the data storage unit 16. As a result, the occupant information is registered.

When the occupant information has been registered in step S301, or when the registration of the occupant information is completed in step S302, the shooting process of the first image proceeds to step S303. Since the processing of steps S303 to S305 is the same as that of steps S101 to S103 of FIG. 2, description thereof will be omitted here.

FIG. 8 is a flowchart showing the abandonment detection process by the abandonment detection device 103. Hereinafter, the abandonment detection process will be described with reference to FIG. The flow of FIG. 8 starts at the timing when the engine of the vehicle starts, for example, at the parking / stopping point. Since steps S401 to S403 of FIG. 8 are the same as steps S201 to S203 of FIG. 3, their description will be omitted here.

If the detection unit 14 determines in step S403 that the abandonment has not occurred, the abandonment detection process ends. On the other hand, in step S403, when the detection unit 14 determines that the abandonment has occurred, the output control unit 15 causes the output device 22 to issue a abandonment warning (step S404).

FIG. 9 shows a left-behind warning screen displayed on the display device 221 in step S404 when the output device 22 includes the display device 221. A warning message "The occupant has been left behind. Do you want to leave as it is?" And "Yes" and "No" selection buttons are displayed. As a result, the driver recognizes that there is an occupant who has not returned to the vehicle while getting off at the parking / stopping point. The driver selects "Yes" when he / she decides that he / she can leave as it is. This selection result is input from the input receiving unit 17 to the communication unit 20, and the communication unit 20 determines that the driver has approved the abandonment (Yes in step S405). In this case, the display screen of the display device 221 transitions from the abandonment warning screen to the travel route guidance screen, and the abandonment detection process ends.

On the other hand, when the driver determines that he / she is waiting for the abandoned occupant, he / she selects “No” on the abandoned warning screen of FIG. This selection result is input from the input receiving unit 17 to the communication unit 20, and the communication unit 20 determines that the driver does not approve the leaving (No in step S405). The communication unit 20 refers to the occupant information of the data storage unit 16 to acquire the contact information of the abandoned occupant's mobile terminal 23, and notifies the mobile terminal 23 that the abandonment has occurred (step S406). .. On the mobile terminal 23 that has received this notification, for example, a display such as "Abandonment has occurred. Please return to the car immediately." Is displayed. As a result, the abandoned occupant recognizes that he / she is left behind and returns to the car to eliminate the abandonment.

On the warning screen of FIG. 9, a warning wording and a selection button were displayed. Information that identifies the abandoned occupant may be added here. For example, the face image of the left-behind occupant stored as occupant information in the data storage unit 16 may be displayed on the warning screen. Further, instead of the face image, a caricature, a character, or the like that can identify the occupant may be used. In this case, it is assumed that these pieces of information are stored in advance in the data storage unit 16 as occupant information.

<C-3. Effect>
In addition to the configuration of the second embodiment, the abandonment detection device 103 of the third embodiment includes an occupant information registration unit 19 for registering the face image of the occupant of the vehicle and the information of the terminal possessed by the occupant in association with each other, and the occupant. The information registration unit 19 includes a communication unit 20 for transmitting information to the effect that the abandonment has occurred to the terminal associated with the face image of the abandoned occupant. Therefore, according to the abandonment detection device 103, the abandonment can be eliminated by notifying the abandoned occupant that the abandonment has occurred and urging the vehicle to return to the vehicle.

<D. Hardware configuration>
FIG. 10 is a diagram showing a hardware configuration of the abandonment detection device 103. Image acquisition unit 11, position acquisition unit 12, determination unit 13, detection unit 14, output control unit 15, data storage unit 16, route search unit 18, and occupant information registration unit 19 (hereinafter, "image acquisition") in the abandonment detection device 103. (Referred to as "part 11 and the like") is realized by the processing circuit 81. That is, the processing circuit 81 includes an image acquisition unit 11, a position acquisition unit 12, a determination unit 13, a detection unit 14, an output control unit 15, a data storage unit 16, a route search unit 18, and an occupant information registration unit 19. The input receiving unit 17 is realized by the input interface 84. The input interface 84 is a touch pad, a mechanical switch, or the like. The communication unit 20 is realized by the communication interface 85.

Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in the memory may be applied. The processor is, for example, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.

When the processing circuit 81 is dedicated hardware, the processing circuit 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable). Gate Array), or a combination of these. Each function of each part such as the image acquisition unit 11 may be realized by a plurality of processing circuits 81, or the functions of each part may be collectively realized by one processing circuit.

When the processing circuit 81 is a processor, the functions of the image acquisition unit 11 and the like are realized by a combination of software and the like (software, firmware or software and firmware). Software and the like are described as programs and stored in memory. As shown in FIG. 11, the processor 82 applied to the processing circuit 81 realizes the functions of each part by reading and executing the program stored in the memory 83. That is, the abandonment detection device is an attribute of the rear road, which is a road on which the vehicle may travel after the parking / stopping point at least one of the parking / stopping points of the vehicle when executed by the processing circuit 81. Alternatively, the first image is an image taken from the step of determining the abandonment detection point based on the state and the camera that photographs the interior of the vehicle from the start of the vehicle running to the time when the occupant disembarks from the vehicle at the abandonment detection point. , The step to acquire the image taken when the vehicle departs from the abandonment detection point as the second image, and the occupant shown in the first image but not in the second image, the vehicle at the abandonment detection point Stores a program that will be executed as a result of the step of determining that the occupant is not abandoned after getting off the vehicle and the step of outputting the information that there is an abandoned occupant to the output device mounted on the vehicle. A memory 83 is provided for this purpose. In other words, it can be said that this program causes the computer to execute the procedure or method of the image acquisition unit 11 or the like. Here, the memory 83 includes, for example, non-volatile such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). Or with volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disk) and its drive device, etc., or any storage medium used in the future. There may be.

The configuration in which each function of the image acquisition unit 11 and the like is realized by either hardware or software has been described above. However, the present invention is not limited to this, and a configuration may be configured in which a part of the image acquisition unit 11 or the like is realized by dedicated hardware and another part is realized by software or the like. For example, the image acquisition unit 11 realizes its function by a processing circuit as dedicated hardware, and other than that, the processing circuit 81 as a processor 82 reads and executes a program stored in the memory 83. It is possible to realize the function.

As described above, the processing circuit can realize each of the above-mentioned functions by hardware, software, or a combination thereof. Although the data storage unit 16 is composed of the memory 83, they may be composed of a single memory 83 or each of them may be composed of individual memories.

The abandonment detection device 101-103 is, for example, an in-vehicle device, but is installed in an in-vehicle device, a PND (Portable Navigation Device), a communication terminal (for example, a mobile terminal such as a mobile phone, a smartphone, and a tablet), and the like. It can also be applied to a system constructed as a system by appropriately combining application functions and servers. In this case, each function or each component of the abandonment detection device 101-103 described above may be distributed and arranged in each device for constructing the system, or may be arranged in a concentrated manner in any of the devices. May be good. FIG. 12 is a configuration example of the abandonment detection device 101 by the vehicle 31 and the server 32. In this example, in addition to the camera 21 and the output device 22, the image acquisition unit 11, the position acquisition unit 12, and the output control unit 15 are arranged on the vehicle 31, and the determination unit 13, the detection unit 14, and the data storage unit 16 are located on the server 32. Have been placed.

In the present invention, each embodiment and each modification can be freely combined, and each embodiment and each modification can be appropriately modified or omitted within the scope of the invention. Although the present invention has been described in detail, the above description is exemplary in all embodiments and the present invention is not limited thereto. A myriad of variants not illustrated can be envisioned without departing from the scope of the invention.

11 Image acquisition unit, 12 Position acquisition unit, 13 Determination unit, 14 Detection unit, 15 Output control unit, 16 Data storage unit, 17 Input reception unit, 18 Route search unit, 19 Crew information registration unit, 20 Communication unit, 21 Camera , 22 output device, 23 mobile terminal, 31 vehicle, 32 server, 81 processing circuit, 82 processor, 83 memory, 84 input interface, 85 communication interface, 221 display device, 222 speaker.

Claims (14)

A determination unit that determines at least one of the parking / stopping points of the vehicle as an abandonment detection point based on the attribute or condition of the rear road, which is a road on which the vehicle may travel after the parking / stopping point.
The first image is an image taken from the camera that photographs the interior of the vehicle until the occupant disembarks from the vehicle at the abandonment detection point after the vehicle starts traveling, and the vehicle is from the abandonment detection point. The image acquisition unit that acquires the image taken at the time of departure as the second image, and
A detection unit that detects an occupant that is shown in the first image but is not shown in the second image as an abandoned occupant who has not boarded after getting off the vehicle at the abandonment detection point.
It includes an output control unit that outputs information to the effect that there is an abandoned occupant to an output device mounted on the vehicle.
Abandoned detection device. An information acquisition unit that acquires information on the destination of the vehicle, and
A route search unit for searching a travel route to the destination of the vehicle is further provided.
The rear road is a road on which the vehicle travels along the travel route after the parking / stopping point.
The abandonment detection device according to claim 1. The attributes of the rear road include the distinction between highways and general roads.
The abandonment detection device according to claim 1 or 2. The attributes of the rear road include the road width.
The abandonment detection device according to any one of claims 1 to 3. The attributes of the rear road include the presence or absence of a median strip.
The abandonment detection device according to any one of claims 1 to 4. The attributes of the rear road include the slope of the road.
The abandonment detection device according to any one of claims 1 to 5. The attributes of the rear road include the road density in the area to which the rear road belongs.
The abandonment detection device according to any one of claims 1 to 6. The condition of the rear road includes the traffic volume on the rear road.
The abandonment detection device according to any one of claims 1 to 7. The condition of the rear road includes the degree of congestion on the rear road.
The abandonment detection device according to any one of claims 1 to 7. The state of the rear road includes the presence or absence of construction work on the rear road.
The abandonment detection device according to any one of claims 1 to 7. An information acquisition unit that acquires information on the waypoints and destinations of the vehicle,
A route search unit for searching a travel route of the vehicle reaching the destination via the waypoint is further provided.
The abandonment detection point is any of the waypoints and the destinations, and is the parking / stopping point where the distance to the nearest point along the traveling route is equal to or greater than the threshold value.
The abandonment detection device according to any one of claims 1 to 10. An occupant information registration unit that registers the face image of the occupant of the vehicle and the information of the terminal possessed by the occupant in association with each other.
The occupant information registration unit further includes a communication unit that transmits information to the effect that the abandonment has occurred to the terminal associated with the face image of the abandoned occupant.
The abandonment detection device according to any one of claims 1 to 11. At least one of the parking / stopping points of the vehicle is determined to be the abandonment detection point based on the attribute or condition of the rear road, which is the road on which the vehicle may travel after the parking / stopping point.
The first image is an image taken from the camera that photographs the interior of the vehicle until the occupant disembarks from the vehicle at the abandonment detection point after the vehicle starts traveling, and the vehicle is from the abandonment detection point. The images taken at the time of departure are acquired as the second images, respectively.
An occupant who appears in the first image but does not appear in the second image is detected as an abandoned occupant who has not boarded after getting off the vehicle at the abandonment detection point.
The information that there is an abandoned occupant is output to the output device mounted on the vehicle.
Abandonment detection method. Computer,
A determination unit that determines at least one of the parking / stopping points of the vehicle as an abandonment detection point based on the attribute or condition of the rear road, which is a road on which the vehicle may travel after the parking / stopping point.
The first image is an image taken from the camera that photographs the interior of the vehicle until the occupant disembarks from the vehicle at the abandonment detection point after the vehicle starts traveling, and the vehicle is from the abandonment detection point. The image acquisition unit that acquires the image taken at the time of departure as the second image, and
A detection unit that detects an occupant that is shown in the first image but is not shown in the second image as an abandoned occupant who has not boarded after getting off the vehicle at the abandonment detection point.
To operate as an output control unit that outputs information to the effect that there is an abandoned occupant to the output device mounted on the vehicle.
Abandonment detection program.

Images