JP2020166524A - Monitoring system, monitoring method, and computer program - Google Patents

Abstract

To provide a monitoring system, a monitoring method, and a computer program for accurately estimating an owner of a lost item generated inside of a vehicle.SOLUTION: A sensor information storing portion 22 stores information about an inside of a vehicle 10 detected by a lost item sensor 20. A lost item detection portion 32 detects that a lost item is generated inside of the vehicle 10. An estimation portion 34 estimates an owner of the lost item based on the information when the lost item stored in the sensor information storing portion 22 is generated when the lost item is detected by the lost item detection portion 32. A notification portion 38 notifies an outside of information in which the lost item detected by the lost item detection portion 32 and the owner estimated by the estimation portion 34 correspond with each other.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to data processing techniques, in particular to monitoring systems, monitoring methods and computer programs.

An in-vehicle device that detects a forgotten object in a vehicle by image processing of a camera image installed in the vehicle has been proposed (see, for example, Patent Document 1). In the in-vehicle device of Patent Document 1, the person sitting at the position where the forgotten item is placed is determined to be the owner of the forgotten item.

Japanese Unexamined Patent Publication No. 2012-123491

The occupant of the vehicle may get off from the door opposite to the seat on which he / she was sitting, depending on the surrounding conditions at the time of getting off. In such a case, the technique of Patent Document 1 may not be able to correctly determine the owner of the forgotten item. ..

The present disclosure has been made in view of these problems, and one purpose is to accurately estimate the owner of a lost item generated in the vehicle.

In order to solve the above problems, the monitoring system of a certain aspect of the present disclosure includes a storage unit that stores information about the inside of the vehicle detected by a predetermined sensor, and a detection unit that detects that a lost object has occurred in the vehicle. , When a lost item is detected by the detection unit, an estimation unit that estimates the owner of the lost item based on the information when the lost item is stored in the storage unit, a lost item detected by the detection unit, and an estimation unit. It is provided with a notification unit for notifying the outside of information associated with the owner estimated by.

Another aspect of the disclosure is a monitoring method. In this method, an information processing system including a storage unit that stores information about the inside of the vehicle detected by a predetermined sensor detects that a lost object has occurred in the vehicle, and when the lost object is detected, it is stored in the storage unit. The owner of the lost item is estimated based on the information when the dropped item is generated, and the information in which the detected lost item is associated with the estimated owner is notified to the outside.

It should be noted that any combination of the above components, the expression of the present disclosure, converted between a device, a computer program, a recording medium on which the computer program is recorded, a monitoring system, a vehicle equipped with the monitoring device, etc. It is effective as a mode of disclosure.

According to the present disclosure, it is possible to accurately estimate the owner of a lost item generated in the vehicle.

It is a block diagram which shows the functional block of the vehicle of 1st Example. It is a flowchart which shows the operation of the monitoring system of 1st Example. It is a flowchart which shows the detail of the lost property detection process of S14 of FIG. It is a flowchart which shows the detail of the owner identification process of S18 of FIG. It is a figure which shows the example of the lost property information. It is a figure which shows the example of the lost property information. It is a flowchart which shows the operation of the monitoring system of 2nd Example. It is a flowchart which shows the detail of the lost property detection process of S44 of FIG. It is a flowchart which shows the detail of the owner identification process of S60 of FIG. It is a figure which shows the example of the lost property information.

First, an outline will be described. In recent years, car sharing and ride sharing have become widespread. In car sharing and ride sharing, since one vehicle is used by many people, it is important to confirm what is left behind when the occupant (hereinafter, also referred to as "user") changes or when getting on and off. This is because some stranger may bring something left behind by one person.

In the above-mentioned Patent Document 1, an in-vehicle device that detects a forgotten item in a vehicle by image processing based on a camera image installed in the vehicle and determines that a person sitting at a position where the forgotten item is placed is the owner of the forgotten item. Are listed. However, the occupant of the vehicle may get off from the door opposite to the seat on which he / she was sitting, depending on the surrounding conditions at the time of getting off. In such a case, the owner of the forgotten item may not be correctly determined. If the owner of the forgotten item cannot be correctly determined, the forgotten item may be passed on to someone other than the original owner, resulting in loss or theft. In addition, privacy can be infringed by checking the contents of the forgotten item by someone other than the original owner.

Therefore, the monitoring system proposed in the examples and modifications of the present disclosure includes a storage unit that stores information about the inside of the vehicle detected by a predetermined sensor, a detection unit that detects that a lost object has occurred in the vehicle, and a detection unit. When a lost item is detected by the unit, the estimation unit estimates the owner of the lost item based on the information when the lost item is stored in the storage unit, and the estimation unit estimates the lost item detected by the detection unit. It is provided with a notification unit that notifies the outside of the information associated with the owner.

The monitoring system proposed in the examples and modifications detects the owner of the forgotten item more accurately by detecting from which person the forgotten item came from, not the sheet or position where the forgotten item is placed. Identify. As a result, it is possible to prevent the forgotten item from being passed on to a person other than the original owner, and it is possible to prevent a person other than the original owner from checking the contents of the forgotten item and invading privacy. The "lost item" in the following description can be appropriately rephrased as a "lost item".

(First Example)
The vehicle 10 is a block diagram showing a functional block of the vehicle 10 of the first embodiment. Each block shown in the block diagram of the present disclosure can be realized by an element or a mechanical device such as a CPU and a memory of a computer in terms of hardware, and can be realized by a computer program or the like in terms of software. , Draws a functional block realized by their cooperation. It is understood by those skilled in the art that these functional blocks can be realized in various forms by combining hardware and software.

The vehicle 10 includes an occupant sensor 12, a display device 14, and a monitoring system 16. The occupant sensor 12 is a sensor that detects the presence or absence of an occupant in the vehicle 10 by a known method. The occupant sensor 12 may detect the presence or absence of an occupant in the vehicle 10 by analyzing the pressure on the seat, the state of the seat bell, the analysis of the image of the interior of the vehicle 10, and the like.

The display device 14 is a device provided in the vehicle interior of the vehicle 10 and displays various information to be presented to the occupant (for example, the driver). The display device 14 may be a display unit of an IVI (In-Vehicle Infotainment system) or a car navigation device. Alternatively, the display device 14 may be a dedicated device for displaying the information output from the monitoring system 16.

The monitoring system 16 is an information processing system that monitors lost items in the vehicle 10. The monitoring system 16 includes a lost item sensor 20, a sensor information storage unit 22, a lost item storage unit 24, and a processing unit 26. The lost-and-found sensor 20 corresponds to the sensor shown in the above outline, and in the embodiment, it is an image sensor (image sensor) that images the interior of the vehicle 10. The sensor information storage unit 22 corresponds to the storage unit shown in the above outline, and in the embodiment, stores image data (each frame of the image) showing the state of the vehicle interior captured by the lost-and-found sensor 20. The lost item storage unit 24 stores information about the lost item generated by the processing unit 26.

The processing unit 26 includes a boarding / alighting detection unit 30, a lost item detection unit 32, an estimation unit 34, a generation unit 36, and a notification unit 38. A computer program including a plurality of modules corresponding to the plurality of functional blocks may be stored in the ROM of the monitoring system 16. The CPU of the monitoring system 16 may exert the functions of the plurality of functional blocks by reading the computer program into the RAM and executing the program. The processing unit 26 may be realized by one information processing device (for example, an ECU (Electronic Control Unit)), or may be realized by coordinating a plurality of information processing devices.

The boarding / alighting detection unit 30 detects when a occupant gets off the vehicle 10 based on the signal output from the occupant sensor 12.

The lost item detection unit 32 detects that a lost item has occurred in the vehicle interior of the vehicle 10. The lost item detection unit 32 may detect a lost item by a known method. For example, the lost object detection unit 32 compares the image showing the state of the vehicle interior before the occupant gets on the vehicle stored in the sensor information storage unit 22 with the image showing the state of the vehicle interior after the occupant gets off, and makes a difference. May be detected as a lost item.

In addition, the lost item detection unit 32 may detect that a lost item has occurred by a general object detection method. When an object having predetermined characteristics (shape, pattern, color, etc.) of a lost item is detected from an image in the vehicle interior, the lost item detection unit 32 may detect the object as a lost item.

Further, a plurality of pressure sensors may be installed in the vehicle interior of the vehicle 10 (for example, inside a plurality of seats). In this case, the lost item detection unit 32 may detect the presence / absence and position of the lost item in the vehicle interior based on the size and pressure to which the pressure is applied detected by the pressure sensor.

The estimation unit 34 corresponds to the estimation unit shown in the above outline. In the embodiment, when the lost item is detected by the lost item detection unit 32, the estimation unit 34 analyzes the image when the lost item is stored in the sensor information storage unit 22 and estimates the owner of the lost item.

The generation unit 36 generates lost item information in which the lost item detected by the lost item detection unit 32 and the owner estimated by the estimation unit 34 are associated with each other, and stores the lost item information in the lost item storage unit 24.

The notification unit 38 notifies the external device of the monitoring system 16 of the lost item information stored in the lost item storage unit 24. In the embodiment, the notification unit 38 transmits the lost item information to the display device 14 and causes the display device 14 to display the lost item information.

The operation of the monitoring system 16 with the above configuration will be described.
FIG. 2 is a flowchart showing the operation of the monitoring system 16 of the first embodiment. The monitoring system 16 starts the monitoring process when a predetermined start condition such as a user's boarding, an engine on, or a specific operation input by the user is satisfied (S10). Specifically, the lost-and-found sensor 20 starts a process of capturing an image of the inside of the vehicle, and stores video data indicating the state of the inside of the vehicle in the sensor information storage unit 22. The boarding / alighting detection unit 30 starts a process of detecting the user getting off based on the detection result of the occupant sensor 12. While the user's disembarkation is not detected (N in S12), images showing the state of the vehicle interior are sequentially accumulated in the sensor information storage unit 22.

When the boarding / alighting detection unit 30 detects that the user has disembarked (Y in S12), the lost-and-found detection unit 32 executes a lost-and-found detection process described later (S14). When a lost item is detected in S14 (Y in S16), the owner identification process described later is executed (S18). The notification unit 38 transmits the lost item information generated in the owner identification process and stored in the lost item storage unit 24 to the display device 14 to display it (S20). If no lost item is detected in S14 (N in S16), the processes of S18 and S20 are skipped.

FIG. 3 is a flowchart showing details of the lost property detection process in S14 of FIG. The lost-and-found detection unit 32 acquires an image (t) from the sensor information storage unit 22 (S22). The image (t) is an image (or a frame of an image) captured at the time point t. The initial value of t is the time when the user's disembarkation is detected by the boarding / alighting detection unit 30. The lost item detection unit 32 uses the image (t) to determine whether or not the lost item is reflected in the image (t). As described above, the lost item detection unit 32 may determine the presence or absence of a lost item by a known method. For example, when an object (wallet, smartphone, etc.) that is not shown in the image when the user is on board is shown in the image (t), the lost item detection unit 32 may determine that the object is a lost item.

When a lost item is detected from the image (t) (Y in S24), the lost item detection unit 32 subtracts t by one unit (for example, one frame) (S26) and returns to S22. For example, the lost item detection unit 32 acquires an image captured immediately before the image in which the lost item is detected, and determines again whether or not the lost item is reflected in the image. In this way, the lost item detection unit 32 traces the image stored in the sensor information storage unit 22 from the time when the occupant gets off to the image in which the lost item is not detected.

If no lost item is detected from the image (t) (N in S24), if no lost item is detected even once (N in S28), the flow of this figure ends. On the other hand, if the lost item is detected at least once (Y in S28), the lost item detection unit 32 sets the lost item initial detection time to t + Δ (S30). That is, the lost-and-found detection unit 32 determines the time following the time t when the lost-and-found object is no longer detected (that is, t + Δ) as the first lost-and-found detection time. Further, the lost item detection unit 32 refers to the image (t + Δ) stored in the sensor information storage unit 22 and identifies the position (for example, the seat position) in the vehicle interior where the lost item first occurs (S32).

FIG. 4 is a flowchart showing details of the owner identification process of S18 of FIG. The estimation unit 34 acquires an image (t + Δ) from the sensor information storage unit 22 (S34). The image (t + Δ) is an image taken at the time when the lost item was first detected. That is, the image (t + Δ) is an image in which the image stored in the sensor information storage unit 22 is specified retroactively from the time when the occupant gets off, and is an image in which the lost item is first reflected. The estimation unit 34 identifies the position of the person reflected in the image (t + Δ) by a known method (S36). For example, the estimation unit 34 identifies which seat each occupant of the vehicle 10 is sitting on by image processing.

The estimation unit 34 estimates that among the occupants whose positions are specified in S36, the occupant closest to the position of the lost item in the image (t + Δ) is the owner of the lost item (S38). The generation unit 36 generates lost item information regarding the lost item detected by the lost item detection unit 32 and the owner estimated by the estimation unit 34, and records it in the lost item storage unit 24 (S39).

FIG. 5 shows an example of lost property information. The generation unit 36 may extract the owner image 42 in which the owner's face is reflected from the image (for example, the image (t + Δ)) in which the owner is estimated by the estimation unit 34. The owner image can be said to be the image of the owner of the lost item. The owner image may be an image showing the owner (in other words, the owner), and the image in which the owner is shown is predominantly shown by trimming the part other than the owner of the image in which the owner is shown. Is more preferable.

Further, the generation unit 36 may extract the lost item image 44 in which the lost item is reflected from the image (for example, the image (t + Δ)) in which the lost item is detected by the lost item detection unit 32. The generation unit 36 may generate the lost item information 40 including the set of the owner image 42 and the lost item image 44. The notification unit 38 may display the lost item information 40 shown in FIG. 5 on the display device 14.

FIG. 6 also shows an example of lost property information. The estimation unit 34 may detect the position of the owner from the image (for example, the image (t + Δ)) in which the owner is estimated. In the embodiment, the position of the owner is the position where the owner was seated during the ride (also referred to as the seat position). The generation unit 36 may generate an owner position image 46 showing the seating position of the owner. The owner position image 46 of FIG. 6 shows that the seating position of the owner is to the right rear of the vehicle 10. The generation unit 36 may generate the lost item information 40 including the set of the owner position image 46 and the lost item image 44. The notification unit 38 may display the lost item information 40 shown in FIG. 6 on the display device 14. As a modification, the generation unit 36 may generate an owner position image 46 showing the initial position of the lost item detected by the lost item detection unit 32.

When the monitoring system 16 of the first embodiment detects a lost item in the vehicle 10, the owner of the lost item is estimated based on the image when the lost item occurs. As a result, the owner of the lost item generated in the vehicle 10 can be accurately estimated. Further, the monitoring system 16 of the first embodiment causes the display device 14 mounted on the vehicle 10 to display information in which the lost item and its owner are associated with each other. This makes it easier for the driver or the like to understand the lost item and its owner, and it is possible to prevent the lost item from being passed on to an unrelated third party.

The present disclosure has been described above based on the first embodiment. The first embodiment is an example, and those skilled in the art understand that various modifications are possible for each component of the first embodiment or a combination of each processing process, and that such modifications are also within the scope of the present disclosure. It is about to be done.

A modified example of the first embodiment will be described. The lost-and-found detection unit 32 may trace back the image stored in the sensor information storage unit 22 from the time when the occupant gets off, and even if the lost-and-found object is no longer detected from the image, the image for a predetermined time may be further traced back. Here, let t be the last time to go back. In the process of tracing back the image, if the image in which the lost object is not reflected is changed to the image in which the dropped object is not reflected and then the image in which the dropped object is not reflected continues for a predetermined time (α), the dropped object detection unit 32 does not display the dropped object first. The time point (t + α + Δ) following the time point (t + α) may be set as the first detection time of the dropped object. In S34 of FIG. 4, the estimation unit 34 specifies the image (image (t + α + Δ)) at the next time after the image (image (t + α)) in which the lost item is not reflected first, as the image in which the lost item is first reflected. May be good.

According to this modification, it is possible to avoid setting the momentary interruption point of the lost item detection as the first detection time. As a result, even if the lost item is blocked from the lost item sensor 20 for a short time, the time when the lost item actually occurs can be specified, and the estimation accuracy of the owner of the lost item can be further improved. The predetermined time (α) may be set to an appropriate value for excluding the momentary interruption of the lost item detection by the knowledge of the developer or the experiment using the monitoring system 16.

(Second Example)
The functional block of the monitoring system 16 of the second embodiment is the same as the functional block of the monitoring system 16 of the first embodiment shown in FIG. Hereinafter, the contents described in the first embodiment will be omitted again as appropriate, and the differences from the first embodiment will be mainly described.

The boarding / alighting detection unit 30 determines whether or not the occupant of the vehicle 10 is about to get off. For example, the boarding / alighting detection unit 30 may be connected to the navigation system of the vehicle 10 and obtain the distance or time to reach the destination of the occupant from the navigation system. The boarding / alighting detection unit 30 may determine that it is immediately before getting off when the distance or time to reach the destination is less than a predetermined threshold value. Further, when the boarding / alighting detection unit 30 receives a signal indicating that the parking brake of the vehicle 10 has been activated via the vehicle-mounted network, it may determine that the vehicle is about to get off.

The lost item detection unit 32 detects that a lost item has occurred in the vehicle interior of the vehicle 10. The lost item detection unit 32 may detect a lost item by a known method. For example, the lost object detection unit 32 compares the image of the vehicle interior before the occupant boarding stored in the sensor information storage unit 22 with the latest image of the vehicle interior captured while the occupant is riding, and determines an object to be a difference. It may be detected as a lost item. Further, the lost-and-found detection unit 32 may detect the lost-and-found object by a general object detection method based on the latest image of the vehicle interior.

The lost item storage unit 24 stores a list of lost item candidates. When a lost item is detected by the lost item detection unit 32, the generation unit 36 records information about the lost item in the candidate list. Information about lost items is also referred to as "lost item information" below, and includes images and positions of lost items (positions in the vehicle interior). The image of the lost item may be an image showing the lost item, and it is more preferable that the image showing the lost item is predominantly shown by trimming a portion of the image other than the lost item.

The estimation unit 34 estimates the occupant closest to the position of the lost item as the owner of the lost item in the image when the lost item is first detected by the lost item detection unit 32. The notification unit 38 transmits the lost item information to the display device 14 and displays it when the predetermined condition regarding the disembarkation of the occupant is satisfied, and when it is determined that the occupant is about to disembark in the second embodiment.

The operation of the monitoring system 16 with the above configuration will be described.
FIG. 7 is a flowchart showing the operation of the monitoring system 16 of the second embodiment. The monitoring system 16 starts the monitoring process when the same predetermined start conditions as those in the first embodiment are satisfied (S40). If it is not detected by the boarding / alighting detection unit 30 that the occupant is about to get off (N in S42), the lost-and-found detection process described later is executed (S44).

When the boarding / alighting detection unit 30 detects that the occupant is about to get off (Y in S42) and the lost item is detected in S44 (Y in S46), the notification unit 38 is stored in the lost item storage unit 24. Lost and found information is transmitted to the display device 14 and displayed (S48). If no lost item is detected in S44 (N in S46), the process in S48 is skipped.

FIG. 8 is a flowchart showing details of the lost property detection process of S44 of FIG. The lost-and-found detection unit 32 acquires an image (t), which is the latest image captured by the lost-and-found sensor 20, from the sensor information storage unit 22 (S50). Based on the image (t), the lost-and-found detection unit 32 detects when there is a lost-and-found object in the vehicle interior. When a lost item is detected from the image (t) (Y in S52), the lost item detection unit 32 refers to the candidate list stored in the lost item storage unit 24, and is the information about the lost item detected this time registered in the candidate list? It is determined whether or not (S54).

If the lost item detected this time is not registered in the candidate list (N in S56), the lost item detection unit 32 identifies the position in the vehicle interior where the lost item has occurred (S58) based on the image (t). The estimation unit 34 executes the owner identification process described later (S60). The generation unit 36 generates lost item information in which the lost item (type, image, etc.) detected by the lost item detection unit 32, its position, and the owner estimated by the estimation unit 34 are associated with each other. The generation unit 36 records the lost item information in the candidate list of the lost item storage unit 24 (S62). If the lost item is not detected from the image (t) (N in S52) or the lost item detected this time is registered in the candidate list (Y in S56), the subsequent processing is skipped.

FIG. 9 is a flowchart showing details of the owner identification process of S60 of FIG. The estimation unit 34 acquires the image (t) from the sensor information storage unit 22 (S70). The estimation unit 34 identifies the position of the person appearing in the image (t) by a known method (S72). The estimation unit 34 estimates that among the occupants whose positions are specified in S72, the occupant closest to the position of the lost item in the image (t) is the owner of the lost item (S74).

The monitoring system 16 of the second embodiment has the same effect as the monitoring system 16 of the first embodiment. That is, when the monitoring system 16 of the second embodiment detects a lost item in the vehicle 10, the owner of the lost item is estimated based on the image when the lost item occurs. As a result, the owner of the lost item generated in the vehicle 10 can be accurately estimated. Further, the monitoring system 16 of the second embodiment causes the display device 14 mounted on the vehicle 10 to display information in which the lost item and its owner are associated with each other. This makes it easier for the driver or the like to understand the lost item and its owner, and it is possible to prevent the lost item from being passed on to an unrelated third party. Furthermore, according to the monitoring system 16 of the second embodiment, it is possible to notify the lost item before the occupant who owns the lost item gets off the vehicle, so that it becomes easy to prevent the lost item from being left behind. ..

The present disclosure has been described above based on the second embodiment. The second embodiment is an example, and those skilled in the art understand that various modifications are possible for each component of the second embodiment or a combination of each processing process, and that such modifications are also within the scope of the present disclosure. It is about to be done.

As a modification applicable to both the first embodiment and the second embodiment, variations of the method of notifying information on lost items will be described. The notification methods shown below can be in any combination with the notification methods described in the examples. It is also possible to appropriately combine a plurality of notification methods shown below.

(1) When the monitoring system 16 can identify the individual user in the vehicle interior, the notification unit 38 may notify the lost item information to the user who is presumed to be the owner of the lost item. For example, when the user's face photograph is registered in advance in association with the user account, the notification unit 38 determines that the owner of the lost item estimated by the estimation unit 34 is the owner reflected in the image captured by the lost item sensor 20. The contact information of the owner (for example, an e-mail address or identification information of the user terminal) may be specified by collating with the face photograph of each user registered in advance. The notification unit 38 may transmit and display the lost item information on the terminal of the owner of the lost item based on the specified contact information.

In order to show that the person boarding the vehicle 10 is a legitimate user, the user holds a user terminal (smartphone or the like) over the authentication device provided in the vehicle 10 to provide a lost object monitoring service using the monitoring system 16. You may log in. The monitoring system 16 may store the identification information of the user (that is, the occupant of the vehicle 10) who has logged in to the lost property monitoring service. Further, the monitoring system 16 stores a face image of a legitimate user registered in the lost property monitoring service, or makes it possible to acquire the face image from the server by communication. In this case, it is not necessary to collate the face image of the occupant captured by the drop sensor 20 of the vehicle 10 with the face images of all the users registered in the drop monitoring service, and the registered face image of the occupant of the vehicle 10 is not required. You only have to match with. According to this aspect, the target of image matching can be narrowed down to the occupants, and the load and time of the process of identifying the owner of the lost item can be reduced.

(2) The notification unit 38 may transmit and store the lost item information to the device of the management center connected to the vehicle 10 via the communication network. This makes it easier to prevent the driver of the vehicle 10 and the like from stealing lost items. In addition, it becomes easy for the management center to respond to inquiries from the owner of the lost item.

(3) The notification unit 38 may notify the lost item information to the user himself immediately before the user gets off the vehicle. For example, the generation unit 36 may include the owner's position information (seat position, etc.) estimated by the estimation unit 34 in the lost item information. The notification unit 38 may transmit voice data notifying the position of the owner, such as "Customer in XX seat, have you forgotten?", To the speaker in the vehicle 10 and output it. At the same time, the notification unit 38 may display an image of the lost item on the navigation screen of the vehicle 10 or the monitor for the rear seat of the vehicle 10.

(4) When the user who gets on the vehicle 10 can be identified in the ride sharing service or the like, the notification unit 38 may notify the driver or the device of the service provider of the lost item information after the user gets off or immediately before getting off. .. Further, the notification unit 38 may register a set of a user's account (for example, an account issued by a service provider) and an image of a lost item in an external server.

(5) When the user who gets on the vehicle 10 can be identified by the ride sharing service or the like, the notification unit 38 may notify the user's terminal of the lost item information after the user gets off or immediately before getting off. For example, the notification unit 38 may send lost item information including an image of the lost item to the user terminal by e-mail. Further, the notification unit 38 may display the lost item information on the service screen that can be viewed on the user terminal by registering the lost item information in the server of the service provider.

In addition, the generation unit 36 may specify the type of lost item (wallet, smartphone, etc.) by a known image analysis process (object detection process) and generate lost item information including the type of lost item. The notification unit 38 is lost item information including only the type of lost item, and may notify the user terminal of lost item information that does not include an image of the lost item, for example. This lost item information may be, for example, the message "Did you forget your wallet in your car?" This makes it possible to prevent a third party other than the owner of the dropper from impersonating the owner.

Another modification will be described. The estimation unit 34 traces the image stored in the sensor information storage unit 22 back from the time when the lost item is detected by the lost item detection unit 32, in other words, traces back from the image in which the lost item is detected, and determines the owner of the lost item. It may be specified. For example, the estimation unit 34 may specify the occupant at the position closest to the lost item as the owner when the lost item occurs, as in the first embodiment. The estimation unit 34 may further specify the current position of the specified owner in the vehicle interior by a known technique such as face recognition, and specify the current position as the position of the owner. The generation unit 36 may generate lost item information including the current position of the owner, and the notification unit 38 may display the lost item information on the display device 14. According to this modification, even if the seating position of the owner of the lost item changes after the lost item occurs, the true owner of the lost item can be notified.

Yet another modification applicable to both the first embodiment and the second embodiment will be described. In the above embodiment, the lost item was detected and the owner was estimated based on the image of the inside of the vehicle. As a modification, a lost item may be detected and its owner may be estimated based on the pressure value detected in the vehicle.

In this modification, the lost-and-found sensor 20 of the monitoring system 16 is a pressure sensor arranged on the interior surface of the vehicle 10 (in other words, the surface of the living room of the vehicle 10, for example, a seat or a floor). A plurality of pressure sensors may be arranged on a plurality of surfaces or a plurality of regions in the interior of the vehicle 10. This pressure sensor detects the pressure applied to the surface of the interior of the vehicle 10 and is arranged, and outputs pressure information indicating the detection result. The sensor information storage unit 22 of the monitoring system 16 sequentially stores the pressure information sequentially output from the pressure sensor as the lost item sensor 20.

The lost item detection unit 32 detects the presence / absence and position of a lost item in the vehicle 10 based on the size and pressure to which the pressure detected by the lost item sensor 20 indicates, which is indicated by the pressure information stored in the sensor information storage unit 22. .. When a lost item is detected by the lost item detection unit 32, the estimation unit estimates the owner of the lost item based on the pressure information stored in the sensor information storage unit 22 when the lost item occurs.

For example, when this modification is applied to the first embodiment, the image (t) is replaced with the pressure information (t) in the process shown in FIG. The lost-and-found detection unit 32 may detect the time and position of the lost item by tracing back the pressure information stored in the sensor information storage unit 22 from the time when the occupant disembarked. That is, the lost-and-found detection unit 32 may go back to the time when the lost-and-found object is not detected based on the pressure, and determine the time following the time when the lost-and-found object is first detected.

Further, in this modification, the image (t + Δ) is replaced with the pressure information (t + Δ) in the process shown in FIG. The estimation unit 34 can estimate the position of each occupant based on the pressure information stored in the sensor information storage unit 22. The estimation unit 34 may estimate the position of each occupant at the time when the lost item is first detected. The estimation unit 34 may estimate the owner of the lost item as the occupant closest to the lost item position detected by the lost item detection unit 32 at the lost item initial detection time detected by the lost item detection unit 32.

Further, when this modification is applied to the second embodiment, the image (t) is replaced with the pressure information (t) in the process shown in FIG. The lost-and-found detection unit 32 may acquire the latest pressure information from the sensor information storage unit 22, and detect if there is a lost-and-found object in the vehicle interior based on the pressure information. Further, in this modification, the image (t) shown in FIG. 9 is replaced with the pressure information (t). The estimation unit 34 may acquire the latest pressure information from the sensor information storage unit 22 and estimate the position of each occupant based on the pressure information. The estimation unit 34 may estimate the owner of the lost item as the occupant closest to the lost item position detected by the lost item detection unit 32.

FIG. 10 shows an example of lost property information in this modified example. The generation unit 36 generates an owner position image 46 showing the seating position of the owner of the lost item estimated by the estimation unit 34. At the same time, the generation unit 36 generates a lost item position image 48 showing the lost item position detected by the lost item detection unit 32. The generation unit 36 generates lost item information 40 including a set of the owner position image 46 and the lost item position image 48. The notification unit 38 causes the display device 14 to display the lost item information 40 shown in FIG.

The monitoring system 16 of this modified example also has the same effect as the monitoring system 16 of the first embodiment and the second embodiment. Further, the monitoring system 16 of the present modification stores pressure information, but since the pressure information is generally smaller in data size than the image data, the amount of memory used can be reduced. The monitoring system 16 may have a configuration in which a lost item detection process and an owner identification process based on an image and a lost item detection process and an owner identification process based on pressure information are combined. For example, while detecting a lost item based on an image, the owner of the lost item may be estimated based on pressure information. According to this aspect, it is not necessary to accumulate image data having a relatively large data size over the entire traveling period, and the amount of memory used can be reduced.

Any combination of the examples and modifications described above is also useful as an embodiment of the present disclosure. The new embodiments resulting from the combination have the effects of the combined examples and the modifications. It is also understood by those skilled in the art that the functions to be fulfilled by each of the constituent requirements described in the claims are realized by a single component or a cooperation thereof shown in the examples and modifications.

The techniques described in the examples and modifications may be specified by the following items.
[Item 1]
A storage unit that stores information about the inside of the vehicle detected by a predetermined sensor,
A detector that detects that a lost item has occurred in the vehicle,
When a lost item is detected by the detection unit, an estimation unit that estimates the owner of the lost item based on the information when the lost item is generated stored in the storage unit.
A notification unit that notifies the outside of information relating the lost item detected by the detection unit and the owner estimated by the estimation unit.
Surveillance system with.
[Item 2]
The notification unit displays information in which the lost item and the owner are associated with each other on a display device mounted on the vehicle.
The monitoring system according to item 1.
[Item 3]
The storage unit stores an image in the vehicle captured by the image pickup device as the predetermined sensor, and stores the image in the vehicle.
The estimation unit estimates the owner of the lost item based on the image when the lost item is generated stored in the storage unit.
The monitoring system according to item 1 or 2.
[Item 4]
The detection unit detects the lost item from the image stored in the storage unit.
The notification unit displays a set of an image of the lost item and an image of the owner as information in which the lost item and the owner are associated with each other.
The monitoring system according to item 3.
[Item 5]
The detection unit detects the position of the lost item from the image stored in the storage unit.
The estimation unit detects the position of the owner from the image stored in the storage unit, and determines the position of the owner.
The notification unit displays a set of an image of the lost item and an image indicating the position of the owner as information in which the lost item and the owner are associated with each other.
The monitoring system according to item 3.
[Item 6]
The estimation unit identifies the owner of the image stored in the storage unit retroactively from the time when the lost item is detected, further specifies the current position of the owner, and uses the current position as the position of the owner. Identify,
The monitoring system according to item 5.
[Item 7]
The position of the owner is the position where the owner was seated while riding.
The monitoring system according to item 5.
[Item 8]
The detection unit detects that a lost item has occurred based on the image when the occupant gets off, and
The estimation unit estimates the occupant closest to the lost item as the owner of the lost item in the image in which the lost item is first reflected, which is identified retroactively from the time when the occupant disembarked from the image stored in the storage unit. ,
The monitoring system according to any one of items 3 to 7.
[Item 9]
When the image stored in the storage unit is traced back from the time when the occupant got off and the image in which the lost item is not reflected continues for a predetermined time, the image at the time following the image in which the lost item is not displayed first is the lost item. Is identified as the first image shown,
The monitoring system according to item 8.
[Item 10]
The estimation unit estimates the occupant closest to the lost item as the owner of the lost item in the image when the lost item is first detected by the detection unit.
When the predetermined condition regarding the disembarkation of the occupant is satisfied, the notification unit notifies the information corresponding to the lost item and the owner.
The monitoring system according to any one of items 3 to 7.
[Item 11]
The storage unit stores pressure information applied to the interior surface of the vehicle detected by the pressure sensor as the predetermined sensor, and stores the pressure information.
The detection unit detects the lost item based on the pressure information stored in the storage unit.
The estimation unit estimates the owner of the lost item based on the pressure information when the lost item is generated stored in the storage unit.
The monitoring system according to item 1 or 2.
[Item 12]
An information processing system equipped with a storage unit that stores information about the inside of the vehicle detected by a predetermined sensor
Detecting that a lost item has occurred in the vehicle,
When the lost item is detected, the owner of the lost item is estimated based on the information stored in the storage unit when the lost item occurs.
Notify the outside of information that associates the detected lost item with the estimated owner.
Monitoring method.
[Item 13]
An information processing system equipped with a storage unit that stores information about the inside of the vehicle detected by a predetermined sensor.
Detecting that a lost item has occurred in the vehicle,
When the lost item is detected, the owner of the lost item is estimated based on the information stored in the storage unit when the lost item occurs.
Notify the outside of information that associates the detected lost item with the estimated owner.
A computer program to do things.

10 Vehicles, 14 Display devices, 16 Monitoring systems, 20 Lost and found sensors, 22 Sensor information storage, 32 Lost and found detectors, 34 Estimators, 38 Notifications.

Claims (13)

A storage unit that stores information about the inside of the vehicle detected by a predetermined sensor,
A detector that detects that a lost item has occurred in the vehicle,
When a lost item is detected by the detection unit, an estimation unit that estimates the owner of the lost item based on the information when the lost item is generated stored in the storage unit.
A notification unit that notifies the outside of information relating the lost item detected by the detection unit and the owner estimated by the estimation unit.
Surveillance system with. The notification unit displays information in which the lost item and the owner are associated with each other on a display device mounted on the vehicle.
The monitoring system according to claim 1. The storage unit stores an image in the vehicle captured by the image pickup device as the predetermined sensor, and stores the image in the vehicle.
The estimation unit estimates the owner of the lost item based on the image when the lost item is generated stored in the storage unit.
The monitoring system according to claim 1 or 2. The detection unit detects the lost item from the image stored in the storage unit.
The notification unit displays a set of an image of the lost item and an image of the owner as information in which the lost item and the owner are associated with each other.
The monitoring system according to claim 3. The detection unit detects the position of the lost item from the image stored in the storage unit.
The estimation unit detects the position of the owner from the image stored in the storage unit, and determines the position of the owner.
The notification unit displays a set of an image of the lost item and an image indicating the position of the owner as information in which the lost item and the owner are associated with each other.
The monitoring system according to claim 3. The estimation unit identifies the owner of the image stored in the storage unit retroactively from the time when the lost item is detected, further specifies the current position of the owner, and uses the current position as the position of the owner. Identify,
The monitoring system according to claim 5. The position of the owner is the position where the owner was seated while riding.
The monitoring system according to claim 5. The detection unit detects that a lost item has occurred based on the image when the occupant gets off, and
The estimation unit estimates the occupant closest to the lost item as the owner of the lost item in the image in which the lost item is first reflected, which is identified retroactively from the time when the occupant disembarked from the image stored in the storage unit. ,
The monitoring system according to any one of claims 3 to 7. When the image stored in the storage unit is traced back from the time when the occupant got off and the image in which the lost item is not reflected continues for a predetermined time, the image at the time following the image in which the lost item is not displayed first is the lost item. Is identified as the first image shown,
The monitoring system according to claim 8. The estimation unit estimates the occupant closest to the lost item as the owner of the lost item in the image when the lost item is first detected by the detection unit.
When the predetermined condition regarding the disembarkation of the occupant is satisfied, the notification unit notifies the information corresponding to the lost item and the owner.
The monitoring system according to any one of claims 3 to 7. The storage unit stores pressure information applied to the interior surface of the vehicle detected by the pressure sensor as the predetermined sensor, and stores the pressure information.
The detection unit detects the lost item based on the pressure information stored in the storage unit.
The estimation unit estimates the owner of the lost item based on the pressure information when the lost item is generated stored in the storage unit.
The monitoring system according to claim 1 or 2. An information processing system equipped with a storage unit that stores information about the inside of the vehicle detected by a predetermined sensor
Detecting that a lost item has occurred in the vehicle,
When the lost item is detected, the owner of the lost item is estimated based on the information stored in the storage unit when the lost item occurs.
Notify the outside of information that associates the detected lost item with the estimated owner.
Monitoring method. An information processing system equipped with a storage unit that stores information about the inside of the vehicle detected by a predetermined sensor.
Detecting that a lost item has occurred in the vehicle,
When the lost item is detected, the owner of the lost item is estimated based on the information stored in the storage unit when the lost item occurs.
Notify the outside of information that associates the detected lost item with the estimated owner.
A computer program to do things.

Images