JP2024039511A - Information processing device, information processing program, and information processing method - Google Patents

Abstract

[Problem] To reduce the workload of checking for remaining objects or people remaining inside a railway vehicle. [Solution] The information processing device according to the present application includes an acquisition unit that acquires analysis information related to the analysis results of image analysis of a vehicle image that is an image corresponding to the interior of the railway vehicle, and a generation unit that generates notification information that associates vehicle identification information capable of identifying the railway vehicle with remaining object information related to remaining objects or people remaining inside the railway vehicle based on the analysis information. [Selected Figure] Figure 3

Description

The present invention relates to an information processing device, an imaging system, an information processing program, an imaging program, a railway vehicle, and an information processing method.

2. Description of the Related Art Conventionally, techniques for detecting objects and people left behind inside and outside a vehicle are known. For example, a camera is used to take pictures of the inside of the vehicle when getting on the vehicle and when getting off the vehicle, and the conditions inside the vehicle when getting on the vehicle are compared with the conditions inside when getting off the vehicle to see if there is anything left behind. There are known techniques for determining.

Japanese Patent Application Publication No. 2005-115911

However, the above-mentioned conventional technology only determines whether there is an item left behind in a vehicle such as a passenger car that has one compartment, and it is possible to reduce the work burden of checking for residual items or persons remaining inside the railway vehicle. It is not always possible to do so.

Therefore, the present disclosure provides an information processing device, an imaging system, an information processing program, an imaging program, a railway vehicle, and an information processing device that can reduce the work burden in confirming residues or persons remaining inside a railway vehicle. The purpose is to provide a processing method.

The information processing device according to the present application includes an acquisition unit that acquires analysis information regarding an analysis result of an image analysis on a vehicle image that is an image corresponding to the inside of a railway vehicle, and an information processing device that is capable of identifying the railway vehicle based on the analysis information. The railway vehicle includes a generation unit that generates notification information in which vehicle identification information is associated with residue information regarding a residue or a person remaining inside the railway vehicle.

The acquisition unit obtains, as the analysis information, information regarding a difference image obtained by extracting a difference between the correct vehicle image, which is an image corresponding to the inside of the railway vehicle in which the residue and the residual person do not exist, and the vehicle image. and the generation unit generates the notification information based on information regarding the difference image.

The acquisition unit acquires, as the analysis information, information regarding the processing result of image recognition processing using a machine learning model for the vehicle image, and the generation unit generates the notification information based on the information regarding the processing result. generate.

The generation unit generates the notification information in which the vehicle identification information is associated with the residue information, which is information indicating the presence or absence of the residue or the remaining person.

The generation unit generates the notification information in which the vehicle identification information is associated with the residue information, which is information indicating the position of the residue or the residue inside the railway vehicle.

The acquisition unit acquires the analysis information when request information regarding confirmation of an internal situation of the railway vehicle is received from a terminal device used by an administrator who manages the operation of the railway vehicle.

When the acquisition unit detects that the railway vehicle has entered an area within a first range from position coordinates corresponding to a specific position based on position information of a terminal device mounted on the railway vehicle, , obtain the analysis information.

The acquisition unit acquires the analysis information when a short-range wireless communication receiver installed at a specific position receives a signal transmitted from the short-range wireless communication transmitter mounted on the railway vehicle. get.

The particular location is located within a second range from a station or depot where the rail vehicle arrives.

The acquisition unit acquires the analysis information, which is information regarding the analysis results of image analysis on the vehicle images, which are images corresponding to different locations inside the railway vehicle, and the generation unit generates information based on the analysis information. The location identification information that can identify each of the different locations inside the railway vehicle is associated with the residue information that is information regarding the residue or the person remaining at each of the different locations inside the railway vehicle. Generate the notification information.

The vehicle image is an image taken by each of a plurality of imaging devices arranged at positions of the railway vehicle corresponding to different locations inside the railway vehicle.

The acquisition unit acquires the analysis information, which is information regarding the analysis result of image analysis on the vehicle image, which is an image corresponding to the inside of each of the plurality of connected railway vehicles, and the generation unit, based on the vehicle identification information that can identify each of the plurality of connected railway vehicles, and the residual information that is information regarding residues or residual persons remaining inside each of the plurality of connected railway vehicles. The notification information associated with the object information is generated.

The vehicle image is an image taken by each of a plurality of imaging devices arranged in each of the plurality of connected railway vehicles.

The railway vehicle further includes a notification unit that transmits the notification information to a terminal device used by an administrator who manages the operation of the railway vehicle.

The imaging system according to the present application is an imaging system provided in one or a plurality of connected railway vehicles, and includes an imaging unit that captures a vehicle image that is an image corresponding to the inside of the railway vehicle, and an imaging unit that captures a vehicle image that corresponds to the inside of the railway vehicle. and an estimating unit that estimates residue information regarding residues or persons remaining inside the railway vehicle based on the information.

The estimation unit calculates the residue information based on a difference image obtained by extracting a difference between the correct vehicle image, which is an image corresponding to the inside of the railway vehicle where the residue and the residue do not exist, and the vehicle image. Estimate.

The estimation unit estimates the residue information based on a processing result of image recognition processing using a machine learning model on the vehicle image.

The estimating unit estimates information indicating the presence or absence of the residue or the remaining person as the residue information.

The estimating unit estimates, as the residue information, information indicating a position of the residue or the remaining person inside the railway vehicle.

The imaging unit further includes a lamp or a lamp.

The information processing program according to the present application includes an acquisition procedure for acquiring analysis information regarding the analysis result of image analysis on a vehicle image that is an image corresponding to the inside of a railway vehicle, and a method for identifying the railway vehicle based on the analysis information. A computer is caused to execute a generation procedure for generating notification information in which vehicle identification information is associated with residue information regarding a residue or a person remaining inside the railway vehicle.

The information processing program according to the present application includes an imaging procedure for capturing a vehicle image that is an image corresponding to the interior of a railway vehicle, and a method for detecting residues remaining inside the railway vehicle or residues related to residual persons based on the vehicle image. A computer is caused to perform an estimation procedure for estimating information.

A railway vehicle according to the present application includes the imaging system.

An information processing method according to the present application is an information processing method executed by an information processing system including an imaging system and an information processing device, wherein the imaging system captures a vehicle image that is an image corresponding to the inside of a railway vehicle. and estimate residue information regarding residues or persons remaining inside the railway vehicle based on the vehicle image, and the information processing device includes vehicle identification information that can identify the railway vehicle, Notification information associated with the residue information is generated, and the notification information is notified to a manager who manages the operation of the railway vehicle.

According to one aspect of the embodiment, it is possible to reduce the work load in the work of confirming residues or persons remaining inside a railway vehicle.

FIG. 1 is a diagram for explaining the differences between trains and other public transportation systems. FIG. 2 is a diagram illustrating an example of arrangement of imaging devices in a railway vehicle according to an embodiment. FIG. 3 is a diagram showing an overview of information processing according to the first embodiment. FIG. 4 is a diagram illustrating a configuration example of a terminal device according to the first embodiment. FIG. 5 is a diagram illustrating a configuration example of an information processing apparatus according to the first embodiment. FIG. 6 is a diagram illustrating a configuration example of an imaging device according to the first embodiment. FIG. 7 is a diagram illustrating a configuration example of an analysis device according to the first embodiment. FIG. 8 is a diagram showing an example of an operation screen according to the first embodiment. FIG. 9 is a diagram showing an example of a notification screen according to the first embodiment. FIG. 10 is a diagram showing an overview of information processing according to the second embodiment. FIG. 11 is a diagram illustrating a configuration example of an imaging device according to the second embodiment. FIG. 12 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device.

Below, embodiments for implementing the information processing device, imaging system, information processing program, imaging program, railway vehicle, and information processing method according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. explain. Note that the information processing apparatus, imaging system, information processing program, imaging program, railway vehicle, and information processing method according to the present application are not limited by this embodiment. Further, in each of the embodiments below, the same parts are given the same reference numerals, and redundant explanations will be omitted.

[1. Introduction]
FIG. 1 is a diagram for explaining the differences between trains and other public transportation. In FIG. 1, differences between buses and trains, which are examples of other public transportation systems, will be explained. As shown in FIG. 1, a bus is generally a one-car train and includes a driver's seat and one cabin (also referred to as a guest room). Additionally, since the width of the passenger compartment of a bus is generally constant, there are no obstructions other than the seats that obstruct the view, making it easy to see through. Therefore, the bus crew can visually confirm the residue or persons remaining inside the passenger compartment of the bus in a short period of time. Note that the same applies to passenger vehicles other than buses. That is, the driver or passenger of the passenger car can visually confirm the residue or persons remaining inside the cabin of the passenger car in a short period of time.

On the other hand, a train is generally configured by connecting a crew cabin and a plurality of railway vehicles (hereinafter also referred to as vehicles). Further, generally, a passageway of a connecting portion that connects railway vehicles is designed to be narrower than the width of the railway vehicle. Therefore, when attempting to visually check an adjacent vehicle from a predetermined vehicle, a part of the field of view is obstructed by the connecting portion. Furthermore, since a train consists of multiple vehicles, train crew members are characterized by the fact that it takes time to check for residue or persons remaining inside each of the multiple vehicles. In addition, compared to other public transportation systems, although it takes time to check for residue or persons remaining on a train, the time available for turning back from the terminal station or entering the depot is very limited. It has the characteristic of being short. As described above, compared to other public transportation systems, the work burden of confirming residues or persons remaining inside railway vehicles is considered to be heavy.

Therefore, the information processing system according to the present embodiment includes an imaging system provided in one or a plurality of connected railway vehicles, and an information processing device. For example, an imaging system includes a plurality of imaging devices provided in one or a plurality of connected railway vehicles. The imaging system captures a vehicle image that is an image corresponding to the inside of a railway vehicle. The information processing device acquires analysis information regarding the analysis results of the image analysis for the vehicle image, and based on the acquired analysis information, the information processing device acquires vehicle identification information that can identify the railway vehicle and residual or residue remaining inside the railway vehicle. Generates notification information that is associated with residue information regarding the person. As a result, the information processing system can, for example, notify the administrator who manages the operation of railway vehicles of notification information, so that the administrator can confirm the residue or residual persons remaining inside each of multiple vehicles. The time required can be shortened. Therefore, the information processing system can reduce the operational burden of confirming residues or persons remaining inside the railway vehicle.

FIG. 2 is a diagram illustrating an example of arrangement of imaging devices in a railway vehicle according to an embodiment. FIG. 2 shows a top view of the interior (also referred to as a cabin) of a three-car train in which three vehicles 10 to 30 are connected by a connecting portion. Although the crew cabin is omitted in FIG. 2, for example, the crew cabin (not shown) may be connected to the vehicle 10 by a connecting portion on the left side of the vehicle 10.

The imaging device 300 is provided in one or a plurality of connected railway vehicles. Specifically, when a train is a one-car train, a plurality of imaging devices 300 are provided in one railway vehicle. Further, in the case where the train is composed of a plurality of connected railway vehicles, a plurality of imaging devices 300 are provided in each of the plurality of railway vehicles. FIG. 2 shows how eight imaging devices 300 are arranged in each of three connected vehicles 10 to 30. Furthermore, the imaging device 300 is arranged at positions corresponding to different locations inside one railway vehicle. FIG. 2 shows how eight imaging devices 300 are arranged at positions where eight different locations inside each vehicle can be photographed. In FIG. 2, the imaging system (not shown) includes 24 imaging devices 300 installed in three connected vehicles 10-30.

Further, the plurality of imaging devices 300 arranged in each vehicle are respectively arranged at positions to comprehensively image the interior space of each vehicle. For example, the plurality of imaging devices 300 arranged in each vehicle are arranged at symmetrical positions with respect to the traveling direction of the train (also referred to as the path direction of each vehicle). Further, the plurality of imaging devices 300 arranged in each vehicle are arranged at equal intervals in each vehicle. Further, the plurality of imaging devices 300 arranged in each vehicle are each arranged at a predetermined height position (including the ceiling of the passenger compartment) from the floor surface (also referred to as a passageway) of the passenger compartment of each vehicle.

From here, using the vehicle 10 shown in FIG. 2 as an example, the positions of the eight imaging devices 300 arranged in each vehicle will be described in detail. In FIG. 2, when the vehicle 10 is viewed from above, the passenger compartment of the vehicle 10 has a rectangular shape that is long with respect to the traveling direction of the train. Further, the passenger compartment of the vehicle 10 is provided with four doors D1-1 to D1-4 on the side wall corresponding to one side with respect to the traveling direction of the train, and between each of the four doors D1-1 to D1-4. Three seating seats E2-1 to E2-3 are provided. For example, the four doors D1-1 to D1-4 and the three seating seats E2-1 to E2-3 are arranged at equal intervals. In addition, the passenger compartment of the vehicle 10 is provided with four doors D1-5 to D1-8 on the side wall corresponding to the other side with respect to the train's traveling direction, and between each of the four doors D1-5 to D1-8. Three seating seats E2-4 to E2-6 are provided. For example, the four doors D1-5 to D1-8 and the three seating seats E2-4 to E2-6 are arranged at equal intervals. In FIG. 2, among the eight imaging devices 300-11 to 300-18 arranged in the vehicle 10, four imaging devices 300-11 to 300-14 are located on one side with respect to the traveling direction of the train. It is placed on the ceiling within a predetermined range from the side wall of the guest room. For example, the four imaging devices 300-11 to 300-14 are arranged at equal intervals. In addition, among the eight imaging devices 300-11 to 300-18, the remaining four imaging devices 300-15 to 300-18 are located at a predetermined distance from the side wall of the cabin on the other side with respect to the traveling direction of the train. Placed on the ceiling within range. For example, the four imaging devices 300-15 to 300-18 are arranged at equal intervals.

Note that the eight imaging devices 300-11 to 300-18 provided in the vehicle 10 can be placed at a position other than the ceiling if they are placed at a predetermined height from the floor of the passenger compartment of the vehicle 10. Good too. For example, the four imaging devices 300-11 to 300-14 are arranged at a predetermined height from the floor of the passenger compartment of the vehicle 10 on the side wall of the passenger compartment on one side with respect to the traveling direction of the train. may be done. Furthermore, the four imaging devices 300-15 to 300-18 are arranged at a predetermined height from the floor of the passenger compartment of the vehicle 10 on the side wall of the passenger compartment on the other side with respect to the traveling direction of the train. may be done.

In addition, when the plurality of imaging devices 300 arranged in each vehicle are arranged at symmetrical positions with respect to the direction of train movement, a predetermined range from the side wall of the cabin on one side with respect to the direction of movement of the train. The imaging device 300 placed inside the train is located on the other side with respect to the traveling direction of the train, in the aisle of the cabin, seating seats (including the upper and lower sides of the seats), luggage racks (including the upper side of the luggage rack), and windows. , may be placed at a position in the vehicle 10 where the door, side wall, or ceiling can be photographed.

For example, the imaging device 300-11 is arranged at a position in the vehicle 10 where it can photograph a location within a predetermined range from the door D1-8 and the seating seat E2-6 inside the vehicle 10. More specifically, the imaging device 300-11 detects a passageway inside the vehicle 10 located within a predetermined range from the door D1-8, the door D1-8, a side wall or the ceiling, and a predetermined distance from the seating seat E2-6. A passage inside the vehicle 10 located within the range of , a seating seat E2-6 (including the upper and lower sides of the seat), a luggage shelf located above the seating seat E2-6 (including the upper side of the luggage shelf), It is placed at a position in the vehicle 10 where a window, side wall, or ceiling can be photographed.

Further, the imaging device 300-12 is arranged at a position in the vehicle 10 where it can photograph a location within a predetermined range from the door D1-7, seating seat E2-6, and seating seat E2-5 inside the vehicle 10. More specifically, the imaging device 300-12 captures the interior passage of the vehicle 10 located within a predetermined range from the door D1-7, the door D1-7, the side wall or the ceiling, and the seating seat E2-6 or the seating area. A passage inside the vehicle 10 located within a predetermined range from the seat E2-5, the seating seat E2-6 or the seating seat E2-5 (including the upper and lower sides of the seat), the seating seat E2-6 or the seating seat E2 -5 is placed at a position in the vehicle 10 where the luggage rack (including the upper side of the luggage shelf), window, side wall, or ceiling can be photographed.

Further, the imaging device 300-13 is arranged at a position in the vehicle 10 where it can photograph a location within a predetermined range from the door D1-6, seating seat E2-5, and seating seat E2-4 inside the vehicle 10. More specifically, the imaging device 300-13 captures the interior passage of the vehicle 10 located within a predetermined range from the door D1-6, the door D1-6, the side wall or the ceiling, and the seating seat E2-5 or the seating area. A passage inside the vehicle 10 located within a predetermined range from the seat E2-4, the seating seat E2-5 or the seating seat E2-4 (including the upper and lower sides of the seat), the seating seat E2-5 or the seating seat E2 -4 is located at a position in the vehicle 10 where the luggage rack (including the upper side of the luggage rack), window, side wall, or ceiling located above the vehicle 10 can be photographed.

Further, the imaging device 300-14 is arranged at a position in the vehicle 10 where it can photograph a location within a predetermined range from the seating seat E2-4 and the door D1-5 inside the vehicle 10. More specifically, the imaging device 300-14 captures a passageway inside the vehicle 10 located within a predetermined range from the seating seat E2-4, the seating seat E2-4 (including the upper and lower sides of the seat), and the seating area. The luggage shelf located above the seat E2-4 (including the upper side of the luggage shelf), the window, the side wall or the ceiling, and the passage inside the vehicle 10 located within a predetermined range from the door D1-5, the door D1- 5. The vehicle 10 is placed at a position where the side wall or ceiling can be photographed.

Further, the imaging device 300-15 is arranged at a position in the vehicle 10 where it can photograph a location within a predetermined range from the door D1-4 and the seating seat E2-3 inside the vehicle 10. More specifically, the imaging device 300-15 detects a passageway inside the vehicle 10 located within a predetermined range from the door D1-4, the door D1-4, a side wall or the ceiling, and a predetermined distance from the seating seat E2-3. A passage inside the vehicle 10 located within the range of , a seating seat E2-3 (including the upper and lower sides of the seat), a luggage shelf located above the seating seat E2-3 (including the upper side of the luggage shelf), It is placed at a position in the vehicle 10 where a window, side wall, or ceiling can be photographed.

Further, the imaging device 300-16 is arranged at a position in the vehicle 10 where it can photograph a location within a predetermined range from the door D1-3, the seating seat E2-3, and the seating seat E2-2 inside the vehicle 10. More specifically, the imaging device 300-16 captures the inside passage of the vehicle 10 located within a predetermined range from the door D1-3, the door D1-3, the side wall or the ceiling, and the seating seat E2-3 or the seating area. A passage inside the vehicle 10 located within a predetermined range from the seat E2-2, the seating seat E2-3 or the seating seat E2-2 (including the upper and lower sides of the seat), the seating seat E2-3 or the seating seat E2 -2 is placed at a position in the vehicle 10 where it is possible to photograph the luggage shelf (including the upper side of the luggage shelf), window, side wall, or ceiling located at the top of the vehicle 10.

Further, the imaging device 300-17 is arranged at a position in the vehicle 10 where it can photograph a location within a predetermined range from the door D1-2, the seating seat E2-2, and the seating seat E2-1 inside the vehicle 10. More specifically, the imaging device 300-17 captures the passageway inside the vehicle 10 located within a predetermined range from the door D1-2, the door D1-2, the side wall or the ceiling, and the seating seat E2-2 or the seating area. A passage inside the vehicle 10 located within a predetermined range from the seat E2-1, the seating seat E2-2 or the seating seat E2-1 (including the upper and lower sides of the seat), the seating seat E2-2 or the seating seat E2 -1 is placed at a position in the vehicle 10 where the luggage rack (including the upper side of the luggage rack), window, side wall, or ceiling located at the top of the vehicle 10 can be photographed.

Further, the imaging device 300-18 is arranged at a position in the vehicle 10 where it can photograph a location within a predetermined range from the seating seat E2-1 and the door D1-1 inside the vehicle 10. More specifically, the imaging device 300-18 captures a passage inside the vehicle 10 located within a predetermined range from the seating seat E2-1, the seating seat E2-1 (including the upper and lower sides of the seat), and the seating area. The luggage shelf located above the seat E2-1 (including the upper side of the luggage shelf), the window, the side wall or the ceiling, the passage inside the vehicle 10 located within a predetermined range from the door D1-1, and the door D1- 1. The vehicle 10 is placed at a position where the side wall or ceiling can be photographed.

Note that when the plurality of imaging devices 300 arranged in each vehicle are arranged at symmetrical positions with respect to the direction of train movement, a predetermined distance from the side wall of the cabin on one side with respect to the direction of movement of the train is The imaging device 300 placed within the range includes a passageway of a cabin on one side with respect to the train's traveling direction, seating seats (including the upper and lower sides of the seats), luggage racks (including the upper side of the luggage rack), It may be placed at a position in the vehicle 10 where a window, door, side wall, or ceiling can be photographed.

As described above, in FIG. 2, the imaging system (not shown) has eight imaging devices 300-11 to 300-18 arranged at eight different positions inside the vehicle 10, so that there is no blind spot. Thus, the interior space of the vehicle 10 can be comprehensively photographed. In addition, the imaging system (not shown) is arranged so that eight imaging devices 300 are arranged in each of the three connected vehicles 10 to 30 so that there is no blind spot. It is possible to comprehensively photograph each interior space.

Note that, although FIG. 2 illustrates a case in which eight imaging devices 300 are arranged in each vehicle, the number of imaging devices 300 arranged in each vehicle may be nine or more, or seven or less.

[2. First embodiment]
FIG. 3 is a diagram showing an overview of information processing according to the first embodiment. As shown in FIG. 3, the information processing system 1 includes a terminal device 100, an information processing device 200, an imaging device 300, and an analysis device 400. The terminal device 100, the information processing device 200, the imaging device 300, and the analysis device 400 are connected to be able to communicate with each other by wire or wirelessly via a network (not shown).

The terminal device 100 is an information processing device used by a manager A1 (hereinafter referred to as manager A1) who manages the operation of railway vehicles. For example, the terminal device 100 is a smart device such as a smartphone or a tablet terminal, a feature phone, a PC (Personal Computer), a PDA (Personal Digital Assistant), an AV device with a communication function, a car navigation system, a smart watch, or a head-mounted display. wearable devices, smart glasses, etc. Further, the terminal device 100 may be an electronic device or the like compatible with IOT (Internet of Things).

Note that the manager who manages the operation of a railway vehicle includes a crew member of the railway vehicle (for example, a conductor, etc.) and a person who plays the role of a crew member (for example, a station staff member at a terminal station).

The information processing device 200 acquires analysis information regarding the analysis results of image analysis for a vehicle image (hereinafter also referred to as a vehicle image), which is an image corresponding to the inside of a railway vehicle, and can identify the railway vehicle based on the analysis information. The present invention is an information processing device that generates notification information that associates vehicle identification information with information about residues or persons remaining inside a railway vehicle. In addition, below, the vehicle image which is an image corresponding to the inside of each vehicle may be described as the vehicle image of each vehicle. The information processing device 200 is realized by a cloud system in which a plurality of server devices and a plurality of storage devices work together. Note that the information processing device 200 may be realized by a single server device.

The imaging device 300 is an information processing device that captures vehicle images. A plurality of imaging devices 300 are provided in one or a plurality of connected railway vehicles. The imaging device 300 also has a function of communicating with the information processing device 200 via a network (not shown). Note that the imaging device 300 may include a lamp (for example, a light emitting diode fluorescent lamp, hereinafter referred to as an "LED fluorescent lamp") or a lamp. For example, the imaging device 300 may be a security camera with an integrated LED fluorescent lamp integrated with a lamp or a lamp.

The analysis device 400 is an information processing device that estimates residue information regarding residues or persons remaining inside a railway vehicle based on vehicle images. The analysis device 400 is realized by a cloud system in which a plurality of server devices and a plurality of storage devices work together. Note that the analysis device 400 may be realized by a single server device.

Returning to the explanation of FIG. 3. In FIG. 3, the terminal device 100 receives an operation related to checking the internal situation of the railway vehicle (hereinafter also referred to as confirmation operation) from the manager A1 (step S11). When the terminal device 100 receives the confirmation operation, it transmits request information regarding confirmation of the internal situation of the railway vehicle to the information processing device 200 (step S12).

When the information processing device 200 receives request information regarding confirmation of the internal situation of the railway vehicle from the terminal device 100, the information processing device 200 transmits request information regarding a request to transmit a vehicle image to the imaging device 300 (step S13). When the imaging device 300 receives request information regarding a request to transmit a vehicle image from the information processing device 200, the imaging device 300 transmits a vehicle image corresponding to each vehicle to the information processing device 200 (step S14). When the information processing device 200 receives vehicle images corresponding to each vehicle from the imaging device 300, the information processing device 200 transmits the vehicle images corresponding to each vehicle to the analysis device 400 (step S15). In addition, the information processing device 200 analyzes the correct vehicle image, which is an image corresponding to the inside of each vehicle, taken in a state where there is no residue or person inside each vehicle, along with the vehicle image corresponding to each vehicle. It is transmitted to the device 400. In addition, below, the correct vehicle image which is an image corresponding to the inside of each vehicle may be described as the correct vehicle image of each vehicle.

Here, the term "residue" refers to things that remain inside the railway vehicle. For example, residue includes items left behind or dropped inside a rail vehicle by a passenger or crew member (also referred to as lost property). In addition, "remaining person" refers to a person who remains inside the railway vehicle. For example, residual persons include passengers or crew members remaining inside the rail vehicle.

When the analysis device 400 receives the vehicle image of each vehicle from the information processing device 200, it performs image analysis on the vehicle image of each vehicle (step S16). Specifically, the analysis device 400 generates information regarding a difference image obtained by extracting the difference between the correct vehicle image of each vehicle and the vehicle image of each vehicle. Next, the analysis device 400 estimates residue information regarding the residue remaining inside each vehicle or the person remaining based on the information regarding the difference images of each vehicle. For example, the analysis device 400 estimates, as the residue information, information indicating the presence or absence of residues or persons remaining inside each vehicle. Subsequently, the analysis device 400 transmits analysis information, which is information regarding the analysis results of the image analysis for the vehicle images of each vehicle, to the information processing device 200 (step S17). For example, the analysis device 400 transmits residue information regarding residues or persons remaining inside each vehicle as analysis information.

The information processing device 200 acquires analysis information, which is information regarding the analysis results of image analysis for vehicle images of each vehicle, from the analysis device 400. For example, the information processing device 200 acquires, as analysis information, residue information regarding residues or persons remaining inside each vehicle. Next, based on the analysis information, the information processing device 200 generates notification information that associates vehicle identification information that can identify each vehicle with residue information regarding the residue remaining inside each vehicle or the residue person. is generated (step S18). Subsequently, when the information processing device 200 generates notification information, the information processing device 200 transmits the generated notification information to the terminal device 100 (step S19).

When the terminal device 100 receives notification information from the information processing device 200, it displays the notification information on the screen (step S110).

[2-1. Terminal device configuration]
FIG. 4 is a diagram illustrating a configuration example of a terminal device according to the first embodiment. As shown in FIG. 4, the terminal device 100 includes a communication section 110, a storage section 120, a sensor section 130, an input section 140, an output section 150, and a control section 160.

(Communication Department 110)
The communication unit 110 is connected to a network by wire or wirelessly, and transmits and receives information to and from the information processing device 200. For example, the communication unit 110 is realized by a NIC (Network Interface Card), an antenna, or the like.

(Storage unit 120)
The storage unit 120 stores various information. For example, the storage unit 120 stores information regarding an operation screen for performing operations related to checking the internal situation of the railway vehicle and an application AP1 for displaying notification information received from the information processing device 200. The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk.

(Sensor section 130)
The sensor unit 130 includes various sensors mounted on or connected to the terminal device 100. Note that the connection may be a wired connection or a wireless connection. For example, the sensors may be a detection device other than the terminal device 100, such as a wearable device or a wireless device.

(Input section 140)
The input unit 140 is an input device that receives various operations from the administrator A1. For example, the input unit 140 includes buttons for inputting characters, numbers, and the like. Note that the input unit 140 may be an input/output port (I/O port), a USB (Universal Serial Bus) port, or the like. Further, when the output section 150 is a touch panel display, a part of the output section 150 functions as the input section 140. Further, the input unit 140 may be a microphone or the like that receives voice input from the administrator A1. The microphone may be wireless.

(Output section 150)
The output unit 150 is a display device that displays various information. For example, the output unit 150 is a display screen realized by a liquid crystal display (LCD), an organic electro-luminescent display (EL display), or the like. The output unit 150 displays various information under the control of the control unit 160. For example, the output unit 150 displays an operation screen for performing an operation related to checking the internal situation of the railway vehicle and notification information received by the receiving unit 163. Note that when a touch panel is employed in the terminal device 100, the input section 140 and the output section 150 are integrated. Furthermore, in the following description, the output unit 150 may be referred to as a screen.

(Control unit 160)
The control unit 160 is a controller, and for example, various programs stored in the storage device inside the terminal device 100 use the RAM as a work area by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. This is achieved through execution. Further, the control unit 160 is a controller, and is realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 160 has a reception unit 161, a transmission unit 162, and a reception unit 163 as functional units, and may realize or execute the information processing operation described below. Note that the internal configuration of the control unit 160 is not limited to the configuration shown in FIG. 4, and may be any other configuration as long as it performs information processing to be described later. Further, each functional unit indicates a function of the control unit 160, and does not necessarily have to be physically distinct.

(Reception Department 161)
The reception unit 161 receives an operation (also referred to as a confirmation operation) related to checking the internal situation of the railway vehicle from the manager A1. Specifically, the reception unit 161 receives a confirmation operation from the administrator A1 via the input unit 140.

More specifically, the reception unit 161 receives an operation to start the application AP1 from the administrator A1 via the input unit 140. When the reception unit 161 receives the activation operation of the application AP1, the output unit 150 displays an operation screen for performing an operation related to checking the internal situation of the railway vehicle. FIG. 8 is a diagram showing an example of an operation screen according to the first embodiment. In the example shown in FIG. 8, when the reception unit 161 receives an operation to start the application AP1, the output unit 150 displays an operation screen for selecting a vehicle to be checked. For example, the output unit 150 outputs an operation button B1 to be selected when selecting car No. 1, an operation button B2 to be selected when selecting car No. 2, an operation button B3 to be selected when selecting car No. 3, and an operation button B3 to be selected when selecting car No. 4. An operation screen including an operation button B4 to be selected when selecting a subsequent vehicle is displayed. As an example of a confirmation operation, the reception unit 161 receives a selection operation for selecting an operation button corresponding to a vehicle to be confirmed from the manager A1 via the input unit 140. For example, the receiving unit 161 receives a selection operation to select the operation button B1.

Further, in the example shown in FIG. 8, the output unit 150 displays an operation screen including an operation button B5 to be selected when confirming the vehicle formation. The reception unit 161 may receive an operation of selecting the operation button B5 via the input unit 140 as an example of the confirmation operation. When the reception unit 161 receives the operation of selecting the operation button B5, the output unit 150 displays a composition screen on which the composition of vehicles making up the train can be confirmed. For example, the output unit 150 displays a composition screen in which blocks indicating each vehicle are associated with vehicle identification information that allows identification of each vehicle. For example, if the train is a 10-car train, the output unit 150 outputs 10 blocks indicating cars 1 to 10, and numbers that can identify cars 1 to 10 (for example, numbers 1 to 10). Displays the organization screen that associates each. As an example of a confirmation operation, the reception unit 161 receives a selection operation for selecting a block corresponding to a vehicle to be confirmed from the administrator A1 via the input unit 140.

Furthermore, as an example of the confirmation operation, the reception unit 161 may accept an operation of selecting a plurality of operation buttons at the same time. For example, the reception unit 161 may accept an operation of selecting operation buttons B1 to B3 at the same time.

(Transmission unit 162)
When the reception unit 161 receives the confirmation operation, the transmission unit 162 transmits request information regarding confirmation of the internal situation of the railway vehicle to the information processing device 200. Specifically, the transmitter 162 transmits, via the communication unit 110, request information regarding confirmation of the internal situation of the vehicle corresponding to the operation button selected by the selection operation received by the reception unit 161. For example, when the reception unit 161 receives a selection operation to select the operation button B1, the transmission unit 162 transmits request information regarding confirmation of the internal situation of the No. 1 car corresponding to the operation button B1.

Further, the transmitting unit 162 transmits, via the communication unit 110, request information regarding confirmation of the internal situation of the vehicle corresponding to the block of the composition screen selected by the selection operation received by the receiving unit 161. For example, when the receiving unit 161 receives a selection operation for selecting a block corresponding to car No. 5 on the composition screen, the transmitting unit 162 transmits request information regarding confirmation of the internal situation of car No. 5.

Further, when the reception unit 161 receives an operation for selecting a plurality of operation buttons at the same time, the transmission unit 162 may transmit request information regarding checking the internal status of each of the plurality of vehicles corresponding to each of the plurality of operation buttons. . For example, when the reception unit 161 receives an operation to select operation buttons B1 to B3 at the same time, the transmission unit 162 transmits request information regarding checking the internal status of cars No. 1 to Car No. 3 corresponding to the operation buttons B1 to B3, respectively. Send.

(Receiving unit 163)
The receiving unit 163 receives from the information processing device 200 notification information in which vehicle identification information that can identify a railway vehicle is associated with residue information regarding a residue or a person remaining inside the railway vehicle. Specifically, the receiving unit 163 receives vehicle identification information capable of identifying each vehicle corresponding to the request information transmitted by the transmitting unit 162, and residual information remaining on each vehicle corresponding to the request information transmitted by the transmitting unit 162. Or receive notification information associated with residue information regarding the remaining person.

FIG. 9 is a diagram showing an example of a notification screen according to the first embodiment. FIG. 9 shows a case where the transmitter 162 has previously transmitted request information regarding the internal situation confirmation of each of cars No. 1 to No. 3 corresponding to the operation buttons B1 to B3 shown in FIG. 8, respectively. The receiving unit 163 receives vehicle identification information that can identify each of the first to third cars corresponding to the request information transmitted by the transmitting unit 162, and the remaining vehicle identification information for each of the first to third cars corresponding to the request information transmitted by the transmitting unit 162. The notification information associated with the residue or residue information regarding the person who remains is received. For example, as an example of the vehicle identification information, the receiving unit 163 sends "Car No. 1" to "Car No. 3", which are characters that can identify cars No. 1 to Car No. 3, respectively, and as an example of the residual information, Notification information is received in which vehicle images G1 to G3 corresponding to each interior are associated with each other.

The output unit 150 displays the notification information received by the receiving unit 163. The output unit 150 displays a notification screen corresponding to the notification information received by the receiving unit 163. Specifically, the output unit 150 outputs vehicle identification information capable of identifying each vehicle corresponding to the request information transmitted by the transmitting unit 162, and residue remaining on each vehicle corresponding to the request information transmitted by the transmitting unit 162. Alternatively, a notification screen is displayed that is associated with the residue information regarding the remaining person. In FIG. 9, the output unit 150 outputs “Car No. 1” to “Car No. 3” which are characters that can identify the cars No. 1 to Car No. 3 received by the receiving unit 163, and Cars No. 1 to Car No. 3 received by the receiving unit 163. A notification screen is displayed in which vehicle images G1 to G3 corresponding to each interior are associated with each other. For example, the output unit 150 displays a notification screen in which three vehicle images G1 to G3 are displayed side by side on the screen.

In addition, when displaying a plurality of vehicle images side by side on the screen, the output unit 150 visually displays a vehicle image of a vehicle with a residue or a person remaining than a vehicle image of a vehicle without a residue or a person remaining. may be displayed with emphasis. For example, the output unit 150 may make the area of the display area of the vehicle image of the vehicle in which the residue or the remaining person exists larger than the display area of the vehicle image of the vehicle in which the residue or the remaining person does not exist. In FIG. 9, there were no residues and residues in cars No. 1 and 2, and there were residues and residues in car No. 3. At this time, the output unit 150 displays a display area of the vehicle image G3 of the third car in which the residue and the remaining person are present, rather than a display area of the vehicle images G1 and G2 of the first car and the second car in which the residue and the remaining person do not exist. Increase the area of

Further, when displaying a vehicle image of a vehicle in which a residue or a person remains, the output unit 150 may visually emphasize and display a region including the residue or a person remaining. In FIG. 9, the output unit 150 enlarges and displays the region G31 containing the residue and the region G32 containing the remaining person, and further surrounds the region G31 containing the residue and the region G32 containing the remaining person with a thick frame, respectively. to visually emphasize the display. Further, the output unit 150 may visually emphasize the area containing the residue or the area containing the remaining person, and display the area in a superimposed manner on the display area of the vehicle image of the vehicle in which the residue or the remaining person exists. . In FIG. 9, the output unit 150 visually emphasizes a region G31 containing a residue and a region G32 containing a remaining person, and then displays them superimposed on the display region of the vehicle image G3.

[2-2. Configuration of information processing device]
FIG. 5 is a diagram illustrating a configuration example of an information processing apparatus according to the first embodiment. As shown in FIG. 5, the information processing device 200 includes a communication section 210, a storage section 220, and a control section 230.

(Communication Department 210)
The communication unit 210 is connected to a network by wire or wirelessly, and sends and receives information to and from the terminal device 100, the imaging device 300, and the analysis device 400. For example, the communication unit 210 is realized by a NIC, an antenna, or the like.

(Storage unit 220)
The storage unit 220 stores various information. Specifically, storage unit 220 stores information regarding vehicle images corresponding to each vehicle received from imaging device 300. For example, the storage unit 220 stores information that associates a vehicle image with camera identification information that can identify the imaging device 300 that captured the vehicle image. The storage unit 220 also stores camera identification information that can identify the imaging device 300, vehicle identification information that can identify the railway vehicle in which the imaging device 300 is installed, and locations in the vehicle that can be photographed by the imaging device 300. Stores information associated with identifiable location identification information. Furthermore, the storage unit 220 stores information regarding the analysis results obtained from the analysis device 400. The storage unit 220 is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

(Control unit 230)
The control unit 230 is a controller, and for example, various programs (an example of an information processing program) stored in a storage device inside the information processing device 200 are executed by a CPU, an MPU, etc. using the RAM as a work area. Realized. Further, the control unit 230 is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

The control unit 230 has an acquisition unit 231, a generation unit 232, and a notification unit 233 as functional units, and may realize or execute the information processing operation described below. Note that the internal configuration of the control unit 230 is not limited to the configuration shown in FIG. 5, and may be any other configuration as long as it performs information processing to be described later. Further, each functional unit indicates a function of the control unit 230, and does not necessarily have to be physically distinct.

(Acquisition unit 231)
The acquisition unit 231 receives request information regarding confirmation of the internal situation of the railway vehicle from the terminal device 100 via the communication unit 210 . When the acquisition unit 231 receives request information regarding confirmation of the internal situation of the railway vehicle, it transmits request information regarding a request to transmit a vehicle image to the imaging device 300 .

Furthermore, the acquisition unit 231 receives vehicle images corresponding to each vehicle from the imaging device 300 via the communication unit 210. When receiving the vehicle image corresponding to each vehicle, the acquisition unit 231 transmits the vehicle image corresponding to each vehicle to the analysis device 400. In addition, the acquisition unit 231 sends a correct vehicle image, which is an image corresponding to the inside of each vehicle, which is an image corresponding to the inside of each vehicle, which is taken in a state where there is no residue or person inside each vehicle, to the analysis device, along with the vehicle image corresponding to each vehicle. 400. Note that the acquisition unit 231 may transmit the correct vehicle image of each vehicle to the analysis device 400 in advance before transmitting the vehicle image corresponding to each vehicle.

Further, the acquisition unit 231 acquires analysis information regarding the analysis result of image analysis on a vehicle image, which is an image corresponding to the inside of a railway vehicle, from the analysis device 400 via the communication unit 210. Specifically, the acquisition unit 231 acquires the analysis information when receiving request information regarding confirmation of the internal situation of the railway vehicle from the terminal device 100 used by the administrator A1 who manages the operation of the railway vehicle.

In addition, the acquisition unit 231 acquires, as analysis information, information regarding a difference image obtained by extracting the difference between the correct vehicle image, which is an image corresponding to the inside of the railway vehicle without any residue or persons, and the vehicle image. Specifically, the acquisition unit 231 acquires, as analysis information, residue information regarding residues or persons remaining inside each vehicle, which is estimated by the analysis device 400 based on information regarding the difference images of each vehicle. . In addition, below, the residue information regarding the residue remaining inside each vehicle or the remaining person may be referred to as the residue information of each vehicle. More specifically, the acquisition unit 231 acquires, as analysis information, information in which the residue information of each vehicle is associated with camera identification information that can identify the imaging device 300 that captured the vehicle image of each vehicle. .

Further, the acquisition unit 231 acquires analysis information that is information regarding the analysis results of image analysis on vehicle images that are images corresponding to the insides of each of a plurality of connected railway vehicles. The acquisition unit 231 acquires, as analysis information, information regarding difference images obtained by extracting the differences between correct vehicle images of each of the plurality of connected railway vehicles and vehicle images of each of the plurality of connected railway vehicles. Specifically, the acquisition unit 231 acquires information remaining inside each of the plurality of connected railway vehicles estimated by the analysis device 400 based on information regarding the difference images of each of the plurality of connected railway vehicles, as analysis information. Residue information regarding each residue or residue person is obtained. More specifically, the acquisition unit 231 obtains, as analysis information, residue information of each of the plurality of connected railway vehicles and each of the plurality of imaging devices 300 that respectively capture vehicle images of the plurality of connected railway vehicles. and each of a plurality of pieces of camera identification information that can be identified.

Further, the acquisition unit 231 acquires analysis information that is information regarding the analysis results of image analysis on vehicle images that are images corresponding to different locations inside the railway vehicle. Here, the vehicle images, which are images corresponding to different locations inside the railroad vehicle, are vehicle images captured by each of the plurality of imaging devices 300 arranged at positions for capturing different locations inside the railroad vehicle. refers to. In addition, hereinafter, vehicle images that are images corresponding to different locations inside a railroad vehicle may be referred to as vehicle images of different locations inside the railroad vehicle. Furthermore, hereinafter, correct vehicle images that are images corresponding to different locations inside the railway vehicle may be referred to as "correct vehicle images for each different location inside the railroad vehicle." The acquisition unit 231 obtains, as analysis information, information regarding difference images obtained by extracting the differences between correct vehicle images at different locations inside the railway vehicle and vehicle images at different locations inside the railway vehicle using the analysis device 400. Get each. Specifically, the acquisition unit 231 obtains, as analysis information, residual information remaining at each of the different locations inside the railway vehicle estimated by the analysis device 400 based on information regarding the difference images at each of the different locations inside the railway vehicle. Residue information regarding the substance or the person remaining is obtained respectively.

(Generation unit 232)
Based on the analysis information, the generation unit 232 generates notification information that associates vehicle identification information that can identify a railway vehicle with residue information regarding a residue or a person who remains inside the railway vehicle. Specifically, the generation unit 232 generates notification information based on the analysis information acquired by the acquisition unit 231. More specifically, the generation unit 232 generates notification information based on the residue information regarding the residue remaining inside each vehicle or the remaining person acquired by the acquisition unit 231. For example, the generation unit 232 generates notification information based on information that associates the residue information of each vehicle acquired by the acquisition unit 231 with camera identification information that can identify the imaging device 300 that captured the vehicle image of each vehicle. generate. For example, the generation unit 232 refers to the storage unit 220 and identifies vehicle identification information that is linked to camera identification information that can identify the imaging device 300 that captured the vehicle image of each vehicle acquired by the acquisition unit 231. . Next, the generation unit 232 generates vehicle identification information linked to camera identification information that can identify the imaging device 300 that captured the vehicle image of each vehicle, and information about the residue or person remaining inside each vehicle. Generates notification information that is associated with physical information.

In addition, the generation unit 232 generates vehicle identification information that can identify each of the plurality of connected railway vehicles, and information regarding residues or persons remaining inside each of the plurality of connected railway vehicles, based on the analysis information. Generates notification information that is associated with the residue information. For example, the generation unit 232 refers to the storage unit 220 and generates a plurality of camera identification information that can identify each of the plurality of imaging devices 300 that captured the vehicle images of the plurality of connected railway vehicles acquired by the acquisition unit 231. Identify the vehicle identification information associated with each. Subsequently, the generation unit 232 generates vehicle identification information that is linked to each of the plurality of camera identification information that can identify each of the plurality of imaging devices 300 that captured the vehicle images of each of the plurality of connected railway vehicles. Notification information is generated in which notification information is associated with residue information, which is information regarding residues or persons remaining inside each of the plurality of railway vehicles.

Furthermore, the generation unit 232 generates, based on the analysis information, location identification information that can identify each of the different locations inside the railroad vehicle, and information regarding residues or persons remaining at each of the different locations inside the railroad vehicle. Generate notification information associated with residue information. For example, the generation unit 232 refers to the storage unit 220 and generates vehicle identification information and location identification linked to camera identification information that can identify the imaging device 300 that captured the vehicle image of each vehicle acquired by the acquisition unit 231. Identify information. Next, the generation unit 232 generates vehicle identification information and location identification information that are linked to camera identification information that can identify the imaging device 300 that captured the vehicle image of each vehicle, and information that remains in different locations inside each vehicle. Notification information is generated in which the notification information is associated with the residue information that is information about the residue or the person who remains.

Furthermore, the generation unit 232 generates notification information in which letters or numbers that can identify each vehicle are associated with each other as an example of vehicle identification information, and a vehicle image of each vehicle as an example of residue information. Furthermore, as an example of the residue information, the generation unit 232 may generate notification information in which character information indicating the presence or absence of residue or a person remaining inside each vehicle is associated with vehicle identification information. good. Furthermore, the generation unit 232 generates, as an example of residue information, notification information in which character information indicating the position of residue or a person remaining inside each vehicle is associated with vehicle identification information and location identification information. may be generated.

(Notification section 233)
The notification unit 233 transmits the notification information generated by the generation unit 232 to the terminal device 100 via the communication unit 210.

[2-3. Configuration of imaging device]
FIG. 6 is a diagram illustrating a configuration example of an imaging device according to the first embodiment. As shown in FIG. 6, the imaging device 300 includes a communication section 310, a storage section 320, an imaging section 330, and a control section 340.

(Communication Department 310)
The communication unit 310 is connected to a network by wire or wirelessly, and transmits and receives information to and from the information processing device 200. For example, the communication unit 310 is realized by a NIC, an antenna, or the like.

(Storage unit 320)
The storage unit 320 stores various information. Specifically, the storage unit 320 stores information regarding images captured by the imaging unit 330. For example, the storage unit 320 stores information that associates images captured by the imaging unit 330 with camera identification information that can identify the imaging device 300. The storage unit 320 also stores camera identification information that can identify the imaging device 300, vehicle identification information that can identify the railway vehicle in which the imaging device 300 is installed, and locations in the vehicle that can be photographed by the imaging device 300. Stores information associated with identifiable location identification information. For example, it is realized by a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

(Imaging unit 330)
The imaging unit 330 is an image sensor (camera) that photographs a subject. Specifically, the imaging unit 330 captures a vehicle image that is an image corresponding to the inside of a railway vehicle. For example, the imaging unit 330 is realized by a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge-Coupled Device) image sensor, or the like. Note that the imaging unit 330 is not limited to a built-in camera, and may be an external camera such as a wireless camera that can communicate with the imaging device 300 or a web camera.

(Control unit 340)
The control unit 340 is a controller, and is realized by, for example, a CPU, an MPU, or the like executing various programs stored in a storage device inside the imaging device 300 using the RAM as a work area. Further, the control unit 340 is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

The control unit 340 has a receiving unit 341 and a transmitting unit 342 as functional units, and may realize or execute the information processing operation described below. Note that the internal configuration of the control unit 340 is not limited to the configuration shown in FIG. 6, and may be any other configuration as long as it performs information processing to be described later. Further, each functional unit indicates a function of the control unit 340, and does not necessarily have to be physically distinct.

(Receiving unit 341)
The receiving unit 341 receives request information regarding a request to transmit a vehicle image from the information processing device 200 via the communication unit 310 .

(Transmission unit 342)
When the receiving unit 341 receives request information regarding a request to transmit a vehicle image, the transmitting unit 342 transmits a vehicle image corresponding to each vehicle to the information processing device 200 via the communication unit 310. For example, the transmitting unit 342 refers to the storage unit 320 and transmits a vehicle image of the vehicle corresponding to the request information received by the receiving unit 341.

[2-4. Configuration of analysis device]
FIG. 7 is a diagram illustrating a configuration example of an analysis device according to the first embodiment. As shown in FIG. 7, the analysis device 400 includes a communication section 410, a storage section 420, and a control section 430.

(Communication Department 410)
The communication unit 410 is connected to a network by wire or wirelessly, and transmits and receives information to and from the information processing device 200. For example, the communication unit 410 is realized by a NIC, an antenna, or the like.

(Storage unit 420)
Storage unit 420 stores various information. For example, the storage unit 420 stores information regarding images acquired from the information processing device 200. The storage unit 420 is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk.

(Control unit 430)
The control unit 430 is a controller, and is realized by, for example, a CPU, an MPU, or the like executing various programs stored in a storage device inside the analysis device 400 using the RAM as a work area. Further, the control unit 430 is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

The control unit 430 has an acquisition unit 431, an estimation unit 432, and a transmission unit 433 as functional units, and may realize or execute the information processing operation described below. Note that the internal configuration of the control unit 430 is not limited to the configuration shown in FIG. 7, and may be any other configuration as long as it performs information processing to be described later. Further, each functional unit indicates a function of the control unit 430, and does not necessarily have to be physically distinct.

(Acquisition unit 431)
The acquisition unit 431 receives vehicle images of each vehicle from the information processing device 200 via the communication unit 410. Specifically, the acquisition unit 431 acquires information that associates the vehicle image of each vehicle with camera identification information that can identify the imaging device 300 that captured the vehicle image of each vehicle.

(Estimation unit 432)
When the acquisition unit 431 receives the vehicle image of each vehicle, the estimation unit 432 performs image analysis on the vehicle image of each vehicle. Specifically, the estimation unit 432 generates information regarding a difference image obtained by extracting the difference between the correct vehicle image of each vehicle and the vehicle image of each vehicle. Subsequently, the estimating unit 432 calculates the residue or residual person remaining inside each vehicle based on the information regarding the difference image of each vehicle as analysis information that is information regarding the analysis result of image analysis for the vehicle image of each vehicle. Estimate residue information regarding.

More specifically, the estimating unit 432 estimates, as an example of the residue information, information indicating whether there is a residue or a person remaining inside each vehicle. For example, the estimation unit 432 determines whether there is an area (hereinafter also referred to as a difference area) corresponding to the difference between the correct vehicle image of each vehicle and the vehicle image of each vehicle in the difference image of each vehicle. judge. If the estimation unit 432 determines that a difference region exists in the difference images of each vehicle, it estimates that a residue or a person remains inside each vehicle. On the other hand, if the estimation unit 432 determines that there is no difference region in the difference images of each vehicle, it estimates that there are no residues or persons inside each vehicle.

Furthermore, the estimating unit 432 estimates information indicating the position of a residue or a person remaining inside the railway vehicle, as an example of residue information. For example, when the estimation unit 432 estimates that there is a residue or a person remaining inside a specific vehicle, the estimation unit 432 estimates the position of the difference region in the difference image of the specific vehicle. For example, the estimation unit 432 specifies two-dimensional coordinates indicating the position of the difference region in the difference image of a specific vehicle. Furthermore, the estimating unit 432 estimates two-dimensional coordinates indicating the position of each target object area in the correct vehicle image of the specific vehicle. For example, the estimating unit 432 estimates two-dimensional coordinates indicating the position of a region of a passenger aisle, a seating seat, a luggage rack, a window, a door, a side wall, or a ceiling in the correct vehicle image of a specific vehicle.

Next, the estimation unit 432 compares the two-dimensional coordinates indicating the position of the difference area in the differential image of the specific vehicle with the two-dimensional coordinates indicating the position of each object area in the correct vehicle image of the specific vehicle. Based on this, the three-dimensional position in the specific vehicle of the residue or the remaining person included in the difference area in the difference image of the specific vehicle is estimated. For example, if the two-dimensional coordinates indicating the position of the differential region in the differential image of a specific vehicle match the two-dimensional coordinates indicating the position of the seat area of the passenger seat in the correct vehicle image of the specific vehicle, the estimation unit Step 432 estimates that the residue or residual person included in the difference region in the difference image of the specific vehicle is located at the seat position of the passenger seat in the specific vehicle.

(Transmission unit 433)
The transmitting unit 433 transmits analysis information, which is information regarding the analysis results of the image analysis of the vehicle images of each vehicle performed by the estimating unit 432, to the information processing device 200 via the communication unit 410. Specifically, the transmitting unit 433 sends, as analysis information, residue information regarding the residue or remaining person remaining inside each vehicle estimated by the estimating unit 432, and the imaging device 300 that captured the vehicle image of each vehicle. Sends information associated with identifiable camera identification information.

[3. Second embodiment]
The second embodiment will now be described. In the second embodiment, a case will be described in which the imaging device 300 has a function equivalent to the estimation unit 432 of the analysis device 400. Note that descriptions of points similar to those in the first embodiment will be omitted as appropriate.

FIG. 10 is a diagram showing an overview of information processing according to the second embodiment. The information processing system 1A shown in FIG. 10 differs from the information processing system 1 shown in FIG. 3 in that it does not include the analysis device 400. As shown in FIG. 10, the information processing system 1A includes a terminal device 100, an information processing device 200, and an imaging device 300. The terminal device 100, the information processing device 200, and the imaging device 300 are connected to be able to communicate with each other by wire or wirelessly via a network (not shown).

The image capturing device 300A is an information processing device that captures a vehicle image and estimates residue information regarding a residue or a person remaining inside the railway vehicle based on the vehicle image. A plurality of imaging devices 300A are provided in one or a plurality of connected railway vehicles. The imaging device 300A also has a function of communicating with the information processing device 200 via a network (not shown). Note that the imaging device 300A may include a lamp (for example, an LED fluorescent lamp) or a lamp. For example, the imaging device 300A may be a security camera integrated with an LED fluorescent lamp that is integrated with a lamp or a lamp.

Further, the imaging system includes at least one imaging device 300A. For example, the imaging system includes at least one imaging device 300A and multiple imaging devices 300. Note that the imaging system may include a plurality of imaging devices 300A.

Returning to the explanation of FIG. In FIG. 10, the terminal device 100 receives an operation related to checking the internal situation of the railway vehicle (hereinafter also referred to as confirmation operation) from the manager A1 (step S21). When the terminal device 100 receives the confirmation operation, it transmits request information regarding confirmation of the internal situation of the railway vehicle to the information processing device 200 (step S22).

When the information processing device 200 receives the request information regarding checking the situation inside the railway vehicle from the terminal device 100, the information processing device 200 transmits the request information regarding checking the inside situation of the railway vehicle to the imaging device 300A (step S23). When the imaging device 300A receives request information regarding confirmation of the internal situation of a railway vehicle from the information processing device 200, the imaging device 300A performs image analysis on vehicle images of each vehicle in the same manner as the analysis device 400 described in FIG. S24). Subsequently, the imaging device 300A transmits analysis information, which is information regarding the analysis results of the image analysis of the vehicle images of each vehicle, to the information processing device 200 (step S25).

The information processing device 200 acquires analysis information, which is information regarding the analysis results of image analysis for vehicle images of each vehicle, from the imaging device 300A. Next, based on the analysis information, the information processing device 200 generates notification information that associates vehicle identification information that can identify each vehicle with residue information regarding the residue remaining inside each vehicle or the residue person. is generated (step S26). Subsequently, when the information processing device 200 generates notification information, the information processing device 200 transmits the generated notification information to the terminal device 100 (step S27).

When the terminal device 100 receives the notification information from the information processing device 200, it displays the notification information on the screen (step S28).

[3-1. Configuration of imaging device]
FIG. 11 is a diagram illustrating a configuration example of an imaging device according to the second embodiment. As shown in FIG. 11, the imaging device 300A includes a communication section 310, a storage section 320, an imaging section 330, and a control section 340A. Note that in the imaging device 300A, descriptions of the same points as the imaging device 300 will be omitted as appropriate.

(Control unit 340A)
The control unit 340A is a controller, and is realized by, for example, a CPU, an MPU, or the like executing various programs stored in a storage device inside the imaging device 300A using the RAM as a work area. Further, the control unit 340A is a controller, and is realized by, for example, an integrated circuit such as an ASIC or an FPGA.

The control unit 340A has a receiving unit 341, a transmitting unit 342, and an estimating unit 343A as functional units, and may realize or execute the information processing operation described below. Note that the internal configuration of the control unit 340A is not limited to the configuration shown in FIG. 11, and may be any other configuration as long as it performs information processing to be described later. Further, each functional unit indicates a function of the control unit 340A, and does not necessarily have to be physically distinct.

(Receiving unit 341)
The receiving unit 341 receives request information regarding confirmation of the internal situation of the railway vehicle from the information processing device 200 via the communication unit 310 .

(Transmission unit 342)
The transmitting unit 342 transmits analysis information, which is information regarding the analysis results of image analysis on vehicle images of each vehicle, to the information processing device 200 via the communication unit 310.

(Estimation unit 343A)
The estimation unit 343A estimates residue information regarding residues or persons remaining inside the railway vehicle, based on the vehicle image, in the same way as the estimation unit 432 according to the first embodiment. Specifically, the estimating unit 343A estimates the residue information when the receiving unit 341 receives request information regarding confirmation of the internal situation of the railway vehicle. For example, the estimating unit 343A estimates the residue information based on a difference image obtained by extracting the difference between the correct vehicle image, which is an image corresponding to the inside of the railway vehicle in which there are no residues or residual persons, and the vehicle image. . Furthermore, the estimating unit 343A estimates information indicating the presence or absence of a residue or a remaining person as the residue information. Furthermore, the estimating unit 343A estimates information indicating the position of the remaining object or the remaining person inside the railway vehicle as the remaining object information.

Note that the above-described imaging device 300A may further include a lamp (for example, an LED fluorescent lamp) or a lamp. For example, the imaging device 300A may be a security camera integrated with an LED fluorescent lamp that is integrated with a lamp or a lamp.

[4. Modified example]
In the first and second embodiments, an example of processing has been described. However, each embodiment is not limited to this. Modifications of the process will be described below.

[4-1. Estimation of residue information based on image recognition processing]
In the first and second embodiments described above, the case where the estimation unit 432 or the estimation unit 343A estimates the residue information based on the difference image obtained by extracting the difference between the correct vehicle image and the vehicle image has been described. In a modification, a case will be described in which the estimation unit 432 or the estimation unit 343A estimates residue information based on information regarding the processing result of image recognition processing on a vehicle image.

Specifically, the estimation unit 432 or 343A estimates the residue information based on the processing result of image recognition processing using a machine learning model for the vehicle image. For example, the estimation unit 432 or the estimation unit 343A uses an object detection technique or a segmentation technique to detect an object included in the vehicle image. For example, the estimation unit 432 or the estimation unit 343A uses a first machine learning model that is trained to output information regarding the class and position of an object included in the image as output information when an image is input as input information. is used to estimate the class and position of objects included in vehicle images. For example, the estimating unit 432 or the estimating unit 343A uses the first machine learning model to estimate the cabin aisle, seating seat, luggage rack, window, door, side wall, ceiling, residue, or residual person included in the vehicle image. To detect. Subsequently, when an object included in the vehicle image is detected, the estimating section 432 or 343A selects a passenger passageway, a seating seat, a luggage shelf, a window, a door, Identify objects excluding side walls or ceilings. Subsequently, the estimation unit 432 or 343A estimates the identified object to be a residue or a remaining person.

[4-2. Trigger automation〕
In the first and second embodiments described above, a case will be described in which the information processing device 200 acquires analysis information when the terminal device 100 receives an operation related to checking the internal situation of a railway vehicle from the administrator A1. did. In a modification, a case will be described in which the information processing device 200 automates the trigger for acquiring analysis information. Thereby, the information processing system 1 can further reduce the burden on the administrator A1.

[4-2-1. Utilization of location information (geofence)]
Specifically, a geofence is set in advance in an area within a first range from the position coordinates corresponding to a specific position. For example, the particular location is located within a second range from a station or depot at which the rail vehicle arrives.

Further, the terminal device 100 has a function of identifying its own position using GPS (Global Positioning System) or the like. For example, the sensor unit 130 of the terminal device 100 includes GNSS (Global Navigation Satellite System). A GNSS sensor uses GNSS to receive radio waves containing positioning data transmitted from a navigation satellite. The positioning data is used to detect the absolute position of the terminal device 100 from latitude and longitude information. Note that the GNSS used may be, for example, GPS or another system. Further, the sensor unit 130 outputs positioning data generated by the GNSS sensor to the control unit 160. The control unit 160 identifies the self-location of the terminal device 100 based on the positioning data.

Further, the terminal device 100 is mounted on a railway vehicle. For example, the terminal device 100 is installed near the front of a railway vehicle (for example, in a crew cabin). At this time, when the railway vehicle enters the area within the first range from the position coordinates corresponding to the specific position, the terminal device 100 also enters the area within the first range from the position coordinates corresponding to the specific position. When the terminal device 100 identifies its own location, it transmits location information of its own location to the information processing device 200. The acquisition unit 231 of the information processing device 200 acquires position information of the terminal device 100 mounted on a railway vehicle. When the acquisition unit 231 acquires the position information of the terminal device 100, the acquisition unit 231 moves the railway to an area within a first range from the position coordinates corresponding to the specific position based on the position information of the terminal device 100 mounted on the railway vehicle. Detects vehicle intrusion. The acquisition unit 231 acquires analysis information when it is detected that a railway vehicle has entered an area within a first range from the position coordinates corresponding to a specific position.

[4-2-2. Utilization of sensors]
Specifically, the sensor unit 130 of the terminal device 100 includes a transmitter or a receiver compatible with short-range wireless communication. For example, the sensor unit 130 includes a transmitter or a receiver compatible with wireless LAN (Local Area Network) communication or wireless PAN (Personal Area Network) communication. For example, the sensor unit 130 includes a transmitter using a short-range wireless communication method such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA (Infrared Data Association), etc. Connects to a receiver of a short-range wireless communication method such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc. installed within range (for example, at the end of a station platform or garage). Alternatively, the sensor unit 130 includes a receiver for a short-range wireless communication method such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc., and is within a predetermined range from the station or depot where the railway vehicle arrives (for example, It may also be connected to a transmitter of a short-range wireless communication method such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc. installed at a station platform or the end of a garage, etc.

Moreover, the terminal device 100 is mounted on a railway vehicle. For example, the terminal device 100 is installed near the front of a railway vehicle (for example, in a crew cabin). At this time, when the railway vehicle enters the area within the first range from the position coordinates corresponding to the specific position, the terminal device 100 also enters the area within the first range from the position coordinates corresponding to the specific position. For example, the terminal device 100 includes a receiver for short-range wireless communication such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc. installed within a predetermined range from the station or depot where the railway vehicle arrives. Receive radio waves of short-range wireless communication such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc. from a transmitter of short-range wireless communication system such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc. . A receiver for short-range wireless communication such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc., transmits short-range wireless communication such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc. from the terminal device 100. If a radio wave is received from the terminal device 100, information indicating that a radio wave of short-range wireless communication such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc. has been received is transmitted to the information processing device 200. . The acquisition unit 231 of the information processing device 200 receives information from a receiver of a short-range wireless communication system such as Bluetooth (registered trademark), ZigBee (registered trademark)), IrDA, etc. Receives information indicating that short-range wireless communication radio waves have been received. The acquisition unit 231 acquires analysis information when receiving information indicating that radio waves of short-range wireless communication such as Bluetooth (registered trademark), ZigBee (registered trademark), IrDA, etc. have been received.

In this way, the acquisition unit 231 performs analysis when a short-range wireless communication receiver installed at a specific position receives a signal transmitted from a short-range wireless communication transmitter mounted on a railway vehicle. Get information. For example, the particular location is located within a second range from a station or depot at which the rail vehicle arrives.

[4-3. Update of teacher data]
The correct vehicle images described in the first and second embodiments described above may be updated periodically. For example, the imaging device 300 captures a vehicle image, which is an image corresponding to the inside of the railway vehicle, when the railway vehicle leaves the warehouse or when the terminal device 100 is activated. The imaging device 300 or the analysis device 400 performs image analysis on a vehicle image captured when the railway vehicle leaves the warehouse or when the terminal device 100 is activated. When the imaging device 300 or the analysis device 400 determines that there are no residues or residual persons inside the railway vehicle as a result of image analysis, the imaging device 300 or the analysis device 400 determines that there are no residues or residual persons inside the railway vehicle. The image is set as a new correct vehicle image.

The imaging device 300 also captures a vehicle image, which is an image corresponding to the inside of the railway vehicle, in accordance with the frequency of replacing in-vehicle advertisements. The imaging device 300 or the analysis device 400 performs image analysis on captured vehicle images depending on the frequency of replacing in-vehicle advertisements. When the imaging device 300 or the analysis device 400 determines that there are no residues or residual persons inside the railway vehicle as a result of image analysis, the imaging device 300 or the analysis device 400 determines that there are no residues or residual persons inside the railway vehicle. The image is set as a new correct vehicle image.

Furthermore, the imaging device 300 or the analysis device 400 may selectively use correct vehicle images captured during the day and correct vehicle images captured at night. For example, for a vehicle image captured during the daytime (nighttime), the imaging device 300 or the analysis device 400 uses information about a difference image obtained by extracting the difference from a correct vehicle image captured during the daytime (nighttime). , estimate residue information.

[5. effect〕
As described above, the information processing device 200 according to the first embodiment includes the acquisition section 231 and the generation section 232. The acquisition unit 231 acquires analysis information regarding the analysis result of image analysis on a vehicle image that is an image corresponding to the inside of a railway vehicle. Based on the analysis information, the generation unit 232 generates notification information that associates vehicle identification information that can identify a railway vehicle with residue information regarding a residue or a person who remains inside the railway vehicle.

As a result, the information processing device 200 can, for example, notify the manager who manages the operation of the railway vehicles of the notification information so that the manager can confirm the residue or residual persons remaining inside each of the plurality of vehicles. The time required for this can be reduced. Therefore, the information processing device 200 can reduce the operational burden of confirming the residue or persons remaining inside the railway vehicle. In addition, the information processing device 200 can reduce the operational burden of checking for residues or persons remaining inside the railway vehicle, so it is possible to We can contribute to the achievement of "Let's create a foundation for technological innovation."

In addition, the acquisition unit 231 acquires, as analysis information, information regarding a difference image obtained by extracting the difference between the correct vehicle image, which is an image corresponding to the inside of the railway vehicle without any residue or persons, and the vehicle image. The generation unit 232 generates notification information based on information regarding the difference image.

Thereby, the information processing device 200 can generate notification information based on the residue information appropriately estimated based on the information regarding the difference image.

The acquisition unit 231 also acquires, as analysis information, information regarding the processing results of image recognition processing using a machine learning model for the vehicle image. The generation unit 232 generates notification information based on information regarding the processing result.

Thereby, the information processing device 200 can generate notification information based on residue information appropriately estimated based on information regarding the processing result of the image recognition process.

The generation unit 232 also generates notification information in which vehicle identification information is associated with residue information, which is information indicating the presence or absence of residue or a person remaining.

Thereby, the information processing device 200 can notify the administrator who manages the operation of railway vehicles of which of the plurality of vehicles there is (or is not) a residue or a residual person inside. can do. Therefore, the information processing device 200 can shorten the time required for the administrator to check the residue or persons remaining inside each of the plurality of vehicles.

Furthermore, the generation unit 232 generates notification information in which vehicle identification information is associated with residue information that is information indicating the position of a residue or a person remaining inside a railway vehicle.

Thereby, the information processing device 200 can notify the manager who manages the operation of the railway vehicles of the position inside each vehicle where the residue or the remaining person exists. Therefore, the information processing device 200 can shorten the time required for the administrator to check the residue or person remaining inside each vehicle.

Further, the acquisition unit 231 acquires analysis information when receiving request information regarding confirmation of the internal situation of the railway vehicle from the terminal device 100 used by the administrator who manages the operation of the railway vehicle.

Thereby, the information processing device 200 can notify the manager who manages the operation of the railway vehicle of the notification information at the timing desired by the manager.

Further, when the acquisition unit 231 detects that the railway vehicle has entered an area within a first range from the position coordinates corresponding to a specific position based on the position information of the terminal device 100 mounted on the railway vehicle, Obtain analysis information.

Thereby, the information processing device 200 can automatically generate notification information without bothering the administrator who manages the operation of the railway vehicle. Therefore, the information processing device 200 can further reduce the operational burden of confirming the residue or persons remaining inside the railway vehicle.

In addition, the acquisition unit 231 acquires analysis information when a short-range wireless communication receiver installed at a specific position receives a signal transmitted from a short-range wireless communication transmitter mounted on a railway vehicle. get.

Thereby, the information processing device 200 can automatically generate notification information without bothering the administrator who manages the operation of the railway vehicle. Therefore, the information processing device 200 can further reduce the operational burden of confirming the residue or persons remaining inside the railway vehicle.

Additionally, the particular location is located within a second range from a station or depot where the rail vehicle arrives.

Thereby, the information processing device 200 can automatically generate notification information at the timing when the railway vehicle approaches the station or depot where the railway vehicle arrives. Therefore, the information processing device 200 allows the administrator to check the residue or persons remaining inside each vehicle within the short time available for turning back from the terminal station or driving to the garage. The time required for this can be reduced.

Further, the acquisition unit 231 acquires analysis information that is information regarding the analysis results of image analysis on vehicle images that are images corresponding to different locations inside the railway vehicle. Based on the analysis information, the generation unit 232 generates location identification information that can identify each of the different locations inside the railway vehicle, and residual information that is information regarding residues or residual persons remaining at each of the different locations inside the railway vehicle. Generate notification information that is associated with the information.

Thereby, the information processing device 200 can notify the manager who manages the operation of the railway vehicles of the position inside each vehicle where the residue or the remaining person exists. Therefore, the information processing device 200 can shorten the time required for the administrator to check the residue or person remaining inside each vehicle.

Further, the vehicle images are images taken by each of the plurality of imaging devices 300 arranged at positions in the railway vehicle corresponding to different locations inside the railway vehicle.

As a result, the information processing device 200 can, for example, generate appropriate notification information based on a plurality of vehicle images that comprehensively capture the interior space of one vehicle so as to prevent blind spots from occurring. can.

In addition, the acquisition unit 231 acquires analysis information that is information regarding the analysis results of image analysis on vehicle images that are images corresponding to the inside of each of a plurality of connected railway vehicles. The generation unit 232 generates vehicle identification information that can identify each of the plurality of connected railway vehicles, and information regarding residues or residual persons remaining inside each of the plurality of connected railway vehicles, based on the analysis information. Generate notification information associated with residue information.

As a result, the information processing device 200 notifies the manager who manages the operation of the railway vehicles of appropriate notification information based on a plurality of vehicle images that comprehensively photograph the interior spaces of a plurality of vehicles. It can be made possible.

Moreover, the vehicle image is an image photographed by each of the plurality of imaging devices 300 arranged in each of the plurality of connected railway vehicles.

Thereby, the information processing device 200 can generate appropriate notification information based on a plurality of vehicle images comprehensively photographing the interior spaces of a plurality of vehicles.

Furthermore, the information processing device 200 further includes a notification section 233. The notification unit 233 transmits notification information to the terminal device 100 used by an administrator who manages the operation of railway vehicles.

Thereby, the information processing device 200 can notify the administrator who manages the operation of railway vehicles of which of the plurality of vehicles the residue or residual person exists (or does not exist) inside. I can do it.

Further, the imaging system according to the second embodiment (in the second embodiment, the imaging device 300A) is provided in one or a plurality of connected railway cars, and includes an imaging unit (in the second embodiment, the imaging device 300A) 330) and an estimation section (in the second embodiment, estimation section 343A). The imaging unit captures a vehicle image that is an image corresponding to the inside of the railway vehicle. The estimation unit estimates residue information regarding residues or persons remaining inside the railway vehicle based on the vehicle image.

Thereby, the imaging system, in a system provided in one or a plurality of connected railway vehicles, can appropriately estimate residue information regarding residues or residual persons remaining inside the railway vehicle. Further, the imaging system can, for example, be capable of notifying a manager who manages the operation of a railway vehicle of the residue information. In addition, the imaging system is necessary for the administrator to confirm the residue or residual person remaining inside each of multiple vehicles, for example, by notifying the administrator who manages the operation of the railway vehicle of the residue information. It is possible to shorten the time. Therefore, the imaging system can reduce the operational burden of confirming the residue or persons remaining inside the railway vehicle. In addition, the imaging system can reduce the operational burden of checking for residues or people remaining inside railway vehicles, and as a result, it is possible to We can contribute to the achievement of "Let's create a foundation for the future."

The estimation unit also estimates residual object information based on a correct vehicle image, which is an image corresponding to the interior of a railway vehicle where no residual objects or people are present, and a difference image obtained by extracting the difference between the vehicle image.

Thereby, the imaging system can appropriately estimate the residue information based on the information regarding the difference image.

Furthermore, the estimation unit estimates residue information based on the processing result of image recognition processing using a machine learning model for the vehicle image.

Thereby, the imaging system can appropriately estimate the residue information based on the processing result of the image recognition process.

Further, the estimating unit estimates information indicating the presence or absence of a residue or a remaining person as the residue information.

This allows the imaging system to estimate which vehicle of the plurality of vehicles has (or does not have) a residue or a residual person inside.

Further, the estimating unit estimates information indicating the position of the residue or the person remaining inside the railway vehicle as the residue information.

This allows the imaging system to estimate where the residue or person is located inside each vehicle.

Furthermore, the imaging unit further includes a lighting device or a lamp.

Thereby, the imaging system can use a security camera integrated with an LED fluorescent lamp that is integrated with a lamp or a lamp.

Furthermore, the railway vehicle according to the second embodiment includes an imaging system.

Thereby, the railroad vehicle can appropriately estimate residue information regarding the residue or the person remaining inside the railroad vehicle using the imaging system.

Further, the information processing method according to the second embodiment is executed by an information processing system (information processing system 1A in the second embodiment) including an imaging system and an information processing device 200. The imaging system captures a vehicle image, which is an image corresponding to the inside of the railway vehicle, and estimates residue information regarding residues or persons remaining inside the railway vehicle based on the vehicle image. The information processing device 200 generates notification information that associates vehicle identification information that can identify a railway vehicle with residual information, and notifies the manager who manages the operation of the railway vehicle of the notification information.

As a result, the information processing method can, for example, notify the administrator who manages the operation of railway vehicles of notification information, so that the administrator can confirm the residue or residual persons remaining inside each of multiple vehicles. The time required can be shortened. Therefore, the information processing method can reduce the operational burden of confirming the residue or persons remaining inside the railway vehicle. In addition, the information processing method can reduce the operational burden of checking for residues or people remaining inside railway vehicles, so it is possible to We can contribute to the achievement of "Let's create a foundation for innovation."

[6. Hardware configuration]
Further, the information processing apparatuses such as the information processing apparatus 200 according to the first and second embodiments and the imaging apparatus 300A according to the second embodiment described above are realized by a computer 1000 having a configuration as shown in FIG. 12, for example. be done. The information processing apparatus 200 according to the first and second embodiments will be described below as an example. FIG. 12 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing device. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I/F) 1500, an input/output interface (I/F) 1600, and a media interface (I/F) 1700.

CPU 1100 operates based on a program stored in ROM 1300 or HDD 1400, and controls each section. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started, programs depending on the hardware of the computer 1000, and the like.

The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. Communication interface 1500 receives data from other devices via a predetermined communication network and sends it to CPU 1100, and transmits data generated by CPU 1100 to other devices via a predetermined communication network.

The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and mouse via an input/output interface 1600. CPU 1100 obtains data from an input device via input/output interface 1600. Further, CPU 1100 outputs the generated data to an output device via input/output interface 1600.

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1200. CPU 1100 loads this program from recording medium 1800 onto RAM 1200 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the information processing device 200 according to the first embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 230 by executing a program loaded onto the RAM 1200. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from another device via a predetermined communication network.

Some of the embodiments of the present application have been described above in detail based on the drawings, but these are merely examples, and various modifications and variations may be made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure section of the invention. It is possible to carry out the invention in other forms with modifications.

[7. others〕
Furthermore, among the processes described in the above embodiments and modified examples, all or part of the processes described as being performed automatically can be performed manually, or may be described as being performed manually. All or part of this processing can also be performed automatically using known methods. In addition, information including the processing procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Furthermore, each component of each device shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads and usage conditions. Can be integrated and configured.

Furthermore, the above-described embodiments and modifications can be combined as appropriate within a range that does not conflict with the processing contents.

100 Terminal device 110 Communication section 120 Storage section 130 Sensor section 140 Input section 150 Output section 160 Control section 161 Reception section 162 Transmission section 163 Receiving section 200 Information processing device 210 Communication section 220 Storage section 230 Control section 231 Acquisition section 232 Generation section 233 Notification unit 300 Imaging device 310 Communication unit 320 Storage unit 330 Imaging unit 340 Control unit 341 Receiving unit 342 Transmission unit 400 Analysis device 410 Communication unit 420 Storage unit 430 Control unit 431 Acquisition unit 432 Estimation unit 433 Transmission unit 300A Imaging device 343A Estimation unit

The present invention relates to an information processing device , an information processing program, and an information processing method.

Therefore, the present disclosure provides an information processing device , an information processing program, and an information processing method that can reduce the operational burden of confirming residues or persons remaining inside a railway vehicle. With the goal.

Claims (24)

an acquisition unit that acquires analysis information regarding an analysis result of image analysis on a vehicle image that is an image corresponding to the inside of a railway vehicle;
a generation unit that generates notification information that associates vehicle identification information that can identify the railway vehicle with residue information regarding residues or residual persons remaining inside the railway vehicle, based on the analysis information;
An information processing device comprising: The acquisition unit includes:
As the analysis information, information about a difference image obtained by extracting a difference between a correct vehicle image, which is an image corresponding to the inside of the railway vehicle in which the residue and the residual person do not exist, and the vehicle image,
The generation unit is
generating the notification information based on information regarding the difference image;
The information processing device according to claim 1. The acquisition unit includes:
Obtaining information regarding the processing results of image recognition processing using a machine learning model for the vehicle image as the analysis information;
The generation unit is
generating the notification information based on information regarding the processing result;
The information processing device according to claim 1. The generation unit is
generating the notification information in which the vehicle identification information is associated with the residue information, which is information indicating the presence or absence of the residue or the person remaining;
The information processing device according to claim 1. The generation unit is
generating the notification information in which the vehicle identification information is associated with the residue information that is information indicating the position of the residue or the person remaining inside the railway vehicle;
The information processing device according to claim 1. The acquisition unit includes:
acquiring the analysis information when request information regarding confirmation of the internal situation of the railway vehicle is received from a terminal device used by a manager who manages the operation of the railway vehicle;
The information processing device according to claim 1. The acquisition unit includes:
When it is detected that the railway vehicle has entered an area within a first range from the position coordinates corresponding to a specific position based on the position information of a terminal device mounted on the railway vehicle, the analysis information is get,
The information processing device according to claim 1. The acquisition unit includes:
acquiring the analysis information when a short-range wireless communication receiver installed at a specific position receives a signal transmitted from the short-range wireless communication transmitter mounted on the railway vehicle;
The information processing device according to claim 1. the specific location is located within a second range from a station or depot at which the rail vehicle arrives;
The information processing device according to claim 7 or 8. The acquisition unit includes:
obtaining the analysis information, which is information regarding the analysis results of image analysis on the vehicle images, which are images corresponding to different locations inside the railway vehicle;
The generation unit is
Based on the analysis information, location identification information that can identify each of the different locations inside the railroad vehicle; and the residue information that is information regarding residues or residual persons remaining at each of the different locations inside the railroad vehicle. generating the notification information associated with
The information processing device according to claim 1. The vehicle image is an image taken by each of a plurality of imaging devices arranged at positions of the railway vehicle corresponding to different locations inside the railway vehicle.
The information processing device according to claim 10. The acquisition unit includes:
obtaining the analysis information, which is information regarding the analysis result of image analysis on the vehicle image, which is an image corresponding to the inside of each of the plurality of connected railway vehicles;
The generation unit is
Based on the analysis information, the vehicle identification information that can identify each of the plurality of connected railway vehicles, and information regarding residues or residual persons remaining inside each of the plurality of connected railway vehicles. generating the notification information in association with the certain residue information;
The information processing device according to claim 1. The vehicle image is an image taken by each of a plurality of imaging devices arranged in each of the plurality of connected railway vehicles,
The information processing device according to claim 12. further comprising a notification unit that transmits the notification information to a terminal device used by an administrator who manages the operation of the railway vehicle;
The information processing device according to claim 1. An imaging system provided in one or a plurality of connected railway vehicles, comprising:
an imaging unit that captures a vehicle image that is an image corresponding to the inside of the railway vehicle;
an estimation unit that estimates residue information regarding residues or persons remaining inside the railway vehicle based on the vehicle image;
An imaging system comprising: The estimation unit is
Estimating the residue information based on a difference image obtained by extracting a difference between the correct vehicle image, which is an image corresponding to the inside of the railway vehicle where the residue and the residue do not exist, and the vehicle image;
The imaging system according to claim 15. The estimation unit is
estimating the residue information based on a processing result of image recognition processing using a machine learning model for the vehicle image;
The imaging system according to claim 15. The estimation unit is
estimating information indicating the presence or absence of the residue or the residue as the residue information;
The imaging system according to claim 15. The estimation unit is
estimating, as the residue information, information indicating the position of the residue or the residue inside the railway vehicle;
The imaging system according to claim 15. The imaging unit further includes a lamp or a lamp.
The imaging system according to claim 15. an acquisition procedure for acquiring analysis information regarding an analysis result of image analysis for a vehicle image that is an image corresponding to the inside of a railway vehicle;
A generation procedure for generating notification information that associates vehicle identification information that can identify the railway vehicle with residue information regarding residues or residual persons remaining inside the railway vehicle, based on the analysis information;
An information processing program that causes a computer to execute. an imaging procedure for capturing a vehicle image that is an image corresponding to the inside of the railway vehicle;
an estimation procedure for estimating residue information regarding residues or persons remaining inside the railway vehicle based on the vehicle image;
An imaging program that causes a computer to execute A railway vehicle comprising the imaging system according to any one of claims 15 to 20. An information processing method executed by an information processing system including an imaging system and an information processing device,
The imaging system includes:
capturing a vehicle image that is an image corresponding to the inside of the railway vehicle, and estimating residue information regarding residues or residual persons remaining inside the railway vehicle based on the vehicle image;
The information processing device includes:
generating notification information that associates vehicle identification information that can identify the railway vehicle with the residue information, and notifying a manager who manages the operation of the railway vehicle of the notification information;
Information processing method.

Images