JP7491204B2 - Monitoring system, monitoring method, monitoring device, and monitoring program - Google Patents

Description

The present invention relates to a monitoring system, a monitoring method, a monitoring control device, and a monitoring program for monitoring the interior of a moving object where passengers are riding, for example.

In moving vehicles, particularly in transportation using vehicles such as buses and trains, a single monitor may monitor the images of the interior of the vehicle taken by surveillance cameras installed in each of multiple vehicles, for example, to ensure the safety of passengers.

In systems that monitor the interior of autonomous vehicles, monitors monitor camera footage from a remote location in order to respond to abnormalities and inquiries inside the vehicle. As the automation of vehicles and in-vehicle equipment progresses, the frequency of responding to abnormalities inside the vehicle decreases, and the number of vehicles that each monitor must monitor increases.

Assuming that one monitor is responsible for 100 vehicles, it is unrealistic to display images from the interiors of all the vehicles at once, so it is common to split the monitor screen (e.g., into four) and display only a portion of the images (from the four vehicles) by switching them in chronological order.

When all screens are displayed at once, monitors must pay close attention to many screens, which increases the burden on them and may result in them not being able to carry out the necessary checks and overlooking important abnormalities.

On the other hand, with split display, the burden on the monitor is reduced compared to simultaneous display, but the image of the inside of the vehicle cabin where an abnormality that requires display is occurring is not displayed, which creates the problem that the abnormality cannot be detected quickly.

Patent document 1 discloses a technology for monitoring facilities by viewing images captured by surveillance cameras.

Specifically, the image processing device has an index value calculation unit and a presentation unit, and the index value calculation unit acquires multiple images (captured images) captured by a camera and calculates an index value indicating the degree of change in the state of the monitored object shown in the captured images using the acquired captured images, while the presentation unit presents a display based on the index value calculated by the index value calculation unit on the captured images captured by the camera.

According to Patent Document 1, a security officer monitoring a security camera can immediately grasp the current state of the object being monitored.

JP 2020-78058 A

However, the index value indicating the degree of change of the monitored subject in Patent Document 1 is the degree of change when the monitored subject moves of its own volition. In other words, in Patent Document 1, as an example, a station platform is used as the shooting area, and the length and degree of change of the queue of people and baggage are used as the degree of change of the monitored subject's state.

Therefore, for example, in the interior of a vehicle such as a bus, unavoidable changes (movements) that depend on the behavior of the vehicle while it is moving are not taken into consideration.

The present invention aims to provide a monitoring system, monitoring method, monitoring control device, and monitoring program that can determine display priorities based on the amount of passenger movement, which changes depending on the behavior of the moving objects, when one monitor monitors video footage from inside the vehicle cabins of multiple moving objects.

The surveillance system of the present invention is characterized in that it acquires images taken by a photographing unit that photographs the cabins of each of a plurality of moving bodies, acquires the amount of movement of passengers in each of the cabins of each of the plurality of moving bodies per certain period of time based on the acquired images, calculates a priority such that the greater the acquired amount of movement, the higher the priority indicating the degree of need to monitor the passengers , and when determining the target to be displayed on a display device that sequentially switches and displays the cabins of each of the plurality of moving bodies at a display switching period, increases the display frequency as the priority increases .

The monitoring method of the present invention is characterized by extracting passengers based on image information obtained by photographing the passenger compartments of multiple mobile bodies operating over a specified distance with passengers on board, obtaining the amount of movement of the passengers for each fixed period of time, calculating a priority such that the greater the amount of movement, the higher the priority indicating the degree of need to monitor the passengers, and when determining what to display on a display device that sequentially switches and displays the passenger compartments of multiple mobile bodies at display switching times, the higher the priority , thereby monitoring the status of the passenger compartments of the mobile bodies in chronological order.

The monitoring device of the present invention comprises a monitoring control unit that extracts passengers from image information taken in the passenger compartments of each of a plurality of moving bodies that operate over a specified distance with passengers on board, obtains the amount of movement of the passengers for each fixed period of time, and calculates the amount of movement so that the greater the amount of movement, the higher the priority indicating the degree of need to monitor the passengers; a display device that sequentially switches and displays the passenger compartments of the plurality of moving bodies at display switching times; and a monitoring server that monitors the status of the passenger compartments of the moving bodies in chronological order, the monitoring control unit being equipped with a determination unit that determines the passenger compartments of the moving bodies to be displayed on the display device with a higher display frequency the higher the priority calculated by the monitoring control unit.

The monitoring program according to the present invention is characterized in that it causes a computer to function as the monitoring control unit and the monitoring server of the above-mentioned monitoring device.

According to the present invention, when one monitor monitors the video images inside the vehicle cabins of multiple moving objects, the display priority can be determined based on the amount of movement of passengers, which changes depending on the behavior of the moving objects.

1A and 1B are diagrams illustrating a community bus as a mobile body according to a first embodiment of the present invention, in which FIG. 1 is a diagram showing a vehicle monitoring system including a monitoring control device mounted on a community bus in accordance with a first embodiment, and a monitoring server that aggregates information from a plurality of monitoring control devices and displays monitoring images on a display device. 2 is a control block diagram in which processes for monitoring the interior of a vehicle are classified by function in the vehicle monitoring system according to the first embodiment. FIG. 5 is a control flowchart showing a priority calculation routine executed by a monitoring control device mounted on each community bus for monitoring the interior of the vehicle cabin in the vehicle monitoring system according to the first embodiment. 5 is a control flowchart showing a display target selection routine executed by the surveillance server for monitoring the interior of a vehicle cabin in the vehicle surveillance system according to the first embodiment. 5 is a timing chart showing switching control of an image displayed on a display device according to the first embodiment. 3 is a front view of a vehicle interior image divided and displayed on the display device according to the first embodiment. FIG. FIG. 8 is a front view of a display device according to a first modified example of the first embodiment, in which an image at the time of an abnormality and a current image are displayed using multiple screens (two screens) for a vehicle with a high degree of urgency, as compared to FIG. 7 . 1A to 1C are plan views showing passenger movement conditions in the cabins of different vehicles according to a first example of the first embodiment. FIG. 11 is a priority characteristic diagram relating to Example 2 of the first embodiment, which is a diagram of priority characteristics resulting from passenger movement over time in a plurality of (12) vehicles. This is a priority characteristic diagram relating to Example 2 of the first embodiment, which clearly shows passenger movement (sudden increase in priority) over time in multiple (13) vehicles. FIG. 12 is a priority characteristic diagram according to Example 2 of the first embodiment, in which an urgent display threshold value is set for the priority level in contrast to FIG. 11 . FIG. 11 is a plan view showing the movement of passengers in the cabins of different vehicles according to a third embodiment of the first embodiment, and FIG. 12 is a plan view showing the case where the boarding and alighting door is open in the vehicle shown in FIG. FIG. 11 is a control block diagram in which the processes (weighting addition) for monitoring the interior of a vehicle are classified by function in the vehicle monitoring system according to the second embodiment. FIG. 11 is a plan view illustrating a state in which a community bus is operating (traveling) along a predetermined operating route according to a fourth embodiment of the second embodiment. An example of weight setting for Example 6 of the second embodiment, in which (A) to (C) are plan views showing the movement of passengers in the cabins of different vehicles, and in particular (C) showing the situation when the vehicle suddenly decelerates. 13A and 13B are plan views of a vehicle interior in which areas with different weighting levels are provided, showing an example of weight setting according to Example 7 of the second embodiment. FIG. 13 is a plan view of the interior of the vehicle showing an example of weight setting according to Example 7 of the second embodiment, in which weighting is applied depending on the area in which passengers move. FIG. 13 is a plan view of the interior of the vehicle cabin in which different weighting is applied for each of the weighting examples for Example 8 of the second embodiment, where (A) shows the case where the passenger is an adult, and (B) shows the case where the passenger is a child or an elderly person. 1A is a schematic diagram of a vehicle monitoring system according to a first embodiment of the present invention, and FIG. 1B is a communication sequence in the vehicle monitoring system of FIG. 1A. 20A is a schematic diagram of a vehicle monitoring system according to a second system configuration example of the present invention (a modified example of FIG. 20), and FIG. 20B is a communication sequence in the vehicle monitoring system of FIG. FIG. 13 is a front view of a large bus on which a plurality of cameras are installed and which is applied as a moving body according to the third embodiment.

Figure 1 (A) shows a community bus 10 as a mobile body according to the first embodiment. The purpose of the community bus 10 is to carry passengers P (see Figure 9, etc., for example) and transport them to their destination.

A community bus 10 is often a bus that is smaller (has a smaller passenger capacity) than a regular route bus and travels relatively short distances, used as a means of transportation for local residents, but there is no established definition. In other words, the term "mobile vehicle" includes buses other than the community bus 10, taxis, and passenger vehicles in general, including trains, but the following explanation will use the community bus 10 as an example. Note that in some cases it may be referred to as a vehicle.

The community bus 10 applied in the first embodiment is a small mobile vehicle with a capacity of six passengers that travels round trip or makes a circular trip along a predetermined route and is operated by automatic driving.

Autonomous driving refers to a driving automation level equivalent to level 3 or higher in Non-Patent Document 1 shown below, and manual driving refers to a driving automation level equivalent to level 2 or lower in Non-Patent Document 1.

Non-patent document 1 is "Public-Private ITS Initiative Roadmap 2020, Strategic Conference for Promoting the Use of Public and Private Data, Strategic Headquarters for the Promotion of an Advanced Information and Telecommunications Network Society, Cabinet Office, July 15, 2020, p. 23."

The community bus 10 may be either manually driven by a driver or automatically driven, but unless otherwise noted, the following describes an example of an automatically driven vehicle.

The community bus 10 can be boarded by passengers who have made a reservation in advance through the reservation management system 28 (see Figure 2). Details of the reservation management system 28 will be omitted as it is a well-known technology, but at least when a reservation is made, identification information that can identify the person making the reservation (passenger P) and reservation information are obtained, and the identification information and reservation information are stored in a reservation database 28A. Note that instead of a reservation-based system, a route bus or other operating system that allows passengers to board and disembark without a reservation may also be used.

A person who has made a reservation can board the community bus 10 by arriving at the bus stop that he or she has reserved and waiting there before the departure time. Note that boarding the bus by reservation is not required. In other words, passenger P can board the bus by arriving at the bus stop and waiting there before the departure time.

As shown in Fig. 1A, a community bus 10 is provided with a sliding door 12 in the center of the side. The opening and closing operation of this door 12 is also controlled as one of the automatic driving controls. The door 12 may have other opening and closing structures such as a bellows type door seen on route buses, a folding type door, a double door door, etc. Furthermore, the door 12 is not limited to being automatically opened and closed, and may be opened and closed manually by a passenger P or the like operating a button.

As shown in FIG. 1B, the interior of the community bus 10, which is an example of a passenger compartment, has three-seater seats 14 arranged along the rear wall and the right side wall, and passengers P board when the door 12 is opened and move to their seats.

Here, as shown in FIG. 2, the community bus 10 according to this embodiment is equipped with a monitoring control device 18 that monitors the status of passengers P inside the vehicle, which constitutes part of the vehicle monitoring system S.

As shown in FIG. 2, the monitoring and control device 18 includes a microcomputer 20. The microcomputer 20 includes a CPU 20A, a RAM 20B, a ROM 20C, an input/output unit (I/O) 20D, and a bus 20E such as a database or a control bus that connects these components.

The I/O 20D is mainly connected to an ID reading device 15 for reading passenger information for the purpose of fare payment, a camera unit 16, and a large-capacity storage device 22 such as an HDD or SSD.

In addition, I/O 20D is connected to a network 26 such as the Internet via network I/F 24.

The ID reading device 15 reads the passenger P's identification information and boarding information such as the boarding section by, for example, short-range wireless communication when the passenger P who is getting on or off the community bus 10 holds up the ticket (IC card, etc.) held by the passenger P over a specified reading position. The monitoring control device 18 compares this information with the information (identification information and reservation information) stored in the reservation database 28A of the reservation management system 28 via the network 26. Note that it is not necessary to read the passenger P's identification information and boarding information such as the boarding section, and the necessary information may be obtained from image information captured by the camera unit 16.

On the other hand, the camera unit 16 is attached, for example, to the ceiling surface of the community bus 10 and captures images of almost the entire interior of the vehicle.

The monitoring control device 18 extracts a person (passenger P) based on the image information captured by the camera unit 16, and calculates the state of passenger P, i.e., the amount of movement of passenger P.

In addition, by linking the ID reading device 15 and the camera unit 16, the microcomputer 20 can link passengers P at a specific reading position with the identification information read from the ticket, and thereafter track the position of each passenger P within the vehicle.

(Means of extracting passenger P)

One technique that can be applied to extract passengers P is to classify objects from image information captured by the camera 16, and three examples are shown below. Note that techniques for extracting passengers P are not limited to these three examples. The first example is "extraction by background subtraction," the second example is "extraction by object detection," and the third example is "extraction by semantic segmentation."

(Example 1) Extraction using background subtraction

Interior image information of the community bus 10 is acquired in advance and stored as a background image. By taking the difference between this background image and the currently captured image information, a differential image (object) can be obtained. In addition, by comparing this differential image (object) with the seat 14, the size of the object can be recognized.

Object recognition using differential images can use machine learning methods such as neural networks and pattern recognition using supervised learning (SVM, "Support Vector Machine").

(Example 2) Extraction by object detection

Object detection is also known as a technology that directly identifies what is shown where in a single frame of image information. Object detection is mainly performed using a multi-layer neural network (DNN), and examples of such methods include SSD, Yolo, and M2Det, which can obtain the area occupied by an object by using a "bounding box" that surrounds the object with a rectangle.

(Example 3) Extraction using semantic segmentation

Semantic segmentation is a technique for classifying class attributes for each pixel.

For example, a given image is classified into attributes (e.g., people, floor, seats, windows, etc.), and by applying this inside the cabin of the community bus 10, passenger P can be identified.

As shown in FIG. 2, a monitoring server 30, which constitutes the vehicle monitoring system S together with the monitoring control device 18, is connected to the network 26.

The monitoring server 30 includes a microcomputer 32. The microcomputer 32 includes a CPU 32A, a RAM 32B, a ROM 32C, an input/output unit (I/O) 32D, and a bus 32E such as a database and a control bus that connects these components.

A display device 34 and a large-capacity storage device 36 such as an HDD or SSD are connected to the I/O 32D.

In addition, I/O32D is connected to network 26 via network I/F38.

Here, the vehicle monitoring system S in the first embodiment calculates the movement amount (total) of passengers P inside the vehicle cabin based on monitoring by the monitoring control devices 18 installed in multiple community buses 10, and sends it to the monitoring server 30 via the network 26 (wireless communication is used between the community buses 10 and the network 26). The monitoring server 30 calculates the priority for display on the display device 34 based on the received movement amount, and selects a portion of the image of the interior of the community bus 10 in order of highest priority and displays it on the display device 34.

In the first embodiment, 100 community buses 10 are monitored, and the display device 34 simultaneously displays four interior images on each of the four divided screens, and periodically switches the display object (interior image) on each screen (see Figure 7, described in detail later).

Figure 3 is a control block diagram in which the process of selecting the display target for the display device 34, which depends on the amount of movement of the passenger P, in the vehicle monitoring system S is classified by function. Note that each block in Figure 3 does not limit the hardware configuration. For example, the vehicle monitoring program stored in the ROM 20C, 32C (or the large-capacity storage device 22, 36) of the microcomputer 20, 32 may be executed by the CPU 20A, 32A, and some or all of the process may be operated by software.

(Monitoring and control device 18)

As shown in FIG. 3, the monitoring control device 18 installed on each community bus 10 has an image acquisition unit, and when the image acquisition unit 50 acquires an image captured by the camera unit 16, it sends it to an image analysis unit 52. The image analysis unit 52 analyzes the captured image (e.g., image processing suitable for passenger extraction, etc.) and sends it to a passenger extraction unit 54. The passenger extraction unit 54 extracts passengers P based on the object classification technology described above.

The passenger extraction unit 54 is connected to the passenger movement amount calculation unit 56, and calculates the movement amount for each extracted passenger P.

The most accurate interval (resolution) for calculating the amount of movement is, for example, the image reading interval of the camera unit 16, but considering the purpose of remotely monitoring passenger movements, it is also possible to calculate a relatively rough amount of movement, for example, by accumulating the amount of movement in units of about one second. Note that the camera unit 16 needs to monitor images when an abnormality occurs, so it requires an image reading interval and resolution necessary for monitoring.

The passenger-specific movement amount calculation unit 56 is connected to the priority calculation unit 58, and the movement amount calculated for each passenger is sent to the priority calculation unit 58. The priority calculation unit 58 calculates the total of the movement amount calculated for each passenger based on formula (1), and the total is applied as the priority.

Priority is a numerical evaluation that clearly determines the importance of monitoring, and can also be thought of as a risk value.

The priority calculation unit 58 is connected to the priority sending unit 60, which sends the priority calculated by the priority calculation unit 58 from the communication unit 25 (24) via the network 26 by wireless communication to the monitoring server 30.

The monitoring server 30 is equipped with a vehicle-specific priority acquisition unit 62. The vehicle-specific priority acquisition unit 62 acquires the priority from each community bus 10 via the network 26 and the communication unit 39 (38).

The vehicle-specific priority acquisition unit 62 is connected to the priority comparison unit 64. The priority comparison unit 64 compares the priorities of each community bus 10 acquired from the vehicle-specific priority acquisition unit 62, and sends the comparison result to the display vehicle selection unit 66.

A display switching timer 68 is connected to the display vehicle selection unit 66, and basically, based on a switching instruction signal received periodically from the display switching timer 68, four community buses 10 are selected in order of priority and sent to the display switching instruction unit 70.

The display switching instruction unit 70 is connected to the display device 34 and switches the image of the interior of the community bus 10 that is displayed.

The operation of the first embodiment will be explained below with reference to the flowcharts in Figures 4 and 5.

Figure 4 is a control flowchart showing the priority calculation routine executed by the monitoring and control device 18 installed on each community bus 10.

In step 100, it is determined whether or not operation has started. If the determination in step 100 is negative, the routine ends. If the determination in step 100 is positive, the routine proceeds to step 102.

In step 102, images captured by the camera unit 16 are acquired, and then the process proceeds to step 104, where image analysis processing is performed. In the image analysis processing, passengers P are extracted from the captured image information, and their respective positions are identified.

In the next step 106, the amount of movement is calculated for each passenger P. The amount of movement is the difference in the position of the target passenger P at a predetermined time. The interval for calculating the amount of movement is, for example, approximately one second.

In the next step 108, the total number of movements calculated for each passenger P, i.e., the priority, which is the cumulative value of the movements of all passengers P, is calculated, and the process proceeds to step 110.

In step 110, it is determined whether it is time to send the priority. This priority sending time may be at regular intervals, with the shortest time being the priority calculation in step 108, or when the priority reaches or exceeds a predetermined value.

If the result of the judgment in step 110 is negative, it is determined that it is not time to send the priority, and the process returns to step 102 and repeats the above process. If the result of the judgment in step 110 is positive, it is determined that it is time to send the priority, and the process proceeds to step 112, where the priority (cumulative value) is sent to the monitoring server 30, and the process proceeds to step 114.

In step 114, it is determined whether the operation has ended. If the determination in step 114 is negative, it is determined that the operation is continuing, and the process returns to step 102 and the above process is repeated. If the determination in step 114 is positive, it is determined that the operation has ended, and the process proceeds to step 116. When it is confirmed in step 116 that all passengers have disembarked, this routine ends.

Figure 5 is a control flowchart showing the display target selection routine executed by the monitoring server 30.

In step 120, the initial processing is executed. The initial processing may be executed when the display target selection control program in the monitoring server 30 is started, or when work starts each day.

In the next step 122, it is determined whether it is time to switch the display. In steady state, the time to switch the display occurs after a certain period of time has elapsed. If the determination in step 122 is negative, the process proceeds to step 134, where it is determined whether to end the display (described below).

If the result of step 122 is positive, it is determined that it is time to switch the display, and the process proceeds to step 124.

In step 124, the priority of the currently displayed community bus 10 is reset, then the process proceeds to step 126 to obtain the priority from each community bus 10, and then the process proceeds to step 128.

In step 128, the priorities received from each community bus 10 are compared (comparing the numerical values), and then the process proceeds to step 130, where a display target is selected from the comparison results of step 128, and the process proceeds to step 132.

In step 132, a process is executed to switch the image displayed on the display device 34 to an image of the interior of the community bus 10, which is the display target selected in step 130, and the process proceeds to step 134.

The process also proceeds to step 134 if the determination in step 122 is negative, and a decision is made as to whether or not to end the display. For example, it is decided whether or not it is the end of the working day. If the determination in step 134 is negative, the process proceeds to step 122 to continue the display, and the above process is repeated.

Also, if the answer is yes in step 134, this routine ends.

Note that the display switching occurs after a certain period of time has elapsed under normal conditions (see step 122), but if, for example, an abnormal signal or the like is received from a specific community bus 10, it is preferable to display an image related to that specific community bus 10 regardless of whether step 122 is successful.

Figure 6 is a timing chart of display switching on the display device 34. The display screen of the display device 34 is divided into four sections (top left, top right, bottom left, and bottom right), and display switching control is performed independently for each section. The community bus 10 is identified by the serial number in parentheses at the end.

As shown in FIG. 6, community bus 10 (No. 10) is displayed in the upper left of the screen, community bus 10 (No. 1) is displayed at display switching time t1, community bus 10 (No. 5) is displayed at display switching time t2, and community bus 10 (No. 23) is displayed at display switching time t3.

As shown in FIG. 6, community bus 10 (No. 64) is displayed in the upper right corner of the screen, community bus 10 (No. 2) is displayed at display switching time t1, community bus 10 (No. 6) is displayed at display switching time t2, and community bus 10 (No. 76) is displayed at display switching time t3.

As shown in FIG. 6, community bus 10 (No. 38) is displayed in the lower left corner of the screen, community bus 10 (No. 3) is displayed at display switching time t1, community bus 10 (No. 7) is displayed at display switching time t2, and community bus 10 (No. 27) is displayed at display switching time t3.

As shown in FIG. 6, community bus 10 (No. 72) is displayed in the lower right corner of the screen, community bus 10 (4) is displayed at display switching time t1, community bus 10 (No. 8) is displayed at display switching time t2, and community bus 10 (No. 86) is displayed at display switching time t3.

Figure 7 shows an example of a display in which images of four community buses 10 are split and displayed on the display device 34 based on the timing chart in Figure 6.

The display device 34 is divided into four display areas of equal size, and each area displays a different image of the interior of the community bus 10 (here, an image that combines the interior of the community bus 10 with the outline of the side wall of the community bus 10 through image processing).

The current time is displayed on the periphery of the partitioned display areas (top right in FIG. 7), and the vehicle identification number and the time of photographing by the camera unit 16 are displayed in the top right of each display area. The reason this differs from the current time is that the image capture time is delayed from the current time due to communication delays or the like in the wireless communication section between the community bus 10 and the network 26. Also, when a still image is displayed, the image continues to be displayed until the next display change time, so the time when the screen is viewed may be a certain amount of time after the display start time.
This indicates that the image may be a still image at the time of display switching.

The images displayed in the four display areas of the four images may be arranged so that the image with the highest priority is displayed in the display area that is most likely to attract the attention of the observer (for example, the upper right display area).

(Variation 1)

As shown in FIG. 8, each of the four divided areas does not need to be an image of the interior of a different community bus 10. For example, if the lower left and upper left of the display area are images of the interior of the same community bus 10, with the lower left being a still image at the time the display is switched and the upper left being a moving image that changes in real time from the time the display is switched, it becomes easier to grasp the situation of passenger P and the accuracy of passenger P's risk assessment can be improved. More specifically, if a sudden abnormality occurs, a still image of the moment the abnormality occurs and a current moving image are displayed in real time, allowing passengers to simultaneously see at a glance what abnormality has occurred and what the current condition is, which is effective in achieving safe monitoring.

(Example 1)

Figure 9 shows an example (Example 1) of vehicle monitoring using the vehicle monitoring system S of the first embodiment, where (A) to (C) are plan views of the interiors of different community buses 10, and the procedure for determining the display priority for these three community buses 10 is explained.

In Figure 9 (A), two passengers P are on board the community bus 10A. One passenger P (A1) is standing and moves as shown by the dotted arrow a certain time after the black dot, while the other passenger P (A2) is seated and does not move for a certain time before or after. The length of the dotted arrow is proportional to the amount of movement, and passenger P (A1) in Figure 9 (A) has a movement amount of 1 and passenger P (A2) has a movement amount of 0, resulting in a total movement amount of 1, which is priority 1.

In Figure 9 (B), there are three passengers P on the community bus 10B. When the amount of movement of each is calculated, the amount of movement of the seated passenger P (B1) is 0, while the two standing passengers P (B2) and P (B3) have amounts of movement of 5 and 4, respectively, for a total amount of movement of 9, which is a priority of 9.

In Figure 9 (C), five passengers P are on board community bus 10C. When the amount of movement of each is calculated, the seated passengers P (C1), P (C4), and P (C5) have a movement amount of 0, while the two standing passengers P (C2) and P (C3) have a movement amount of 2 and 3, respectively, for a total movement amount of 5, which is a priority of 5.

As a result of the above, the display priority order is community bus 10B → 10C → 10A, and images of the interior of the vehicle, which pose a relatively high risk of passengers falling, can be displayed preferentially on the display device 34.

(Example 2)

Figure 10 shows an example (Example 2) of vehicle monitoring using the vehicle monitoring system S of the first embodiment, and is a priority transition identification diagram for 12 community buses 10(i). Note that the variable i is a serial number for identifying the community bus 10, and is assigned a positive integer starting from 1, for example.

As shown in FIG. 10, at the first display switching time t1, four community buses 10 with high priority are selected as vehicles to be displayed and are displayed on the display device 34. This display continues until the next display switching time t2, but at the time of the display switching time t1, the accumulated priorities are reset.

If there continues to be no (low) risk to passengers P on all community buses 10, the interiors of different community buses 10 will be displayed every four buses in a predetermined order over a certain period of time (see the direct proportionality characteristics of each characteristic diagram in Figure 10).

However, if passenger P moves significantly within the passenger compartment of any community bus 10 (see characteristic diagram X in Figure 10) before the next display switching time t2, the priority of that community bus 10 will increase in a short period of time.

As a result, the community bus 10 in the characteristic diagram X is selected as the vehicle to be displayed preferentially at the display switching time t2.

Similarly, at the display switching time t3, if a passenger P moves significantly inside the vehicle cabin of any of the community buses 10 (see characteristic diagram Y in Figure 10), the priority of that community bus 10 increases in a short period of time, and the community bus 10 in characteristic diagram Y is selected as the vehicle to be displayed preferentially at the display switching time t3.

(Variation 2)

Figure 11 adds characteristic diagram Y of the community bus 10 in which passenger P moves and the priority suddenly increases, and characteristic diagram Z of the community bus 10 in which the state has changed, to Figure 10.

By suddenly increasing the priority, the probability that characteristic diagram Y and characteristic diagram Z will be selected as the display target at the next display switching time increases.

(Variation 3)

The timing of image display switching is not limited to a predetermined fixed interval. As shown in FIG. 12, a priority threshold value S may be set in advance, and the image may be switched to the image of the interior of the vehicle immediately after the priority exceeds this threshold value S.

(Example 3)

Figure 13 shows an example (Example 3) of vehicle monitoring using the vehicle monitoring system S of the first embodiment, where (A) to (C) show images of the interior of different community buses 10, and the procedure for determining the display priority for these three community buses 10 will be explained.

In Figure 13(A), two passengers P are on board the community bus 10A. One passenger P (A1) is standing and moves as shown by the dotted arrow a certain time after the black dot, while the other passenger P (A2) is seated and does not move for a certain time before or after. The length of the dotted arrow is proportional to the amount of movement, and passenger P (A1) in Figure 9(A) has a movement amount of 1 and passenger P (A2) has a movement amount of 0, resulting in a total movement amount of 1, which is priority 1.

In Figure 13 (B), the door 12 of the community bus 10B is open and three passengers P are about to board the community bus 10B. When the movement amount of each passenger is calculated, the movement amount of passenger P (B1) is 4, the movement amount of passenger P (B2) is 5, and the movement amount of passenger P (B3) is 4, for a total movement amount of 13, which is a priority of 13.

In Figure 13 (C), five passengers P are on board community bus 10C. When the amount of movement of each is calculated, the seated passengers P (C1), P (C4), and P (C5) have a movement amount of 0, while the two standing passengers P (C2) and P (C3) have a movement amount of 2 and 3, respectively, for a total movement amount of 5, which is a priority of 5.

As a result of the above, in normal calculations, the display priority order would be community bus 10B → 10C → 10A, but since the doors 12 are open, meaning that community bus 10B is stopped, it is determined that passenger P is not moving due to the shaking of community bus 10B while it is moving, and the priority of community bus 10B is determined to be 0. Naturally, even if the community bus 10 is stationary, passenger P may trip over unevenness in the interior structure and fall over as he moves, so the priority may be calculated regardless of the state of the doors 12.

As a result, the display priority order is community bus 10C → 10A → 10B, and images of the interior of the vehicle, which pose a relatively high risk of passengers falling, can be displayed preferentially on the display device 34.

As described above, according to the first embodiment, when displaying images of the interiors of multiple community buses 10 (100 in the first embodiment) on a display device 34 with a limited number of display areas (four screens in the first embodiment), the images are not simply switched at regular intervals, but rather a priority is calculated according to the amount of movement of passengers P inside the bus, and images are displayed on the display device 34 with priority according to the priority. The priority can be determined by factoring in the amount of relative movement inside the bus of passengers P aboard the community bus 10, and the priority is determined by periodically calculating and adding up the total amount of movement of all passengers P inside the bus over a fixed period of time.

This allows priority monitoring of community buses 10 where there is a high risk of passengers falling inside the vehicle, for example.

In addition, according to the first embodiment and each example (Example 1 to Example 3), the following variations are possible.

It is preferable to display the interior of the community bus 10, and when the elapsed time since the previous display becomes 0, the amount of relative movement within the vehicle of the passenger P of the community bus 10 is also reset to 0.

The priority is displayed inside the community bus 10 and increases in proportion to the product of the number of passengers and the time elapsed since the previous display, but it may also be calculated using, for example, an exponential or logarithmic function with the elapsed time as a variable.

The priority is displayed inside the community bus 10 and increases in proportion to the product of the number of passengers and the time elapsed since the previous display, but it may also be calculated using a formula that increases the priority in proportion to the elapsed time, regardless of the number of passengers.

The amount of relative movement within the vehicle may be the movement of passengers per size of the community bus 10.

It is preferable to use the acceleration sensor of the community bus 10, correct the observed movement of passengers with a general human model movement calculated from the acceleration sensor, and use the corrected value as the in-vehicle relative movement amount.

Vehicle information may be used, and it is not necessary to use the sum of the relative movement amounts within the vehicle of passengers P inside the community bus 10. For example, when the doors are open, there is movement due to passengers getting on and off, so the movement amount of the relevant passengers may not be added, and the sum of the movement amounts occurring in other areas may be used.

Specific locations within the vehicle may be set as locations that are not included in the priority calculation. For example, if an employee of the company that operates the community bus 10 is on board, the employee's movement from the pre-defined area where the employee is staying to check for safety inside and outside the vehicle is not judged to be a dangerous movement, and an unnecessary increase in priority can be prevented.

(Second embodiment)

In the first embodiment, the movement volume of passengers P was simply accumulated to determine the priority of each community bus 10.

On the other hand, in the second embodiment, the amount of movement of passenger P detected based on the images captured by the camera unit 16 is weighted according to various conditions during operation of the community bus 10 and the status of the passengers, and the amount of movement is corrected according to the operating conditions and the status of passenger P.

Figure 14 is a control block diagram showing the processing (weighting) for monitoring the interior of the vehicle in the vehicle monitoring system according to the second embodiment, classified by function.

In the second embodiment, components that are the same as those in FIG. 3 (first embodiment) are designated with the same reference numerals followed by the letter "A," and descriptions of those components are omitted.

As shown in FIG. 14, the priority (risk value) calculated by the priority calculation unit 58A is sent to the weighting correction unit 72.

The weighting correction unit 72 is connected to a vehicle status identification unit 73 and a weighting database 74. The vehicle status identification unit 73 identifies the vehicle status from the operation information and customer information and sends it to the weighting correction unit 72.

The weighting correction unit 72 stores a table that associates weighting factors classified into multiple types of vehicle conditions and passenger conditions with weights for the weighting factors.

The weighting correction unit 72 performs weighting correction based on the type read from the weighting database 74 based on formula (2) and sends the result to the priority sending unit 60A.

Here, the weighting correction unit 72 is connected to a setting information storage unit 76. The weighting correction unit 72 determines the type of weighting to be imported from the weighting database 74 based on the weighting type stored in the setting information storage unit 76. In the setting information storage unit 76, at least one of the weighting types can be stored by setting, and the setting can be changed as appropriate.

Each type of weighting will be explained in detail in Examples 4 to 9.

(Example 4)

Figure 15 is a plan view showing the state in which the community bus 10 is operating (traveling) according to a predetermined operating route.

As shown in FIG. 15, stops A and B are located along road 80 (a road with one lane in each direction) on which community bus 10 travels, and community bus 10 basically stops at stops A and B.

Because of this stop, for example, the community bus 10 will decelerate until it stops and accelerate after it departs, which increases the risk of strain (falling, etc.) on passengers P compared to normal driving (constant speed driving). In addition, some passengers P may stand up from their seats in preparation to get off the bus.

Therefore, in Example 4, a risk area RA is set near bus stops A and B, and weighting is performed in advance in anticipation of entering the risk area RA. For example, depending on the surrounding environment, the weighting is set to 3.0 at bus stop A (point A) and 2.0 at bus stop B (point B), so that weighting is performed before entering the risk area RA. Table 1 shows an example of weighting settings for each point, including points A and B. Note that the weighting values can be changed as appropriate.

When the community bus 10 reaches each point (risk area RA), weighting is performed based on formula (3) from the time it reaches each point (risk area RA), even if the bus does not actually slow down or accelerate, or if passengers P stand up, etc.

Figure 15 assumes that the number and location of passengers P differs at bus stop A and bus stop B while the community bus 10 is in operation.

Just before stop A, the following weighting is applied to two passengers P:

(Passenger P (A1)) Movement amount 1 × weight 3.0
(Passenger P (A2)) Movement amount 0 x weight 3.0
Before the bus stop B, the following weighting is performed for the three passengers P:
(Passenger P (B1)) Movement amount 4 × weight 2.0
(Passenger P (B2)) Movement amount 5 x weight 2.0
(Passenger P (B3)) Movement amount 4 x weight 2.0

When the bus passes stop B, the following weighting is applied to the five passengers P:

(Passenger P (C1)) Movement amount 0 x weight 1.0
(Passenger P (C2)) Movement amount 2 x weight 1.0
(Passenger P (C3)) Movement amount 3 x weight 1.0
(Passenger P (C4)) Movement amount 0 x weight 1.0
(Passenger P (C5)) Movement amount 0 x weight 1.0

In Example 4, monitoring begins after arrival at the risk area RA to which the weight is applied according to the display switching cycle, but by predicting arrival, display switching can be performed from the moment the vehicle enters the risk area RA to which the weight is set.

(Example 5)

In Example 5, by setting weights according to the vehicle's condition, risks to passenger P caused by the vehicle's condition can be detected early and made the risk a target for monitoring (display) can be determined.

The relationship between vehicle condition and weighting is shown in Table 2. Note that the weighting values can be changed as appropriate.

When the state of the community bus 10 becomes a predetermined state (see Table 2), weighting is performed based on equation (4).

(Example 6)

In Example 6, by setting weights according to the vehicle's behavior, risks to passenger P caused by the vehicle's behavior can be detected early and made a target for monitoring (display).

The relationship between vehicle condition and weighting is shown in Table 3. Note that the weighting values can be changed as appropriate.

When the behavior of the community bus 10 becomes a predetermined state (see Table 3), weighting is performed based on equation (5).

When the behavior of the community bus 10 corresponds to multiple predetermined states (see Table 3), for example, the largest weight that matches the conditions may be applied, or the sum of the corresponding weights may be used, or the average of the corresponding weights (arithmetic mean, weighted mean, square mean, etc.) may be used, or the product of the corresponding weights may be used.

Figure 16 is a plan view of the interior of a community bus 10 according to Example 6 of the second embodiment.

Figures 16(A) and (B) are the previously mentioned Figures 9(A) and (B).

In other words, in Figure 16 (A), two passengers P are on board the community bus 10A, one of the passengers P (A1) is standing and moves as shown by the dotted arrow a certain time after the black dot, while the other passenger P (A2) is seated and does not move for a certain time before or after. The length of the dotted arrow is proportional to the amount of movement, and passenger P (A1) in Figure 16 (A) has a movement amount of 1 and passenger P (A2) has a movement amount of 0, resulting in a total movement amount of 1, which is priority 1.

In Figure 16 (B), there are three passengers P on board the community bus 10B. When the amount of movement of each is calculated, the amount of movement of the seated passenger P (B1) is 0, while the two standing passengers P (B2) and P (B3) have amounts of movement of 5 and 4, respectively, for a total amount of movement of 9, which is a priority of 9.

On the other hand, in Figure 16 (C), the community bus 10C has five passengers P on board and is experiencing sudden deceleration.

When calculating the amount of movement for each passenger, due to the sudden deceleration, seated passengers P (C1), P (C4), and P (C5) have a movement amount of 1, while the two standing passengers P (C2) and P (C3) have movement amounts of 1 and 2, respectively, for a total movement amount of 6. However, because the community bus 10 in Figure 16 (C) is in a state of sudden deceleration, each movement amount is multiplied by a weight of 3 based on Table 3. This becomes priority 18.

(Example 7)

In Example 7, the interior of the community bus 10 is divided and weights are set according to the location of passenger P, so that the risk to passenger P caused by the area in which the passenger is located (passenger location) can be detected quickly and the passenger can be targeted for monitoring (display).

In Example 7, the criteria for the compartment of the community bus 10 were set based on a radius centered on the center of the width of the opening and closing door 12. In other words, the maximum weight was assigned to the range within a radius of less than 1 m from the center, and the minimum weight was assigned to the range beyond a radius of 2 m from the center.

The relationship between passenger P's position and weight is shown in Table 4. In Table 4, a weight of 3.0 is given for less than 1m, a weight of 2.0 is given for 1m to less than 2m, and a weight of 1.0 is given for more than 2m, but these values are not limiting.

When the position of passenger P inside the community bus 10 is in a predetermined state (see Table 4), weighting is performed based on equation (6).

Based on Figures 17 (A) and (B), an example of weighting based on passenger position when passenger P moves within the cabin of the community bus 10 is shown.

In Figure 17 (A), passenger P (A1) has a movement amount of 1, but because it is movement in area β, it is multiplied by a weight of 2.0, making it 2; passenger P (A2) has a movement amount of 2, but because it is movement in area γ, it is multiplied by a weight of 1.0, making it 2; passenger P (A3) has a movement amount of 0, making the total movement amount 4, which is a priority of 4.

In Figure 17 (B), passenger P (B1) has a movement amount of 3, but because it is a movement in area α, it is multiplied by a weight of 3.0, which is 9; passenger P (B2) has a movement amount of 2, but because it is a movement in area α, it is multiplied by a weight of 3.0, which is 6; passenger P (B3) has a movement amount of 2, but because it is a movement in area β, it is multiplied by a weight of 2.0, which is 4; passenger P (B4) has a movement amount of 2, but because it is a movement in area β, it is multiplied by a weight of 1.0, which is 6; passenger P (B5) has a movement amount of 0, and the total movement amount is 21, which is a priority of 21.

Next, based on FIG. 18, when weighting when one passenger P moves across areas within the vehicle cabin, the area can be set when the following conditions are met:

(Area setting 1) The area visited at the start of the display switching time (Area setting 2) The area visited at the end of the display switching time (Area setting 3) The area with the largest average value during the display switching period (interval) (Area setting 4) The area visited the most during the display switching period (interval) (Area setting 5) The area that is the start position of movement during the display switching period (Area setting 6) The area that is the end position of movement during the display switching period

As shown in FIG. 18, in the case of area setting 1 and area setting 5, the priority can be set higher when the movement starts in a dangerous area, so it is possible to focus on passenger P who is already in the dangerous area.

As shown in FIG. 18, in the case of area setting 2 and area setting 6, the priority is calculated based on the location at the time of display switching, so that the priority setting can be realized that best reflects the situation in which the image of the interior of the community bus 10 is displayed.

As shown in FIG. 18, in the case of Area Setting 3 and Area Setting 4, even if the user happens to be in a dangerous area for just a moment at the start or end of a periodic priority calculation interval, the priority can be neither over- nor under-calculated, and the priority can be set to reliably reflect the movement trajectory.

(Example 8)

In Example 8, by setting weights according to the type of passenger P, the risk to passenger P caused by the type of passenger P can be detected early and made the passenger P a target for monitoring (display).

The relationship between the type of passenger P and the weight is shown in Table 5. Note that the weighting values can be changed as appropriate.

Weighting is performed based on formula (5) according to the type of passenger P riding on the community bus 10 (see Table 3).

Figure 19 is a plan view of the interior of a community bus 10 according to Example 7 of the second embodiment.

In Figure 19 (A), all passengers P (A1), P (A2), and P (A3) are adult males (no weighting = 1), and in (B), the seated passenger P (B1) is an adult, the standing passenger P (B2) is a child, and the standing passenger P (B2) is an elderly person.

In other words, in Figure 19 (A), passenger P (A1) has a movement amount of 2, passenger P (A2) has a movement amount of 2, and each has a weight of 1, making the total movement amount 4, which is a priority of 4.

On the other hand, in Figure 19 (B), the movement amount of seated passenger P (B1) is 0, standing passenger P (B2) is a child and has a movement amount of 2, which is multiplied by a weight of 3 to become 6, and passenger P (B3) is an elderly person and has a movement amount of 2, which is multiplied by a weight of 2 to become 4, resulting in a total movement amount of 10, which is a priority of 10.

Even if passenger P is an adult, if an unusual object (such as a cane) is extracted through image analysis, a weight corresponding to the unusual object (for example, a cane may be weighted the same as an elderly person) may be assigned.

(Example 9)

Example 9 combines all the weighting types used in Examples 4 to 8.

As shown in equation (8), the weighted results of equations (3) to (7) can be added together, but it is also possible to select between two or more risks (weights) rather than adding them all together.

The weights shown in Tables 1 to 5 are merely examples, and may be changed depending on the operating hours, operating area, etc.

(System construction example 1)

In the above first and second embodiments (including each example), as shown in FIG. 20(A), the monitoring control device 18 mounted on each community bus 10 calculates the amount of movement and calculates the priority (including weighting), notifies the monitoring server 30, and controls the timing of display switching on the display device 34. The communication sequence in such a system is shown in FIG. 20(B).

(System construction example 2)

In contrast, as shown in FIG. 21(A), the system configuration may be such that each community bus 10 transmits images (camera shooting information) captured by the camera unit 16 to the monitoring server 30, which then analyzes the captured images, calculates the amount of movement, and calculates the priority (including weighting), and controls the timing of display switching on the display device 34. The communication sequence in such a system is shown in FIG. 21(B).

(Third embodiment)

In the above first and second embodiments (including each example), a community bus 10 has been described as an example of a moving object, but as shown in FIG. 22, the present invention can also be applied to moving objects such as a so-called large bus 82 (for example, with a capacity of about 40 to 50 people) or a train. In this case, multiple camera units 16 may be installed to capture about 5 to 10 passengers, and each camera unit 16 may be set as an independent priority calculation area (a monitoring and control device 18 must be installed for each camera unit 16), or the total amount of movement obtained from images captured by some or all of the camera units 16 may be used as the priority (a total number of monitoring and control devices 18 may be installed).

In addition, since the interior space of vehicles, including the community bus 10 and large bus 82, varies, the amount of movement may be corrected based on the vehicle size.

In addition, if another sensor or the like detects that an abnormality has clearly occurred in any of the community buses 10, the interior of the community bus 10 in which the abnormality occurred may be displayed, with even higher priority given to the priority based on the amount of movement.

This invention is based on the premise of autonomous driving, so a device required for an autonomous driving control system is used as the imaging unit, but in order to realize the invention, images from a retrofitted camera may also be used.

Note that, although the community bus 10 has been described above as the subject of acceleration, acceleration also occurs in the movement of individual passengers P.

Therefore, the priority of passenger P's movement may also be calculated using passenger P's acceleration as a weighting factor.

Specifically, consider a case where the priority calculation unit 58 calculates the movement amount at 3-second intervals in a scene where the vehicle suddenly decelerates as in Example 7. In this case, if passenger P moves with high acceleration in the first 1 second of the 3 seconds, stops suddenly, moves by a movement amount of 1, and does not move at all in the remaining 2 seconds, the total movement amount is calculated as 1. On the other hand, if another passenger P' starts moving with low acceleration in the 3 seconds, stops with low acceleration, and moves by a movement amount of 1, the total movement amount is also calculated as 1. In this way, when a high acceleration that suggests a sudden behavior is detected in the vehicle cabin, it is considered that there is a high risk of an abnormality occurring in the vehicle, and it is desirable to set the priority high. However, even if passenger P moves in the vehicle with high acceleration in a short time, if other passengers P move with low acceleration within the calculation interval of the movement amount, the passenger P who moved with high acceleration will not be calculated as having a high priority, and there is a possibility that the occurrence of a danger cannot be notified to the monitor.

To solve this problem, the priority calculation unit 58 can shorten the time interval at which it calculates the amount of movement of passenger P. This makes it easier to grasp instantaneous movements (movements with high acceleration), but on the other hand, it can make it easier to erroneously determine normal posture changes (general behaviors such as changing the position of the legs or picking up luggage) as dangerous movements, which may result in the abnormal behavior that one wants to extract being hidden.

Therefore, by calculating the time average of passenger P's past movement amount and using the calculated time average movement amount as the movement amount for calculating the priority, it is possible to grasp instantaneous movement (movement with large acceleration) while suppressing erroneous determination of normal posture changes, and set a priority according to the risk of abnormality occurring inside the vehicle.

P Passenger P, S Vehicle monitoring system, 10 Community bus, 12 Opening and closing door, 14 Seat, 15 ID reading device, 16 Camera unit, 18 Monitoring control device, 20 Microcomputer, 20A CPU, 20B RAM, 20C ROM, 20D Input/output unit (I/O), 20E Bus, 22 Large capacity storage device, 24 Network I/F, 25 Communication unit, 26 Network, 28 Reservation management system, 28A Reservation database, 30 Monitoring server, 32 Microcomputer, 32A CPU, 32B RAM, 32C ROM, 32D Input/output unit (I/O), 32E Bus, 34 Display device, 36 Large capacity storage device, 38 Network I/F, 39 Communication unit, 50 Image acquisition unit, 52 Image analysis unit, 54 Passenger extraction unit, 56 Passenger-specific movement amount calculation unit, 58 Priority calculation unit, 60 Priority sending unit, 62 Vehicle priority acquisition unit, 64 Priority comparison unit, 66 Display vehicle selection unit, 68 Display switching timer, 70 Display switching instruction unit, 72 Weighting correction unit, 74 Weighting database, 76 Setting information storage unit, 80 Road, A Bus stop, B Bus stop, RA Risk area

Claims (13)

(16) acquiring photographed images taken by a photographing unit that photographs the passenger compartments of each of the plurality of moving bodies;
(56) acquiring a movement amount of passengers present in each of the passenger compartments of the plurality of moving bodies per certain period of time based on the acquired photographed images;
Calculating the priority such that the greater the amount of movement obtained, the higher the priority indicating the degree of need to monitor the passenger (58);
When determining a display target on a display device that sequentially switches and displays the guest rooms of each of the plurality of moving bodies at a display switching time, the higher the priority, the higher the display frequency (64, 66).
Surveillance system (S). extracting passengers based on image information obtained by photographing the passenger compartments of a plurality of moving bodies operating within a predetermined section with the passengers in the passenger compartments;
Obtain the amount of movement of the passenger for a certain period of time;
calculating a priority indicating a degree of necessity for monitoring the passenger such that the greater the amount of movement, the higher the priority;
When determining the display target on a display device that sequentially switches between the rooms of a plurality of moving bodies at a display switching time, the higher the priority, the higher the display frequency ;
Time-series monitoring of vehicle cabin conditions,
Monitoring methods. a monitoring control unit that extracts passengers from image information captured in the passenger compartments of each of a plurality of moving bodies that operate within a predetermined section with passengers in the passenger compartments, obtains the amount of movement of the passengers per certain period of time, and calculates the amount of movement so that the greater the amount of movement, the higher the priority indicating the degree of necessity for monitoring the passengers;
a display device that sequentially switches and displays a plurality of guest rooms of the moving body at display switching times, and a determination unit that determines the guest rooms of the moving body to be displayed on the display device with a higher display frequency as the priority calculated by the monitoring control unit increases , and a monitoring server that monitors the status of the guest rooms of the moving body in chronological order;
A monitoring device having the above configuration. The monitoring device according to claim 3, wherein the monitoring control unit is provided in each of a plurality of moving objects and notifies the monitoring server of the priority. The monitoring device according to claim 3, wherein the monitoring control unit is provided at a monitoring base where the monitoring server is provided, and aggregates the photographed information of the rooms of multiple moving objects. The monitoring control unit determines the priority by adding a weight assigned to a predetermined weighting factor to the amount of movement. The monitoring device according to any one of claims 3 to 5. The monitoring device according to claim 6, wherein the weighting factor is at least one of the following information: location information of the operating section of the mobile body, status information of the mobile body, behavior information of the mobile body, position information of passengers on the mobile body, and type information of passengers on the mobile body. The monitoring device according to any one of claims 3 to 7, wherein the monitoring control unit corrects the amount of movement according to the size of the passenger compartment of the moving body when acquiring the amount of movement. A monitoring device according to any one of claims 3 to 8, in which the display area of the display device is divided into multiple areas, each of which can control the display target independently, and the monitoring server determines the position of the divided display area based on the priority. A monitoring device according to any one of claims 3 to 9, in which the display area of the display device is divided into multiple areas, each of which can control the display target independently, and the monitoring server displays images of the interior of the same vehicle at different times in two or more divided display areas. The monitoring device according to any one of claims 3 to 10, wherein, when an abnormality occurs in any of the moving objects, the monitoring server displays the guest room of the moving object in which the abnormality occurred, with a higher priority than the priority based on the amount of movement. The monitoring device according to any one of claims 3 to 11, wherein the monitoring server displays the interior of the vehicle of a moving object whose priority exceeds a predetermined threshold value, regardless of the display switching timing, when the priority value exceeds a predetermined threshold value. Computer,
The monitoring device according to any one of claims 3 to 12 functions as the monitoring control unit and the monitoring server.
Surveillance program.