JP2017212647A - Seat monitoring apparatus, seat monitoring system, and seat monitoring method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a seat monitoring apparatus capable of protecting privacy of a user imaged in a picture obtained by imaging a monitoring area while visualizing a use state on unauthorized use and the like of each seat on the picture obtained by imaging the monitoring area.SOLUTION: A seat monitoring apparatus comprises: a seat management information acquisition unit 53 for acquiring seat management information on the propriety of use for each seat; a sitting detection unit 54 for detecting a user sitting on each seat on the basis of a result of person detection with respect to an imaged picture obtained by imaging a monitoring area; a seat use information acquisition unit 55 for acquiring seat use information on illegal use of each seat, on the basis of the seat management information and a sitting detection result; and a monitoring picture generation unit 58 that on the basis of the seat use information, changes a display mode of an identification picture for each user arranged on the imaged picture to generate a monitoring picture in which a use situation for each seat is visualized and picture processing for privacy protection is performed on the user's person area on the imaged picture.SELECTED DRAWING: Figure 9

Description

  The present invention relates to a seat monitoring device, a seat monitoring system, and a seat monitoring method that generate output information that visualizes the use status of a seat by a user based on an image obtained by imaging a monitoring area where a plurality of seats are arranged. .

  In recent years, trains have been provided with seats that can be used by paying seat usage fees so that passengers can comfortably use free seats. In such trains, in order to prevent unauthorized use (illegal boarding) of seats, in-car check was performed by the crew member to check the ticket of the passenger using the seat. Therefore, a system for performing in-car inspection using an IC card is operated.

  However, such a system that performs in-car inspection using an IC card can unmanned in-car inspection, but cannot prevent unauthorized use of the seat itself. For this reason, the only way to know the status of unauthorized use of seats is for the crew to go around the car. On the other hand, it is conceivable to take a picture of a seat in the car with a camera, but it is impossible to immediately grasp the situation of unauthorized use of the seat only with the taken image. Therefore, a technique for visualizing and displaying the status of unauthorized use of the seat is desired so that the supervisor can immediately grasp the status of unauthorized use of the seat.

  As for the technology to visualize and display the situation of unauthorized use of such seats, conventionally, the inside of a train is imaged with a camera, and passengers using the seat are detected from the image in the vehicle, Based on the detection result and the reservation information on the reserved seat, it is determined that the seat is illegally used, that is, the passenger is using a seat that has not been reserved, and an object representing the unauthorized use of the seat is determined. A technique for displaying an image in a vehicle in a superimposed manner is known (see Patent Document 1).

  In addition, the audience seats in a movie theater or stadium are imaged with a camera, the audience using the seats is detected from the images of the audience seats, and the seats used by the audience and the reservation status thereof are determined by the audience. A technique is known in which an observer can easily identify that a spectator is using an unreserved seat by displaying the image on a seat image (see Patent Document 2).

JP 2011-210125 A Japanese Patent No. 4367177

  Now, as in the prior art, by displaying the captured image obtained by capturing the monitoring area, the user can grasp the actual situation of the monitoring area. However, by displaying the captured image, the user's It is desirable to avoid infringement of privacy. In particular, for unauthorized users, it is necessary to display captured images as they are to identify individuals, but for legitimate users, there is no need to identify individuals, so always protect privacy. Is desired. However, the above conventional technique has no consideration on protecting the privacy of the user in the captured image, and there is a problem that the privacy of the user cannot be protected.

  Accordingly, the present invention provides a seat monitoring device that can visualize the usage situation regarding unauthorized use of each seat on an image obtained by imaging the monitoring area and protect the privacy of the user shown in the image obtained by imaging the monitoring area. The main object of the present invention is to provide a seat monitoring system and a seat monitoring method.

  A seat monitoring device according to the present invention is a seat monitoring device that generates output information that visualizes the use status of a seat by a user based on an image obtained by imaging a monitoring area in which a plurality of seats are arranged. A seat usage information acquisition unit that acquires seat usage information relating to the usage status of the vehicle, and a display mode of an identification image for each user arranged on a captured image obtained by imaging the monitoring area based on the seat usage information And a monitoring image generation unit that generates a monitoring image in which a privacy protection image process is performed on at least a part of a person area of the user in the captured image, And an output information generation unit that generates the output information including:

  The seat monitoring system of the present invention is a seat monitoring system that generates output information that visualizes the use status of a seat by a user based on an image obtained by imaging a monitoring area in which a plurality of seats are arranged. A camera that picks up an area and detects a person in the picked-up image, a server device that generates the output information that visualizes the use status of a seat by a user, and a user terminal device that displays a browsing screen based on the output information The server device includes: a seat usage information acquisition unit that acquires seat usage information related to a seat usage status by a user; and for each user arranged on the captured image based on the seat usage information The usage status of each seat is visualized by changing the display mode of the identification image, and the person area of the user in the captured image is visualized. A monitoring image generation unit that generates a monitoring image that has been subjected to privacy-protected image processing for at least a part, and an output information generation unit that generates the output information including the monitoring image .

  Also, the seat monitoring method of the present invention is a seat that causes an information processing device to perform processing for generating output information that visualizes the use status of a seat by a user based on an image obtained by imaging a monitoring area where a plurality of seats are arranged. It is a monitoring method, the seat usage information regarding the seat usage status by the user is acquired, and based on the seat usage information, the display mode of the identification image for each user arranged on the captured image obtained by imaging the monitoring area The use status for each seat is visualized by changing the image, and a monitoring image in which privacy-protected image processing is performed on at least a part of the person area of the user in the captured image is generated, and the monitoring image is It is set as the structure which produces | generates the said output information containing.

  According to the present invention, since the usage situation of each seat is visualized in the monitoring image, the user can immediately grasp the usage situation of each seat. Further, since privacy-protected image processing is performed, it is possible to protect the privacy of the user shown in the monitoring image.

Overall configuration diagram of a seat monitoring system according to the first embodiment Explanatory drawing which shows the arrangement | positioning condition of the camera 1 in the vehicle of a train Explanatory drawing showing a route map image display screen Explanatory drawing showing the train map image display screen Explanatory drawing which shows a vehicle map image display screen Explanatory drawing showing the seat map image display screen Explanatory drawing showing the monitoring image display screen The block diagram which shows the hardware constitutions of the camera 1, the server apparatus 2, and the user terminal device 3 Functional block diagram of camera 1 and server device 2 Functional block diagram of the camera 1 and the server device 2 according to a modification of the first embodiment Explanatory drawing which shows the monitoring image display screen which concerns on 2nd Embodiment. Functional block diagram of camera 1 and server device 2 according to the second embodiment Functional block diagram of the camera 1 and the server device 2 according to a modification of the second embodiment

  A first invention made to solve the above-described problem is a seat monitoring device that generates output information that visualizes the use status of a seat by a user based on an image obtained by imaging a monitoring area where a plurality of seats are arranged. A seat usage information acquisition unit that acquires seat usage information related to a seat usage status by a user, and an identification image for each user arranged on a captured image obtained by imaging the monitoring area based on the seat usage information A monitoring image that visualizes the usage situation for each seat by changing the display mode of the image and generates a monitoring image in which privacy-protected image processing is performed on at least a part of the person area of the user in the captured image It is set as the structure provided with the production | generation part and the output information generation part which produces | generates the said output information containing the said monitoring image.

  According to this, since the usage status of each seat is visualized in the monitoring image, the user can immediately grasp the usage status of each seat. Further, since privacy-protected image processing is performed, it is possible to protect the privacy of the user shown in the monitoring image.

  The identification image may be arranged on or near the person area of the user in the monitoring image. Changing the display mode of the identification image includes changing the display form such as the display color of the identification image. The use status for each seat can be visualized by changing the display form of the identification image and changing whether or not to display the identification image.

  In addition, the second invention further includes a seat management information acquisition unit that acquires seat management information regarding availability of each seat, and the presence of the user in each seat based on the result of person detection for the captured image. A seat detection unit for detecting, based on the seat management information and the detection result of the presence detection unit, the seat use information acquisition unit determines whether each seat is illegally used, The seat use information related to unauthorized use is acquired.

  According to this, the user can immediately grasp whether or not the unauthorized use. In this case, whether each seat is used illegally or not by comparing the seat usage information on whether or not seat usage is permitted, that is, whether or not seat usage is permitted, and the detection result of the presence detection unit for each seat. It can be determined whether or not.

  According to a third aspect of the present invention, the monitoring image generation unit performs image processing for privacy protection on a person area of a user who is not unauthorized use based on the seat usage information, and The privacy protection image processing is not performed on the person area.

  According to this, the privacy of a legitimate user can be protected. On the other hand, since an unauthorized user is displayed in an unprocessed state, an individual can be identified.

  In addition, according to a fourth aspect of the invention, the monitoring image generation unit generates the monitoring image that has been subjected to any discrimination reduction processing of mosaic processing, blurring processing, and blending processing as the privacy protection image processing. And

  According to this, by performing any one of the distinctiveness reduction processes of the mosaic process, the blurring process, and the blending process, it becomes impossible to identify the individual, so that the privacy of the user can be reliably protected.

  According to a fifth aspect of the present invention, the monitoring image generation unit superimposes a frame image surrounding at least a part of a user's person area on the distinguishability reduced image on which the distinguishability reduction processing has been performed. And the usage status of each seat is visualized by changing the display form of the frame image based on the seat usage information.

  According to this, the usage state of the seat by the user (for example, whether or not it is unauthorized use) is expressed by the display mode (for example, display color) of the frame image for each user. Can be grasped immediately.

  According to a sixth aspect of the invention, the monitoring image generation unit performs image processing for replacing the user's person area in the captured image with a mask image representing the shape of the person area as the privacy protection image processing. The monitoring image is generated.

  According to this, by replacing the person area in the captured image with the mask image, it becomes impossible to identify the individual, so that the privacy of the user can be reliably protected.

  Moreover, 7th invention sets it as the structure which visualizes the utilization condition for every seat by the said monitoring image production | generation part changing the display form of the said mask image based on the said seat utilization information.

  According to this, the usage state of the seat by the user (for example, whether or not it is unauthorized use) is expressed by the display mode (for example, display color) of the mask image for each user. Can be grasped immediately.

  According to an eighth aspect of the present invention, the monitoring area is in a train of a train on which a seat charge is charged, and the monitoring image generation unit captures when the train starts or stops in the station premises. The monitoring image on which the privacy protection image processing has been performed is generated based on the result of person detection on the captured image.

  According to this, since the movement of the passenger's body is caused by the shaking of the vehicle when the train starts or stops, it is possible to reliably perform person detection based on the moving object detection, and thus image processing for privacy protection for the passenger Can generate a properly implemented monitoring image.

  According to a ninth aspect of the invention, the monitoring area is in the train of a train on which a seat is charged, and the seat management information acquisition unit is at the time of departure or stop of the train within the station premises. The seat management information is acquired.

  According to this, since passengers often sit and leave when the train starts or stops, appropriate seat management information can be acquired.

  The tenth aspect of the invention further generates a seat map image composed of a schematic diagram showing a seat arrangement state in the monitoring area, and based on the seat use information, a seat image representing each seat in the seat map image By changing the display mode, a map image generation unit that visualizes the usage status of the user for each seat is provided.

  According to this, it is possible to immediately grasp the usage status of each seat in the monitoring area.

  In addition, according to an eleventh aspect of the invention, the output information generation unit corresponds to a seat section to which a seat specified by the user belongs in response to an operation input of the user specifying the seat on the screen displaying the seat map image. The output information including the monitoring image is generated.

  According to this, the usage status of each seat is grasped in the seat map image, and by specifying the seat of interest in this seat map image, the monitoring image showing the seat is displayed and the actual situation of the seat of interest is displayed. Since it can be confirmed, it is possible to efficiently check the seat usage status.

  In a twelfth aspect of the invention, the monitoring area is in a train on which a seat where a seat usage fee is charged is arranged, and the seat usage information acquired by the seat usage information acquisition unit is , A seat usage information aggregating unit that aggregates at least one of a train unit and a route unit, a route map image that visualizes a usage situation for each route based on the seat usage information aggregated by the seat usage information aggregating unit, a train A map image generation unit that generates at least one of a train map image that visualizes the usage status of each vehicle and a vehicle map image that visualizes the usage status of each vehicle.

  According to this, the user can grasp the use status of the seat in units of vehicles, trains or routes.

  According to a thirteenth aspect of the present invention, in the initial state, the monitoring image generation unit generates the monitoring image that has been subjected to the privacy protection image processing, and in response to a user operation input for releasing privacy protection, the privacy image is generated. The monitoring image is generated without performing the protection image processing.

  According to this, the user can cancel the privacy protection as necessary, and can display the monitoring image on which the privacy protection image processing is not performed.

  A fourteenth aspect of the present invention is a seat monitoring system that generates output information that visualizes the use status of a seat by a user based on an image obtained by imaging a monitoring area in which a plurality of seats are arranged. A camera that picks up an image and detects a person on the captured image; a server device that generates the output information that visualizes the use status of a seat by a user; a user terminal device that displays a browsing screen based on the output information; The server device includes: a seat usage information acquisition unit that acquires seat usage information related to seat usage status by a user; and identification for each user arranged on the captured image based on the seat usage information Visualize the usage status for each seat by changing the display mode of the image, and at least a part of the person area of the user in the captured image To the monitoring image generation unit in the image processing privacy generates a monitoring image that has been performed, the output information generation unit for generating the output information including the monitoring image, a configuration including a.

  According to this, as in the first aspect of the invention, on the image obtained by imaging the monitoring area, the usage situation regarding unauthorized use of each seat is visualized, and the privacy of the user shown in the image obtained by imaging the monitoring area is protected. be able to.

  According to a fifteenth aspect of the invention, there is provided a seat monitoring method for causing an information processing apparatus to perform processing for generating output information that visualizes a use situation of a seat by a user based on an image obtained by imaging a monitoring area where a plurality of seats are arranged. And the seat usage information regarding the usage status of the seat by the user is acquired, and based on the seat usage information, the display mode of the identification image for each user arranged on the captured image obtained by imaging the monitoring area is changed. The use status for each seat is visualized to generate a monitoring image in which privacy-protected image processing is performed on at least a part of the person area of the user in the captured image, and the monitoring image includes the monitoring image. The output information is generated.

  According to this, as in the first aspect of the invention, on the image obtained by imaging the monitoring area, the usage situation regarding unauthorized use of each seat is visualized, and the privacy of the user shown in the image obtained by imaging the monitoring area is protected. be able to.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is an overall configuration diagram of a seat monitoring system according to the present embodiment.

  This seat monitoring system provides a monitoring image generated based on an image of the inside of a train with the inside of the train as a monitoring area and visualizing the use status of each seat to a user such as a supervisor at the monitoring center. It includes a camera 1, a server device (seat monitoring device) 2, a user terminal device (browsing device) 3, an in-car ticket checking device 4, and a seat management device 5.

  The train is a combination of special vehicles (green trains for ordinary trains) with multiple seats that are free seats and are subject to seat usage fees. In these special vehicles, passengers pay seat usage fees. You can select and use seats freely. The special vehicle is provided with a camera 1 and an in-car ticket checking device 4 in order to monitor the use situation of passengers for each seat. The camera 1 and the in-car inspection device 4 are connected to a closed network such as a VLAN (Virtual Local Area Network) via an in-vehicle network and a wireless communication unit 6 and a wireless base station 7.

  The camera 1 is installed on the ceiling in the train and images a seat provided in the train. In the camera 1, a process of detecting a person from a captured image obtained by capturing a seat in a vehicle is performed, and a captured image and person detection information are output.

  The in-car inspection device 4 is installed on the ceiling above each seat and performs unattended in-vehicle inspection to confirm the seat use ticket (green ticket) purchased by the passenger. The reader / writer 8, the display lamp 9, It has. A passenger purchases a seat use ticket using an IC card at a ticket vending machine in a station, gets on the train, and holds the IC card over the reader / writer 8 of the in-car ticket checking device 4 corresponding to the seat on which the passenger is seated. As a result, the display lamp 9 changes from red to green. As a result, the crew or the like can confirm that the in-car check has been completed. In addition, purchase of a seat use ticket and in-car check can also be performed with a mobile terminal device such as a smartphone.

  The seat management device 5 is connected to the in-car inspection device 4 mounted on each train via a closed network, and collects in-vehicle inspection information from the in-car inspection device 4 regarding whether or not the in-vehicle inspection for each seat has been completed. To do. The seat management device 5 is connected to the server device 2 via the closed network, and seat management information relating to whether or not each seat can be used, that is, whether or not the passenger has the authority to use the seat through the in-vehicle ticket gate. Is provided to the server device 2.

  The server apparatus 2 is connected to each camera 1 of each train via a closed network, and receives a captured image transmitted from each camera 1 of each train. The server device 2 is connected to the user terminal device 3 via the Internet, generates a screen to be browsed by the user, distributes it to the user terminal device 3, and acquires information input by the user on the screen. .

  The user terminal device 3 is configured by a smartphone, a tablet terminal, or a PC. In the user terminal device 3, a browsing screen such as a monitoring image distributed from the server device 2 is displayed. When the user browses the viewing screen such as the monitoring image, it is possible to grasp the seat usage status by the passengers in each vehicle of each train.

  In the present embodiment, the server device 2 detects the presence of a passenger in each seat based on the captured image acquired from the camera 1, and the presence management result and the seat management acquired from the seat management device 5. Based on the information, it is determined whether or not the passenger present in the seat is an illegal boarding depending on the presence or absence of an in-car check using an IC card. In other words, a passenger seated in a seat where the IC card check has been completed is regarded as a valid passenger, and a passenger seated in a seat where the IC card check has not been completed is defined as an unauthorized passenger.

  In this embodiment, the purchase of the seat use ticket and the in-car ticket check are all performed using the IC card, and passengers who have not received the in-car check using the IC card are illegally boarded, but the seat use ticket is purchased with a ticket. You may make it do. In this case, passengers who purchase a ticket for a seat use ticket and get on the board cannot perform in-car check using an IC card. Therefore, it is undecided whether a passenger who has not received the in-car check is valid. In this case, the crew member can change the display lamp 9 from red to green by inputting information relating to the end of the in-car check from a management terminal (not shown) when the ticket of the seat use ticket is confirmed. .

  Next, the arrangement state of the camera in each vehicle of the train will be described. FIG. 2 is an explanatory diagram showing an arrangement state of the cameras 1 in the train vehicle.

  A camera 1 is installed on the ceiling of each vehicle C of the train so as to image a plurality of seats S arranged in the vehicle. In particular, in the example shown in FIG. 2, an omnidirectional camera having a 360-degree imaging range using a fisheye lens is adopted as the camera 1, but a camera having a predetermined angle of view, a so-called box camera, is also possible.

  Next, a screen generated by the server device 2 and displayed on the user terminal device 3 will be described. FIG. 3 is an explanatory diagram showing a route map image display screen. FIG. 4 is an explanatory diagram showing a train map image display screen. FIG. 5 is an explanatory diagram showing a vehicle map image display screen. FIG. 6 is an explanatory diagram showing a seat map image display screen. FIG. 7 is an explanatory diagram showing a monitoring image display screen.

  In this embodiment, a route map image display screen (see FIG. 3), a train map image display screen (see FIG. 4), a vehicle map image display screen (see FIG. 5), a seat map image display screen (see FIG. 6), and Screen information related to the monitoring image display screen (see FIG. 7) is generated by the server device 2, and these screens are displayed on the user terminal device 3.

  When a route of interest is selected on the route map image display screen (see FIG. 3), a transition is made to the train map image display screen (see FIG. 4) relating to the selected route. And if the train which attention is selected on a train map image display screen, it will change to the vehicle map image display screen (refer to Drawing 5) about the selected train. Then, when a vehicle of interest is selected on the vehicle map image display screen, a transition is made to a seat map image display screen (see FIG. 6) relating to the selected vehicle. When a seat of interest is selected on the seat map image display screen, a transition is made to a monitoring image display screen (see FIG. 7) relating to the selected seat.

  First, as shown in FIG. 3, a route map image 101 that visualizes the usage situation (the number of illegal rides) for each route is displayed on the route map image display screen. This route map image 101 is a schematic drawing of the laying status of each route in the target area. By changing the display form of the route image 102 representing each route in the route map image 101, the route map image 101 The usage situation of each is visualized. In the example shown in FIG. 3, the display color of the route image 102 represents the number of illegal boardings for each route.

  Next, as shown in FIG. 4, a train map image 111 that visualizes the use situation (the number of illegal boarding) for each train is displayed on the train map image display screen. This train map image 111 is a schematic drawing of the operation status of a train on the target route. By changing the display form of the train image 112 representing each train in the train map image 111, the train map image 111 The usage situation of each is visualized. In the example shown in FIG. 4, the display color of the train image 112 represents the number of illegal rides for each train.

  Next, as shown in FIG. 5, a vehicle map image 121 that visualizes the usage situation (the number of illegal boarding) for each vehicle is displayed on the vehicle map image display screen. This vehicle map image 121 is a schematic drawing of the knitting status of the vehicle in the target train, and by changing the display form of the vehicle image 122 representing each vehicle in the vehicle map image 121, the vehicle map image 121 The usage status of is visualized. In the example shown in FIG. 5, the display color of the vehicle image 122 represents the number of illegal boardings for each vehicle.

  Next, as shown in FIG. 6, a seat map image 131 for visualizing the use situation (whether it is an unauthorized boarding) for each seat is displayed on the seat map image display screen. This seat map image 131 is a schematic drawing of the seat arrangement status in the target vehicle, and the display form (display color) of the seat image 132 representing each seat in the seat map image 131 is changed. Thus, the usage status of the user for each seat is visualized. In the example shown in FIG. 6, the display color of the seat image 132 represents the presence or absence of unauthorized boarding for each seat. For example, the seat image 132 is displayed in green in a seat where a legitimate passenger is present, and the seat image 132 is displayed in red in a seat where an unauthorized passenger is present. When the seat is empty, the seat image 132 is displayed in white.

  Here, in the present embodiment, in the server device 2, whether or not it is an illegal boarding for each seat based on the captured image and person detection information acquired from the camera 1 and the seat management information acquired from the seat management device 5. The display color of the seat image 132 in the seat map image (see FIG. 6) is determined on the basis of the seat usage information for each seat regarding whether or not this is an unauthorized boarding.

  Further, in the server device 2, the seat usage information for each seat is aggregated for each vehicle, the seat usage information for each vehicle (the number of seats for illegal boarding) is acquired, and based on the seat usage information for each vehicle, the vehicle The display color of the vehicle image 122 in the map image (see FIG. 5) is determined. Moreover, the seat use information for each vehicle is aggregated for each train, and unauthorized boarding information for each train is obtained. Based on the seat use information for each train, the train image 112 in the train map image (see FIG. 4) is obtained. The display color is determined. In addition, the seat usage information for each train is aggregated for each route, the seat usage information for each route is acquired, and the route image 102 in the route map image (see FIG. 3) is acquired based on the seat usage information for each route. The display color is determined.

  Next, as shown in FIG. 7, the monitoring image display screen displays a monitoring image 141 that visualizes the use situation (whether or not it is an illegal boarding) for each seat. In this monitoring image 141, a rectangular frame image 143 is superimposed on the discriminability reduction image 142 and displayed. The monitoring image 141 is desirably viewable as a moving image, but may be viewable as a still image distributed at a predetermined time interval.

  The discriminability-reduced image 142 is generated by performing image processing (identification-reducing processing) for reducing the person's discrimination on the captured image as image processing for protecting the privacy of passengers in the captured image. In the example illustrated in FIG. 7, mosaic processing is performed as the distinguishability reduction processing. In the example shown in FIG. 7, the distinctiveness reduction process is performed on each passenger's face area (an image area in which a person's face is shown).

  In the mosaic processing, the captured image is divided into a plurality of blocks, and the pixel values of one pixel in the block or the average value of the pixel values of each pixel in the block are all changed to a single pixel value. This replaces the pixel value of each pixel.

  In this embodiment, the identification reduction process is performed on the face area that is most effective in reducing the person's identification. However, such a person area (an image area in which a person's body is captured) is performed. ), The distinction reduction process may be performed on the entire person area instead of a part of the Moreover, you may make it implement a discrimination reduction process with respect to the whole captured image.

  In the example shown in FIG. 7, mosaic processing is performed as image processing for reducing the identifiability of a person appearing in a captured image, but blur processing or blend processing may be performed in addition to this.

  The blurring process is a filter process using various filter processes such as a blur filter, a Gaussian filter, a median filter, and a bilateral filter. In addition, negative / positive reversal, color tone correction (brightness change, RGB color balance change, contrast change, gamma correction, saturation adjustment, etc.), binarization, and various image processing such as edge filters can also be used. Is possible.

  The blending process combines (blends) two images in a semi-transmissive state, and combines a predetermined image for synthesis and a captured image based on an α value indicating the degree of synthesis.

  In addition, when performing the distinctiveness reduction process on the entire captured image, the image identification is reduced to such an extent that the human distinctiveness is lost without performing special image processing such as mosaic processing, blurring processing, and blending processing. You may make it perform the process which reduces the resolution. In this case, since it is not necessary to mount a special image processing function, the image processing unit can be configured at low cost and the amount of image data can be reduced, so that the communication load on the network can be reduced.

  The frame image 143 is displayed so as to surround each passenger's head region (an image region in which a person's head is shown). The frame image 143 is generated based on the person frame set in the area where the detected person exists in the person detection performed by the camera 1.

  The frame image 143 is an identification image that represents the usage status (whether or not the vehicle is illegally boarded) for each seat, and the usage status for each seat is visualized by changing the display mode of the frame image 143. In the example illustrated in FIG. 7, the display color of the frame image 143 represents the presence or absence of unauthorized boarding for each seat. For example, the frame image 143 is displayed in green for a legitimate passenger, and the frame image 143 is displayed in red for an unauthorized passenger.

  In the present embodiment, the use status for each seat is expressed by the display color of the frame image 143. However, the display element that expresses the use status is limited to the display color (color shade or color intensity). The usage status can also be expressed by line type (broken line etc.) and thickness. In other words, the drawing may be performed so that at least one display element of hue, color strength, line type, and thickness is different.

  In the present embodiment, the frame image 143 that surrounds the passenger's head is displayed as the identification image for visualizing the use situation (whether the boarding is illegal) for each seat, but the upper body of the passenger is surrounded. The frame image 143 may be displayed, and the frame image 143 may be displayed so as to surround the entire person area, not a part of such a person area. Although the frame image 143 is a rectangle, the frame image is not limited to a rectangle, and may be a figure having another shape such as a circle. Furthermore, the image for visualizing the use situation (whether or not the vehicle is illegally boarded) for each seat is not limited to a frame image that surrounds a part or the whole of the person area, and includes at least figures, symbols, and characters having an appropriate shape. As an image composed of any of these, this image may be displayed inside or near the person area.

  In the example shown in FIG. 7A, privacy protection image processing (discriminability reduction processing) is performed for all persons. In the example shown in FIG. 7B, privacy-protected image processing is performed only for legitimate passengers, and privacy-protected image processing is not performed for unauthorized passengers.

  Here, only one of the monitoring image 141 shown in FIG. 7A and the monitoring image 141 shown in FIG. 7B may be displayed, but the monitoring image 141 shown in FIG. The user may be able to select either the display mode for displaying the image or the display mode for displaying the monitoring image 141 shown in FIG.

  Also, as an initial state, the monitoring image 141 shown in FIG. 7A is displayed, and when the user performs an operation to cancel the privacy protection, the monitoring image 141 shown in FIG. 7B is displayed. Good. In this case, a button for releasing privacy protection may be provided on the monitoring image display screen. Further, only when the user authentication is performed and the user authentication is successful, the user can perform an operation of canceling the privacy protection, and the monitoring image 141 in which the privacy protection image processing is not performed can be browsed. You may make it restrict | limit.

  In addition, in the monitoring image 141 shown in FIG. 7B, it is possible to determine whether the passenger is an unauthorized passenger based on the presence / absence of image processing for privacy protection (identity reduction processing). The frame image 143 for visualizing “No” may be omitted.

  Further, in the monitoring image 141 shown in FIG. 7B, since it is possible to determine whether or not the passenger is an unauthorized passenger by the presence / absence of image processing for privacy protection, the frame image is used for the purpose of displaying other information. May be. For example, a frame image may be displayed to discriminate between a vacant seat state and a seated state. In this case, the vacant seat state and the seated state are expressed by the display color of the frame image, for example, a green frame image is displayed in the vacant seat state, and a red frame image is displayed in the seated state. May be.

  In the present embodiment, the frame image 143 is displayed based on the person frame set in the person area in the person detection. However, the user sets a detection area to be a person detection target in advance on the captured image. Thus, a frame image 143 representing the range of the detection area may be displayed. Since the image of the head of the person seated on the seat in the captured image appears at the position of the headrest of the seat, the person seated on the seat can be detected by setting a detection area at the position of the headrest.

  By the way, in this embodiment, all seats on which a seat usage fee is charged are imaged by any one of the plurality of cameras 1, and each camera 1 captures a fisheye image obtained by imaging via a fisheye lens. A plurality of captured images subjected to distortion correction are output, and a monitoring image is generated from the plurality of captured images. For this reason, a part of the seats installed in the vehicle is shown in one monitoring image, and all seats in one vehicle are divided into seat sections (seat groups) for each of the plurality of seats shown in each monitoring image ( Grouped). Then, when a seat is selected on the seat map image display screen (see FIG. 6), a transition is made to the monitoring image display screen (see FIG. 7). At this time, a monitoring image corresponding to the seat section to which the selected seat belongs is displayed. The

  As described above, in the present embodiment, on the route map image display screen (see FIG. 3), the seat usage status is displayed in units of routes, and on the train map image display screen (see FIG. 4), the seat usage status is the train. In the vehicle map image display screen (see FIG. 5), the seat usage status is displayed in units of vehicles. In the seat map image display screen (see FIG. 6), the seat usage status is displayed in seat units. The Therefore, in the monitoring center, it is possible to check the seat usage status while narrowing down in the order of route, train, and vehicle. Then, on the monitoring image display screen (see FIG. 7), the actual situation of the seat of interest can be confirmed. Thereby, the operation | work which confirms the utilization condition of a seat can be performed efficiently.

  In this embodiment, when a seat is selected on the seat map image display screen (see FIG. 6), the monitoring image display screen (see FIG. 7) is transitioned to. However, the monitoring image display window is displayed on the seat map display screen. You may make it display a hop-up. At this time, the monitoring image display window may be hop-up displayed in the vicinity of the selected seat on the seat map display screen, and a plurality of monitoring image display windows may be displayed by sequentially selecting a plurality of seats. Good.

  In the present embodiment, the train map image display screen (see FIG. 4) is selected on the vehicle map image display screen (see FIG. 5) by selecting the train of interest from the trains operating on the route. Although the usage situation for each vehicle in the train is displayed, the usage situation for each vehicle of each train that operates on the route may be displayed side by side for each train.

  Next, schematic configurations of the camera 1, the server device 2, and the user terminal device 3 will be described. FIG. 8 is a block diagram illustrating hardware configurations of the camera 1, the server device 2, and the user terminal device 3.

  The camera 1 includes an imaging unit 11, a control unit 12, an information storage unit 13, and a communication unit 14.

  The imaging unit 11 includes an image sensor, and sequentially outputs captured images (frames) that are temporally continuous, so-called moving images. The control unit 12 performs processing for detecting a person from the captured image output from the imaging unit 11. The information storage unit 13 stores a program executed by a processor constituting the control unit 12 and a captured image output from the imaging unit 11. The communication unit 14 communicates with the server device 2 and transmits the captured image output from the imaging unit 11 and the person detection information output from the control unit 12 to the server device 2 via the network. .

  In addition to the image sensor, the imaging unit 11 includes a fish-eye lens and an image processing circuit that performs distortion correction on a fish-eye image obtained by imaging a subject through the fish-eye lens. A corrected image generated by the processing circuit is output as a captured image.

  The server device 2 includes a control unit 21, an information storage unit 22, and a communication unit 23.

  The communication unit 23 communicates with the camera 1, the user terminal device 3, and the seat management device 5, receives a captured image and person detection information transmitted from the camera 1, and receives the seat management device 5. Seat management information transmitted from the user terminal, user setting information transmitted from the user terminal device 3 is received, and a seat monitoring browsing screen is distributed to the user terminal device 3. The information storage unit 22 stores captured images, person detection information, and seat management information received by the communication unit 23, a program executed by the processor that constitutes the control unit 21, and the like. The control unit 21 generates a monitoring image or the like based on the captured image, the person detection information, and the seat management information, and generates a seat monitoring browsing screen to be distributed to the user terminal device 3.

  The user terminal device 3 includes a control unit 31, an information storage unit 32, a communication unit 33, an input unit 34, and a display unit 35.

  In the input unit 34, the user inputs various setting information. The display unit 35 displays a seat monitoring browsing screen based on the screen information transmitted from the server device 2. The input unit 34 and the display unit 35 can be configured by a touch panel display. The communication unit 33 communicates with the server device 2, transmits the user setting information input by the input unit 34 to the server device 2, and receives screen information transmitted from the server device 2. To do. The control unit 31 controls each unit of the user terminal device 3. The information storage unit 32 stores a program executed by the processor that constitutes the control unit 31.

  Next, functional configurations of the camera 1 and the server device 2 will be described. FIG. 9 is a functional block diagram of the camera 1 and the server device 2.

  The control unit 12 of the camera 1 includes a moving object removal image generation unit 41 and a person detection unit 42. Each part of this control part 12 is implement | achieved by making the processor which comprises the control part 21 execute the program (instruction) for seat monitoring memorize | stored in the information storage part 13. FIG.

  The moving object removed image generation unit 41 generates a moving object removed image (background image) obtained by removing moving objects such as a person from the captured image based on a plurality of captured images (frames) in a predetermined learning period. A known background image generation technique may be used to generate the moving object removal image.

  The person detection unit 42 compares the moving object removal image (background image) acquired by the moving object removal image generation unit 41 with the current captured image output from the imaging unit 11, and calculates the moving object in the captured image from the difference between the two. The image area is identified (moving object detection). When an Ω shape composed of a person's face or head and shoulders is detected in the moving object image area, the moving object is determined to be a person (person detection). A known technique may be used for this moving object detection and person detection. In the person detection unit 42, position information of each passenger's person area in the captured image and position information of a person frame set in each passenger's person area are output as person detection information.

  In this embodiment, standard person detection based on a known technique is performed. However, since the head image of the person seated on the seat appears in the captured image, the headrest of the seat is displayed. If a moving object that is assumed to be the head of a person is detected at the position, it may be determined that the person is a person.

  The control unit 21 of the server device 2 includes a camera image acquisition unit 51, a person detection information acquisition unit 52, a seat management information acquisition unit 53, an occupancy detection unit 54, a seat usage information acquisition unit 55, and a seat usage information. An aggregation unit 56, a map image generation unit 57, a monitoring image generation unit 58, and an output information generation unit 59 are provided. Each unit of the control unit 21 is realized by causing a processor constituting the control unit 21 to execute a seat monitoring program (instruction) stored in the information storage unit 22.

  The camera image acquisition unit 51 acquires captured images that are transmitted from the camera 1 periodically (for example, at intervals of 15 minutes) and received by the communication unit 23. The person detection information acquisition unit 52 acquires the person detection information transmitted from the camera 1 and received by the communication unit 23 at the same time as the captured image or at a different timing.

  The seat management information acquisition unit 53 acquires the seat management information transmitted from the seat management device 5 and received by the communication unit 23. In the present embodiment, in the seat management device 5, in-vehicle ticket gate information regarding the presence or absence of in-vehicle ticket gates for each seat is collected from the in-vehicle ticket checking device 4, and whether or not each seat can be used, that is, the passenger's authority to use the seat by the in-vehicle ticket gate. The seat management information regarding whether or not there is a seat is provided from the seat management device 5 to the server device 2.

  The presence detection unit 54 detects the presence of a passenger in each seat based on the person detection information acquired from the camera 1 by the person detection information acquisition unit 52. In the present embodiment, seat position information related to the position of each seat in the captured image is set in advance, and with reference to this seat position information, from which position of each passenger on the captured image in the person detection information, to which seat Determine if passengers are present. At this time, when the state where the passenger is seated in the seat continues for a certain period of time, it may be determined to be present. The presence detection unit 54 outputs presence detection information (such as a seat number and presence / absence of each seat) related to the detection result of the presence detection of each seat.

  The seat usage information acquisition unit 55 represents the usage status of each seat based on the seat management information acquired from the seat management device 5 by the seat management information acquisition unit 53 and the presence detection information output from the presence detection unit 54. Get seat usage information. In the present embodiment, the seat management information relates to whether or not each seat can be used, that is, whether or not there is an authority to use a seat by an in-car ticket gate, and the seat management information is compared with presence detection information. As a result of the use of each seat, it is determined whether the user who is seated in each seat is illegal boarding, that is, whether the user is using the seat without paying the seat usage fee. The seat usage information indicating whether or not is acquired for each seat.

  The seat usage information aggregating unit 56 aggregates the seat usage information for each seat acquired by the seat usage information acquisition unit 55 for each vehicle, and generates seat usage information for each vehicle. Further, the seat usage information for each vehicle is aggregated for each train, and the seat usage information for each train is generated. In addition, seat usage information for each train is aggregated for each route, and seat usage information for each route is generated. In these aggregation processes, statistical processes such as integration and averaging may be performed on the number of illegally boarded seats as the seat usage information.

  The map image generation unit 57 generates a route map image 101 (see FIG. 3), a train map image 111 (see FIG. 4), a vehicle map image 121 (see FIG. 5), and a seat map image 131 (see FIG. 6). . In this map image generation unit 57, the seat image 132 in the seat map image 131 is displayed in a display form (display color) corresponding to the seat usage information (whether it is an unauthorized boarding) for each seat acquired by the seat usage information acquisition unit 55. Drawn.

  The map image generation unit 57 ranks the seat usage information (number of seats for illegal boarding) acquired by the seat usage information aggregation unit 56 by comparing with a predetermined threshold value, and displays assigned to each rank. In the form (display color), the route image 102 in the route map image 101, the train image 112 in the train map image 111, and the vehicle image 122 in the vehicle map image 121 are drawn.

  The map image (route map image, train map image, vehicle map image, and seat map image) generated by the map image generation unit 57 is accumulated in the information storage unit 22, and the user can specify the date and time of the specified date and time. The map image can be displayed on the map image display screen.

  The monitoring image generation unit 58 generates a monitoring image (see FIG. 7) on which the use situation for each seat is visualized and image processing for protecting the privacy of passengers shown in the captured image is performed. The monitoring image generation unit 58 includes a distinguishability reduction processing unit 61, a frame image generation unit 62, and an image composition unit 63.

  In the distinctiveness reduction processing unit 61, based on the person detection information (position information of the person area) acquired from the camera 1 by the person detection information acquisition unit 52, image processing for protecting the privacy of passengers captured in the captured image is performed. Image processing (identification reduction processing) for reducing discrimination is performed on the person area of each passenger in the captured image acquired from the camera 1 by the camera image acquisition unit 51 to generate a discrimination reduction image. In the present embodiment, the discrimination reduction process is performed on the face area of each passenger.

  At this time, the face area of each passenger is cut out from the captured image, and the distinctiveness reduction process is performed on the image of the facial area. What is necessary is just to superimpose and produce | generate a discriminability reduction image. Here, as shown in FIG. 7A, when privacy protection image processing is performed for all passengers, images of face areas that have been subjected to identification reduction processing for all passengers are displayed. As shown in FIG. 7 (B), in the case where privacy protection image processing is performed only for legitimate passengers, the identification of the face area in which the discrimination reduction processing is performed only for legitimate passengers. What is necessary is just to acquire an image.

  In the frame image generation unit 62, the seat usage information of each seat acquired by the seat usage information acquisition unit 55 and the person detection information (position information of the person frame) acquired from the camera 1 by the person detection information acquisition unit 52. For each passenger seated in each seat, a frame image 143 surrounding the passenger's head is generated in a display color corresponding to the seat usage information of each seat.

  The image composition unit 63 performs image composition to superimpose the frame image 143 generated by the frame image generation unit 62 on the corresponding person area of the passenger in the distinctiveness reduction image generated by the distinguishability reduction processing unit, thereby obtaining a monitoring image. 141 is generated.

  The monitoring image generated by the monitoring image generation unit 58 is accumulated in the information storage unit 22, and when the user specifies the date and time, the monitoring image at the specified date and time can be displayed on the monitoring image display screen.

  In the output information generation unit 59, a route map image display screen (see FIG. 3) based on the route map image 101, the train map image 111, the vehicle map image 121, and the seat map image 131 generated by the map image generation unit 57, Display information relating to the train map image display screen (see FIG. 4) and the seat map image display screen (see FIG. 5) is generated. Further, the output information generation unit 59 generates display information related to the monitoring image display screen (see FIG. 6) based on the monitoring image generated by the monitoring image generation unit 58.

  By the way, in the present embodiment, a moving object removal image is generated and a person is detected from the difference between the moving object removal image and the captured image. However, in such a person detection method based on moving object detection, for example, during sleep If there is no movement of the person, the person cannot be detected as a moving object, so that the person detection fails and the image processing for privacy protection cannot be appropriately performed on the passenger.

  Therefore, in the present embodiment, the moving object removal image is generated from the captured image acquired when the train starts or stops in the station premises, and person detection is performed. In this way, movement of the passenger's body is caused by the shaking of the vehicle when the train starts or stops, so that the passenger can be reliably detected as a moving body, and image processing for privacy protection for the passenger Can generate a properly implemented monitoring image.

  Moreover, in this embodiment, although it can be determined whether the passenger currently seated in a seat is an unauthorized boarding, it is a burden on a crew member to pay attention to a passenger whenever an unauthorized boarding is found. Therefore, it is conceivable to pay attention only to malicious passengers who repeat illegal boarding and to detect illegal boarding. In this case, when a passenger who performs illegal boarding is found, the face image of the passenger performing illegal boarding is extracted from the captured image of the camera 1 and the face image is stored in the information storage unit 22. Then, person verification is performed on the face images stored in the information storage unit 22 as to whether or not they are the same person, and based on the result of the person verification, the number of illegal boardings for each passenger on each route is acquired. As a result, it is possible to find a malicious passenger who repeats illegal boarding.

(Modification of the first embodiment)
Next, a modification of the first embodiment will be described. FIG. 10 is a functional block diagram of the camera 1 and the server device 2 according to a modification of the first embodiment.

  In the first embodiment, the server device 2 includes the distinguishability reduction processing unit 61. However, in this modification, the camera 1 includes the distinguishability reduction processing unit 45. In the distinguishability reduction processing unit 45 of the camera 1, the distinguishability reduction processing is performed on the person area (face area) of each passenger based on the position information of the person area acquired by the person detection unit 42.

  Here, as shown in FIG. 7A, when the privacy protection image processing is performed for all the passengers, the identification area reduction processing unit 45 of the camera 1 uses the person area ( A face area) is cut out, and a discrimination reduction process is performed on the cut-out person image (face image). Next, the person image that has undergone the discrimination reduction process is superimposed on the captured image to generate a discrimination reduction image, and this discrimination reduction image is transmitted to the server device 2. Then, in the image composition unit 63 of the server device 2, the monitoring image 141 is generated by performing image composition in which the frame image 143 generated by the frame image generation unit 62 is superimposed on the discriminability-reduced image acquired from the camera 1.

  The person image and the captured image that have undergone the discrimination reduction process are transmitted from the camera 1 to the server device 2, and the person image that has undergone the discrimination reduction process in the image composition unit 63 of the server device 2 is converted into a captured image. You may make it perform the image composition to superimpose.

  On the other hand, as shown in FIG. 7B, when privacy protection image processing is performed only for legitimate passengers, the distinctiveness reduction processing unit 45 of the camera 1 identifies all passengers. A person image subjected to the reduction process is generated, and the person image and the captured image for each passenger are transmitted to the server device 2. Then, in the image composition unit 63 of the server device 2, a person image of a legitimate passenger is selected based on the seat use information of each seat acquired by the seat use information acquisition unit 55, and the person image of the passenger is captured. The monitoring image 141 is generated by superimposing the frame image 143 on the discriminability-reduced image and further performing image composition for superimposing the frame image 143 generated by the frame image generation unit 62 on the discriminability-reduced image.

(Second Embodiment)
Next, a second embodiment will be described. Note that points not particularly mentioned here are the same as in the above embodiment. FIG. 11 is an explanatory diagram illustrating a monitoring image display screen according to the second embodiment.

  In the first embodiment, image processing for reducing the identifiability of a person appearing in a captured image (identification reduction processing) is performed as image processing for privacy protection. In this embodiment, in the captured image 152, A mask image 153 representing the shape of the person area of each passenger is generated, and image processing is performed to replace the person area of each passenger in the captured image 152 with the mask image 153.

  Further, in the first embodiment, a frame image 143 that surrounds each passenger's person area is drawn as an identification image for visualizing the use situation (whether it is an unauthorized boarding) for each seat, and a display form (display) of the frame image 143 is displayed. By changing the (color), the seat use information (whether the boarding is illegal or not) is visualized. However, in this embodiment, the mask image 153 becomes an identification image, and the display form (display color) of the mask image 153 The seat usage information is visualized by changing the.

  In the example illustrated in FIG. 11A, privacy protection image processing is performed for all passengers, and all passengers are displayed as mask images 153. In this case, by changing the display mode of the mask image 153, the use situation (whether it is an unauthorized boarding) for each seat is visualized. In the example shown in FIG. 11A, the display color of the mask image 153 For example, the mask image 153 is displayed in red for an unauthorized passenger, and the mask image 153 is displayed in green for an authorized passenger.

  In the example shown in FIG. 11B, privacy-protected image processing is performed only for legitimate passengers, and legitimate passengers are displayed as a mask image 153 in a predetermined display color (for example, blue). . On the other hand, image processing for privacy protection is not performed for an unauthorized passenger, and the captured image 152 is displayed as it is in the person area of the unauthorized passenger.

  Next, functional configurations of the camera 1 and the server device 2 according to the second embodiment will be described. FIG. 12 is a functional block diagram of the camera 1 and the server device 2.

  The control unit 12 of the camera 1 is the same as that of the first embodiment (see FIG. 9), but the person detection unit 42 outputs position information of each passenger's person area in the captured image as person detection information. The position information of the person area and the captured image are transmitted from the communication unit 14 to the server device 2.

  The control unit 21 of the server device 2 is substantially the same as that of the first embodiment (see FIG. 9), but the configuration of the monitoring image generation unit 58 is different from that of the first embodiment, and the monitoring image generation unit 58 generates a mask image. A unit 65 and an image composition unit 63 are provided.

  In the mask image generation unit 65, the person detection information (position information of the person area) acquired from the camera 1 by the person detection information acquisition unit 52 and the seat usage information (whether it is fraudulent) of each seat acquired by the seat usage information acquisition unit. ), A mask image 153 for each passenger is generated. At this time, first, the outline of the person area is acquired based on the position information of the person area, and then the inside of the outline is filled with the display color according to the seat usage information, so that each passenger in the captured image is displayed. A mask image 153 having an outline corresponding to the person area is generated.

  Here, as shown in FIG. 11A, when privacy protection image processing is performed for all passengers, a mask image 153 with a display color corresponding to the seat usage information for all passengers. Is generated. On the other hand, as shown in FIG. 11B, in the case where image processing for privacy protection is performed only for legitimate passengers, a mask image 153 with a predetermined display color is generated only for legitimate passengers. .

  The image composition unit 63 performs image composition by replacing the person area in the captured image acquired from the camera 1 by the camera image acquisition unit 51 with the mask image 153 generated by the mask image generation unit 65 to generate a monitoring image.

  The mask image 153 may be displayed in a transparent state where the background can be seen through. In this case, the moving object removal image (background image) generated by the moving object removal image generation unit 41 is transmitted from the camera 1 to the server device. 2, the mask image generation unit 65 generates a mask image 153 in a transparent state based on the moving object removal image.

(Modification of the second embodiment)
Next, a modification of the second embodiment will be described. FIG. 13 is a functional block diagram of the camera 1 and the server device 2 according to a modification of the second embodiment.

  In the second embodiment, the server apparatus 2 includes the mask image generation unit 65. However, in this modification, the camera 1 includes the mask image generation unit 46. The mask image generation unit 46 of the camera 1 generates a mask image for each passenger based on the position information of the person area acquired by the person detection unit 42.

  Here, as shown in FIG. 11 (A), when privacy protection image processing is performed for all passengers, the mask image generation unit 46 of the camera 1 uses two types ( A different color) mask image, that is, a first mask image representing a legitimate passenger and a second mask image representing an unauthorized passenger are generated and transmitted to the server device 2. Then, in the mask image selection unit 66 of the server device 2, one of the first mask image and the second mask image for each passenger based on the seat usage information of each seat acquired by the seat usage information acquisition unit 55. Select. Then, the image composition unit 63 performs image composition to replace the passenger's person area in the captured image with the mask image of each passenger selected by the mask image selection unit 66.

  On the other hand, as shown in FIG. 11B, when privacy protection image processing is performed only for legitimate passengers, the mask image generation unit 46 of the camera 1 uses one type of mask for each passenger. An image is generated and transmitted to the server device 2. Then, the mask image selection unit 66 of the server device 2 selects a legitimate passenger mask image, and the image composition unit 63 selects the legitimate passenger person area in the captured image by the mask image selection unit 66. The image is replaced with the mask image.

  As described above, the embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can be applied to embodiments in which changes, replacements, additions, omissions, and the like have been performed. Moreover, it is also possible to combine each component demonstrated by said embodiment into a new embodiment.

  For example, in the above-described embodiment, a seat for which a seat usage fee is charged for a non-reserved seat is targeted, but a reserved seat may be targeted. In this case, in the server device 2, the seat management information related to the seat designation ticket purchased by the passenger at the ticket vending machine is acquired from the seat management device 5, and the reserved seat, That is, if a passenger is present in a seat other than the seat specified by the sold seat reservation ticket, it may be determined that the boarding is illegal.

  In the above embodiment, the inside of the train is used as a monitoring area to visualize the use situation of the seats installed in the car (the situation of illegal boarding), but the audience of various facilities such as movie theaters and stadiums A seat may be used as a monitoring area. In such a facility, a reserved seat is provided, and when a user is present in a seat other than the seat specified by the sold seat reservation ticket, it can be determined that the seat is illegally used.

  In the above embodiment, the unauthorized use of the seat is visualized based on the availability of each seat, that is, based on the seat management information regarding whether or not the user has the authority to use the seat. You may make it visualize the utilization condition of a seat by using the inside of restaurants, such as a restaurant and a coffee shop, as a monitoring area.

  For example, in a restaurant, as seat usage information, a waiting time required from when a customer is seated in a passenger seat until the store clerk accepts an order is acquired, and the display color of the frame image or mask image for each customer in the monitoring image By changing according to the waiting time, the waiting time situation for each customer can be visualized. Thereby, a user can grasp | ascertain a customer with long waiting time immediately.

  In addition, by acquiring information about the product ordered by the customer seated in the passenger seat as seat usage information, and changing the display color of the frame image or mask image for each customer in the monitoring image according to the ordered product, The order status for each customer can be visualized. As a result, the user can immediately grasp what kind of product the customer who is seated in the customer's order has ordered, and the specific product can be ordered by narrowing down to a specific product (for example, coffee). It is possible for the user to immediately grasp in which customer seat the selected customer is seated.

  Also, as the seat usage information, the stay time in which the customer stays in the passenger seat is acquired, and the display color of the frame image or mask image for each customer in the monitoring image is changed according to the stay time, so that You can visualize the status of your stay. As a result, the user can immediately grasp how long the customer who is seated in the audience is staying, and by displaying only when the stay time exceeds a predetermined threshold, The user can immediately grasp customers who have stayed for a long time.

  The seat monitoring device, the seat monitoring system, and the seat monitoring method according to the present invention make it possible to visualize a use situation regarding unauthorized use of each seat on an image obtained by imaging a monitoring area, and to capture a user in an image obtained by imaging the monitoring area. Monitoring device and seat monitoring that generate output information that visualizes the use status of a seat by a user based on an image obtained by imaging a monitoring area in which a plurality of seats are arranged. It is useful as a system and seat monitoring method.

DESCRIPTION OF SYMBOLS 1 Camera 2 Server apparatus 3 User terminal apparatus 4 In-car check device 5 Seat management apparatus 41 Moving object removal image generation part 42 Person detection part 45 Discrimination reduction process part 46 Mask image generation part 51 Camera image acquisition part 52 Person detection information acquisition part 53 Seat management information acquisition unit 54 Seat presence detection unit 55 Seat use information acquisition unit 56 Seat use information aggregation unit 57 Map image generation unit 58 Monitoring image generation unit 59 Output information generation unit 61 Discrimination reduction processing unit 62 Frame image generation unit 63 Image Compositing unit 65 Mask image generating unit 66 Mask image selecting unit 101 Route map image 102 Route image 111 Train map image 112 Train image 121 Vehicle map image 122 Vehicle image 131 Seat map image 132 Seat image 141 Monitoring image 142 Discriminability reduction image 143 Frame Image 151 Monitoring image 152 Captured image 53 mask image

Claims (15)

A seat monitoring device that generates output information that visualizes the use status of a seat by a user based on an image obtained by imaging a monitoring area where a plurality of seats are arranged,
A seat usage information acquisition unit for acquiring seat usage information related to seat usage by the user;
Based on the seat usage information, the usage status of each seat is visualized by changing the display mode of the identification image for each user arranged on the captured image obtained by imaging the monitoring area, and the user in the captured image is displayed. A monitoring image generation unit that generates a monitoring image in which privacy-protected image processing is performed on at least a part of the person area of
An output information generation unit that generates the output information including the monitoring image;
A seat monitoring device comprising: Furthermore, a seat management information acquisition unit that acquires seat management information regarding availability of each seat;
Based on the result of person detection for the captured image, a presence detection unit that detects the presence of a user in each seat;
With
The seat usage information acquisition unit determines whether or not each seat is illegally used based on the seat management information and the detection result of the presence detection unit, and acquires the seat usage information related to unauthorized use of each seat. The seat monitoring apparatus according to claim 1.   The monitoring image generation unit performs the image processing for privacy protection on a person area of a user who is not unauthorized use based on the seat usage information, and protects the privacy on a person area of a user who is unauthorized use The seat monitoring apparatus according to claim 2, wherein the image processing is not performed.   The monitoring image generation unit generates the monitoring image that has been subjected to a discrimination reducing process of any one of a mosaic process, a blur process, and a blend process as the privacy protection image process. The seat monitoring apparatus according to claim 3.   The monitoring image generation unit generates the monitoring image by superimposing a frame image surrounding at least a part of a user's person area on the discrimination reduced image subjected to the discrimination reduction process, and the seat usage information The seat monitoring apparatus according to claim 4, wherein the usage status of each seat is visualized by changing a display form of the frame image based on the information.   The monitoring image generation unit generates, as the privacy protection image processing, the monitoring image that has been subjected to image processing for replacing a user person area in the captured image with a mask image representing a shape of the person area. The seat monitoring device according to any one of claims 1 to 3, wherein   The seat monitoring apparatus according to claim 6, wherein the monitoring image generation unit visualizes a use situation for each seat by changing a display form of the mask image based on the seat use information. The monitoring area is in a train where a seat where a seat usage fee is charged is arranged,
The monitoring image generation unit generates the monitoring image subjected to the image processing for privacy protection based on a person detection result for the captured image captured when the train starts or stops in a station. The seat monitoring device according to any one of claims 1 to 7, wherein the seat monitoring device is characterized by the following. The monitoring area is in a train where a seat where a seat usage fee is charged is arranged,
The seat monitoring apparatus according to any one of claims 1 to 8, wherein the seat management information acquisition unit acquires the seat management information when a train starts or stops in a station.   Furthermore, by generating a seat map image consisting of a schematic diagram showing the arrangement status of seats in the monitoring area, and changing the display mode of the seat image representing each seat in the seat map image based on the seat usage information, The seat monitoring apparatus according to any one of claims 1 to 9, further comprising a map image generation unit that visualizes a use situation of a user for each seat.   The output information generation unit outputs the output information including the monitoring image corresponding to a seat section to which a seat specified by the user belongs in response to an operation input of the user specifying the seat on a screen displaying the seat map image. The seat monitoring device according to claim 10, wherein the seat monitoring device is generated. The monitoring area is in a train where a seat where a seat usage fee is charged is arranged,
Furthermore, the seat use information collecting unit that collects the seat use information acquired by the seat use information acquiring unit at least one of a vehicle unit, a train unit, and a route unit;
Based on the seat usage information aggregated by the seat usage information aggregating unit, a route map image that visualizes the usage status for each route, a train map image that visualizes the usage status for each train, and a vehicle that visualizes the usage status for each vehicle A map image generation unit that generates at least one of the map images;
The seat monitoring apparatus according to any one of claims 1 to 11, further comprising:   In the initial state, the monitoring image generation unit generates the monitoring image that has been subjected to the privacy protection image processing, and does not perform the privacy protection image processing in response to a user operation input for releasing privacy protection. The seat monitoring apparatus according to any one of claims 1 to 12, wherein a monitoring image is generated. A seat monitoring system that generates output information that visualizes the use status of a seat by a user based on an image of a monitoring area where a plurality of seats are arranged,
A camera that images the monitoring area and detects a person in the captured image;
A server device that generates the output information that visualizes the use status of a seat by a user;
Based on the output information, a user terminal device that displays a browsing screen;
Have
The server device
A seat usage information acquisition unit for acquiring seat usage information related to seat usage by the user;
Based on the seat usage information, the usage status for each seat is visualized by changing the display mode of the identification image for each user arranged on the captured image, and at least the person area of the user in the captured image A monitoring image generation unit that generates a monitoring image in which privacy-protected image processing has been performed on a part thereof;
An output information generation unit that generates the output information including the monitoring image;
A seat monitoring system characterized by comprising: A seat monitoring method for causing an information processing device to perform processing for generating output information that visualizes the use situation of a seat by a user based on an image obtained by imaging a monitoring area where a plurality of seats are arranged,
Obtain seat usage information on seat usage by users,
Based on the seat usage information, the usage status of each seat is visualized by changing the display mode of the identification image for each user arranged on the captured image obtained by imaging the monitoring area, and the user in the captured image is displayed. A monitoring image in which privacy-protected image processing is performed on at least a part of the person area of
The seat monitoring method, wherein the output information including the monitoring image is generated.

Images