JP2019091145A - Image processing apparatus, seat usage status detection system, image processing method, and image processing program - Google Patents

Abstract

To reduce costs of building a system that detects a usage status of a seat provided in a facility such as a vehicle.SOLUTION: An image processing apparatus comprises: person detection means for detecting a person overlapping with a part of a seat of a facility from an image in the facility input from a camera; and seating position detection means for detecting and outputting identification information of the seat on which the person is detected from a portion of the seat reflected in the image, the portion not overlapping with the person.SELECTED DRAWING: Figure 1

Description

  The present invention relates to image processing, and more particularly to image processing for detecting the usage of a seat such as a vehicle.

  Patent Document 1 discloses a system that uses a photographed image to confirm whether or not there is an authorized user whose use is permitted at each use place in a facility such as a vehicle. In this system, a camera is installed corresponding to each seat. This camera shoots toward the corresponding seat, and performs color shooting of the seated user from the front direction.

  In this system, the seating management database stores and manages information on the user for each seat in the vehicle, and has items of "seat number" and "seating image data". "Seat number" is a number unique to the seat, "seated image data" is a captured image captured toward the seat, and if the user is seated, the captured image of the user .

  This system uses this seating management database to check if there is an authorized user at each seat.

JP 2004-280477 A

  As a method of detecting the usage condition of a designated seat such as a vehicle, a method of detecting that a user is seated by providing sensors, for example, a camera (Patent Document 1 above), a far infrared sensor or a load sensor corresponding to each seat. There is. This method requires a large number of sensors and has the problem that the equipment is complicated and expensive. Furthermore, since it is necessary to add sensors to the seat anew after the completion of a vehicle or the like, there also arises a problem that a period of time and costs for construction will be incurred.

  There is a method of using a TOF (Time Of Flight) sensor or a stereo camera which detects a depth position in order to detect the use situation of a plurality of rows of seats as a method which does not attach sensors to each seat. However, this method also has the problem that the equipment becomes expensive.

  An object of the present invention is to provide an image processing device, a seat usage detection system, an image processing method, and an image processing program, which solve the above-mentioned problems.

  In an image processing apparatus according to an embodiment of the present invention, a person detection unit for detecting a person overlapping a part of a seat of the facility from an image in a facility input from a camera, and the person detecting And seating position detecting means for detecting and outputting the identified information of the seat from the portion of the seat shown in the image which is not overlapped with the person.

  An image processing method according to one embodiment of the present invention detects a person overlapping a part of a seat of the facility from an image in a facility input from a camera, and detects the person whose seat is detected. Identification information is detected and output from a portion of the seat shown in the image that does not overlap the person.

  An image processing program according to an embodiment of the present invention detects a person detecting a person overlapping a part of a seat of the facility from an image in a facility input from a camera, and detecting the person The computer is caused to execute a seating position detection process of detecting and outputting the identified information of the seat from the portion of the seat shown in the image which does not overlap the person.

  The image processing apparatus according to the present invention can reduce the construction cost of a system for detecting the use condition of a seat provided in a facility such as a vehicle.

FIG. 1 is a block diagram showing the configuration of a seat use condition detection system 30 according to the present embodiment. FIG. 2 is a side view of the configuration in the train car 40. FIG. 3 shows an example of a field of view (shooting range) of the camera 31 in the vehicle 40. FIG. 4 is a block diagram of the computer device 60. As shown in FIG. FIG. 5 is a block diagram showing the configuration of the image processing apparatus 10 according to the second embodiment.

First Embodiment
<Overview>
In the seat usage detection system 30 of this embodiment, identification information 42 is attached to each seat 41 provided in a facility, for example, a train car 40. At the time of use of the facility, the camera 31 in the facility captures the identification information 42 and a person such as the passenger 43 seated in the seat together. Then, the image processing apparatus 10 and the designated seat status management apparatus 20 detect the usage status of the seat 41 from the image captured by the camera 31. This reduces the construction cost of the seat usage detection system 30.

<Configuration>
FIG. 1 is a block diagram showing the configuration of a seat use condition detection system 30 according to the present embodiment. The seat usage status detection system 30 includes an image processing device 10, a designated seat status management device 20, a camera 31, and a portable terminal device 32. The image processing apparatus 10 and the designated seat status management apparatus 20 are connected. The camera 31 is provided in a facility, for example, a movie theater, a concert hall, a train car 40, and is connected to the image processing apparatus 10 via a wireless or wired communication network. The portable terminal device 32 is connected to the designated seat status management device 20 via a wireless communication network. The portable terminal device 32 is carried by a ticket checker, for example, a guide of a movie theater, and a conductor of a vehicle 40.

  In the following description, it is assumed that the facility is a train car 40, the person using the facility is a passenger 43, and the ticket checker is a conductor.

  The image processing apparatus 10 includes a person detection unit 11, a seating position detection unit 12, and a seat position information storage unit 13. The human detection unit 11 detects a passenger 43 overlapping a part of the seat 41 from the image input from the camera 31. Here, “overlap with a part of the seat 41” is that the person (in this case, the passenger 43) blocks the field of view, and a part of the seat 41 can not be photographed from the camera 31, but the other part is photographed It means that you can.

  The seating position detection unit 12 detects and outputs the identification information 42 of the seat 41 in which the passenger 43 is detected from the image. The seat position information storage unit 13 stores the position information of the seat 41 in association with the identification information 42. The seat position information storage unit 13 stores, for example, a two-dimensional table including a row corresponding to each seat 41, a column corresponding to the identification information 42, and a column corresponding to the position information. Here, the seat position is position information specific to each seat 41, and is, for example, two-dimensional data such as 16-D indicating the row of the seats 41 and the sheet position in the row. The identification information 42 will be described later.

  The designated seat status management device 20 includes a designated seat use detection unit 21 and a designated seat reservation information storage unit 22. The designated seat use detection unit 21 acquires the reservation presence / absence information associated with the position information of the seat 41 input from the image processing apparatus 10 from the designated seat reservation information storage unit 22 and determines the position of the seat 41 without reservation. Output information The designated seat reservation information storage unit 22 stores reservation presence / absence information of the seat 41 in association with the position information of the seat 41. The designated seat reservation information storage unit 22 is, for example, an array of flag variables indicating the presence or absence of a reservation, which has a seat position as an index value. If the seating position is two-dimensional data, this arrangement may be a two-dimensional arrangement.

  The portable terminal device 32 is a terminal device provided with a display unit such as characters, and is, for example, a smartphone.

  FIG. 2 is a side view of the configuration in the train car 40. In the vehicle 40, a plurality of seats 41 are provided. The seats 41 in the vehicle 40 are, for example, a plurality of rows of a plurality of side-by-side seats arranged in front and rear. The seats 41 in the vehicle 40 of FIG. 2 are arranged in five front and rear rows (see FIG. 3) of two side-by-side seats.

  Identification information 42 is attached to the upper back portion of each seat 41. The identification information 42 is information unique to each seat 41, and is information that can be distinguished from the image captured by the camera 31. The identification information 42 is, for example, a pattern of the backrest portion pattern as shown in FIG. 2 or a color. The identification information 42 may be a character string or a number including a symbol written on the backrest portion. The identification information 42 is attached to a position where the seated passenger 43 does not block the view of the camera 31, for example, both left and right ends of the upper portion of the backrest.

  The camera 31 is installed at a position where the identification information 42 of all the seats 41 in the vehicle 40 and at least a part of the passengers 43 seated on the seats 41 can be photographed. The camera 31 is fixed to, for example, a front upper portion of the seat 41 or a front ceiling of the seat 41 of the vehicle 40. Here, “the front surface of the seat 41” is a wall surface in the direction in which the seated passenger 43 faces, and there is a case where the traveling direction of the vehicle 40 does not match. The camera 31 can also be installed at another place by using a mirror or the like.

  FIG. 3 shows an example of a field of view (shooting range) of the camera 31 in the vehicle 40. As shown in FIG. 3, the camera 31 can capture the identification information 42 and the head of the passenger 43 seated in the seat 41 for all the seats in the vehicle 40. The camera 31 may have the entire field of view as shown in FIG. 3 in a fixed state. Alternatively, with the camera 31 fixed, only a part of FIG. 3, for example, the first to third rows is put into view, and the camera 31 is rotated by a motor or the like to change the direction of the lens. Then, the remaining field of view, for example, the fourth and fifth columns may be obtained.

  The camera 31 shoots still images of all the seats 41 of the vehicle 40, for example, periodically, and transmits the images to the image processing apparatus 10. Alternatively, the camera 31 and the portable terminal device 32 are connected by a wireless communication network, and when the instruction is received from the portable terminal device 32, the still image of all the seats 41 of the vehicle 40 is photographed and the image processing device 10 is You may send it.

  Here, the person detection unit 11 and the seating position detection unit 12 of the image processing apparatus 10 are configured by a logic circuit or a dedicated processor and firmware. The seat position information storage unit 13 is a storage device such as a semiconductor storage device, a hard disk drive (HDD), or a solid state drive (SDD).

  The image processing device 10 may be configured by a computer device 60. FIG. 4 is a block diagram of the computer device 60. As shown in FIG. The computer device 60 comprises a processor 61, main storage means 63 and external storage means 62 interconnected by a bus 64.

  The processor 61 reads and writes data from and to the main storage means 63 and the external storage means 62 via the bus 64.

  The processor 61 also executes an image processing program 65 stored in the main storage unit 63. The image processing program 65 is initially stored in the external storage means 62, and the processor 61 may load the image processing program 65 from the external storage means 62 into the main storage means 63 at the time of initial setting of the computer device 60.

  Here, the main storage means 63 is a semiconductor memory means. The external storage unit 62 is a storage unit such as an HDD or an SDD.

  The processor 61 of the computer device 60 functions as the person detection unit 11 and the seating position detection unit 12 by executing the image processing program 65. That is, the processor 61 executes the image processing program 65 to execute the processing performed by the person detection unit 11 and the seating position detection unit 12.

  The main storage unit 63 or the external storage unit 62 is used as the seat position information storage unit 13.

  The designated seat usage detection unit 21 of the designated seat status management device 20 is configured of a logic circuit or a dedicated processor and firmware. The designated seat reservation information storage unit 22 is a storage device such as a semiconductor storage device, an HDD, or an SDD.

  The designated seat state management device 20 may be configured by the computer device 60 in the same manner as the image processing device 10. In this case, the image processing program 65 is replaced with another program, and the processor 61 of the computer device 60 functions as the designated seat use detection unit 21 by executing the other program.

<Operation>
When the operation of a train starts, for example, the camera 31 captures a still image of the seat 41 of the vehicle 40 periodically or according to an instruction from the mobile terminal 32, and inputs the image to the image processing apparatus 10.

  In the image processing apparatus 10, the person detection unit 11 receives the image and detects the passenger 43 overlapping a part of the seat 41. The human detection unit 11 performs this extraction using, for example, a background difference.

  Specifically, for example, the person detection unit 11 acquires images of all the seats 41 of the vehicle 40 with the camera 31 before the operation of the train, and stores the images as background images. Thereafter, the person detection unit 11 may compare the image acquired at the same angle with the background image during operation, and detect the difference in the seat 41 as the passenger 43 overlapping with a part of the seat 41. The person detection unit 11 may further detect the passenger 43 after confirming that the difference has a feature as a person, for example, a shape of a head.

  When detecting the passenger 43 overlapping a part of the seat 41, the person detection unit 11 outputs the position of the passenger 43 in the image to the seating position detection unit 12.

  The seating position detection unit 12 searches around the position of the passenger 43 in the image and detects any of the identification information 42 stored in the seat position information storage unit 13. When the identification information 42 is a pattern pattern, the seating position detection unit 12 detects the identification information 42 stored in the seat position information storage unit 13 that matches the pattern pattern around the position of the passenger 43 in the image. . When the identification information 42 is a color, the seating position detection unit 12 detects the identification information 42 stored in the seat position information storage unit 13 that matches the color around the position of the passenger 43 in the image. When the identification information 42 is a character string or a number, the seating position detection unit 12 is stored in the seat position information storage unit 13 that matches the character string or the number extracted from the periphery of the position of the passenger 43 in the image Detect the identification information 42 that is present.

  Furthermore, the seating position detection unit 12 acquires position information of the seat 41 stored in the seat position information storage unit 13 in association with the detected identification information 42 and outputs the acquired position information to the designated seat condition management device 20.

  The designated seat use detection unit 21 of the designated seat status management device 20 acquires the reservation presence / absence information associated with the position information of the seat 41 input from the image processing device 10 from the designated seat reservation information storage unit 22, When it detects that the seat 41 is not reserved, an alarm is notified. The designated seat usage detection unit 21 transmits, for example, position information of the seat 41 for which the reservation has not been made and an image of the passenger 43 overlapping a part of the seat 41 to the mobile terminal device 32 as an alarm. The alarm may be only the position information of the seat 41 for which the reservation has not been made.

<Effect>
The image processing apparatus 10 according to the present embodiment can reduce the construction cost of the seat usage detection system 30 for detecting the usage of the seat 41 which is a designated seat in a facility such as the vehicle 40. The reason is that the seating position detection unit 12 detects the identification information 42 of the seat 41 whose overlap with a person such as the passenger 43 is detected from the image captured by the camera 31. For this reason, it is possible to detect the use situation of the plurality of seats 41 from one image, and it becomes unnecessary to install the camera 31 for each seat 41. The camera 31 can also divert an existing surveillance camera.

  Furthermore, the image processing apparatus 10 according to the present embodiment can accurately detect the usage condition of the seat 41. The reason is that the seat 41 is identified independently of the position in the image. For this reason, the identification error of the seat 41 does not occur even if the position of the seat 41 in the image is shifted due to the swing or shift of the direction of the camera 31. This property is particularly useful when the camera 31 is installed in a vibrating train.

  Furthermore, the image processing apparatus 10 according to the present embodiment can detect the usage condition of the seat 41 which is disposed in a wide range with one camera 31. The reason is that since the seat 41 is identified independently of the position in the image, the orientation of the camera 31 can be changed by a motor or the like, and a wide range of images can be obtained. This characteristic is particularly useful when diverting existing surveillance cameras.

<Modification>
The image processing device 10 and the designated seat status management device 20 may be integrated and implemented as a single device. In this case, for example, in addition to the person detection unit 11, the seating position detection unit 12, and the seat position information storage unit 13, the image processing apparatus 10 includes a designated seat use detection unit 21 and a designated seat reservation information storage unit 22. Equipped with

  The integrated image processing device 10 may also be configured by the computer device 60. In this case, the processor 61 of the computer device 60 functions as the person detection unit 11, the seating position detection unit 12, and the designated seat use detection unit 21 by executing the image processing program 65. The main storage unit 63 or the external storage unit 62 is used as the seat position information storage unit 13 and the designated seat reservation information storage unit 22.

  The camera 31 may capture a moving image instead of a still image, and the person detecting unit 11 and the seating position detecting unit 12 may extract information such as the seat 41 from a frame of the moving image.

Second Embodiment
FIG. 5 is a block diagram showing the configuration of the image processing apparatus 10 according to the second embodiment. The image processing apparatus 10 includes a person detection unit 11 and a seating position detection unit 12. The person detection unit 11 detects a person such as the passenger 43 overlapping a part of the seat 41 provided in the facility from the image input from the camera 31. The seating position detection unit 12 detects and outputs the identification information 42 of the seat 41 in which the person is detected from the portion of the seat 41 shown in the image that does not overlap with the person.

  The image processing apparatus 10 according to the present embodiment can reduce the construction cost of the seat usage detection system 30 for detecting the usage of the seat 41 in a facility such as the vehicle 40. The reason is that the seating position detection unit 12 detects the identification information 42 of the seat 41 whose overlap with a person such as the passenger 43 is detected from the image captured by the camera 31. For this reason, it is possible to detect the use situation of the plurality of seats 41 from one image, and it becomes unnecessary to install the camera 31 for each seat 41. The camera 31 can also divert an existing surveillance camera.

  Furthermore, the image processing apparatus 10 according to the present embodiment can accurately detect the usage condition of the seat 41. The reason is that the seat 41 is identified independently of the position in the image. For this reason, the identification error of the seat 41 does not occur even if the position of the seat 41 in the image is shifted due to the swing or shift of the direction of the camera 31. This property is particularly useful when the camera 31 is installed in a vibrating train.

  Furthermore, the image processing apparatus 10 according to the present embodiment can detect the usage condition of the seat 41 which is disposed in a wide range with one camera 31. The reason is that since the seat 41 is identified independently of the position in the image, the orientation of the camera 31 can be changed by a motor or the like, and a wide range of images can be obtained. This characteristic is particularly useful when diverting existing surveillance cameras.

  As mentioned above, although this invention was demonstrated with reference to embodiment, this invention is not limited to the said embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Person detection part 12 Sitting position detection part 13 Seat position information storage part 20 Specification seat condition management apparatus 21 Specification seat use detection part 22 Specification seat reservation information storage part 30 Seat usage detection system 31 Camera 32 Portable terminal device DESCRIPTION OF SYMBOLS 40 Vehicle 41 seat 42 Identification information 43 Passenger 60 Computer apparatus 61 Processor 62 External storage means 63 Main storage means 64 Bus 65 Image processing program

Claims (10)

Person detection means for detecting a person overlapping a part of the seat of the facility from an image in the facility input from a camera;
An image processing apparatus comprising: seating position detection means for detecting and outputting identification information of the seat in which the person is detected from a portion of the seat shown in the image which does not overlap the person. The identification information is a pattern or a color different from one seat to another on the upper back portion of the seat,
The image processing apparatus according to claim 1, wherein the seating position detection unit detects the pattern of the color or the pattern from the image. The identification information is a character string or a number, which is different for each seat, written on the upper back portion of the seat,
The image processing apparatus according to claim 1, wherein the seating position detection unit detects the character string or the number from the image. And seat position information storage means for storing the position information of the seat in association with the identification information;
The seating position detection means acquires the position information associated with the identification information of the seat where the person is detected from the seat position information storage means and outputs it. An image processing device. An image processing apparatus according to claim 4, and
Designated seat reservation information storage means for storing the presence / absence information of the seat in association with the position information, and the presence / absence information of the reservation associated with the position information inputted from the image processing apparatus And a designated seat status management device comprising: a designated seat usage detection unit for acquiring the position information of the seat without the reservation acquired from the information storage unit; Further, the camera fixedly installed on a front upper portion of the seat or a front ceiling of the seat of a train vehicle;
6. The seat usage detection system according to claim 5, further comprising: a portable terminal device that receives and displays the position information of the seat output by the designated seat usage detection unit.   The seat usage detection system according to claim 6, wherein the camera changes the seat to be photographed by changing the direction of a lens. From the image in the facility input from the camera, detect a person who overlaps with a part of the seat of the facility,
The image processing method which detects and outputs the identification information of the said seat in which the said person was detected from the part which does not overlap with the said person of the said seat shown by the said image. The identification information is a pattern or a color different from one seat to another on the upper back portion of the seat,
The image processing method according to claim 8, wherein a pattern of the color or the pattern is detected from the image. Person detection processing for detecting a person overlapping a part of a seat of the facility from an image in the facility input from a camera;
Image processing for causing a computer to execute seating position detection processing for detecting and outputting the identification information of the seat in which the person is detected from a portion of the seat shown in the image which does not overlap the person program.

Images