JP2014186650A - Passenger detection system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a passenger detection system 1 highly efficiently detecting passengers while reducing a load of devices constituting a system and a database capacity.SOLUTION: An information reading device 10 is installed at an entrance of a detection area for acquiring initial feature quantity data acquired from boarding ticket information and an initial appearance image of a passenger. A plurality of passenger detection devices 30 are installed within the detection area. A central processor 100 selects at least one passenger detection device of the plurality of passenger detection devices on the basis of information on transportation means included in the boarding ticket information and transit information of the transportation means acquired from transit database 130. When the feature quantity data extracted from the image-captured person matches the initial feature quantity data, the passenger detection device determines to detect a specific passenger.

Description

  The present invention relates to a system for detecting passengers boarding transportation means such as airplanes, ships, and railroads. In particular, passengers who are not boarding transportation means even within the predetermined area even when the departure time of transportation means is about The present invention relates to a passenger detection system for efficient search.

Passengers who board international flights at airport terminal buildings are required to complete boarding procedures in advance of the boarding start time, enter the restricted area through security inspections, and board the boarding before the departure time. Yes.
However, even when the boarding start time has passed due to various reasons, such as when the passenger entered the security checkpoint is approaching the departure time of the boarding flight, or forgetting the departure time while shopping in the restricted area, the boarding gate Passengers may not appear. The airline company makes a call by a local broadcast to find passengers, but it is difficult to find passengers when the restricted area is large.
In order to facilitate the detection of passengers, Patent Document 1 describes a boarding security check system that performs a traceability check of passengers. In the present invention, first, a passenger's face is photographed and registered in a database by a first network camera installed near the entrance of the security checkpoint. Then, the action tracking management unit of the airport server collates the face of the person photographed by the second network camera installed at an arbitrary location in the airport with the face of the passenger registered in the database. When the two faces are collated, the position of the second network camera that captured the passenger and the time when the passenger was captured are registered in a database, and the behavior of the passenger is tracked and managed.

Japanese Patent No. 4528262

In the system described in Patent Document 1, since passengers are tracked and managed, it is easy to grasp the current position of the passengers. However, every time a passenger is photographed, the behavior tracking management unit needs to collate the face of the passenger registered in the database with the face of the photographed person. Therefore, there is a problem that the load on the airport server having the behavior tracking management unit is increased. Moreover, since the information to that effect is registered in the database every time the second network camera captures a passenger, there is a problem that the database capacity tends to increase.
The present invention has been made in view of the above-described circumstances, and an object of the present invention is to provide a passenger detection system for efficiently searching for passengers while reducing the load on the devices constituting the system and the database capacity. .

  In order to solve the above-mentioned problem, the invention according to claim 1 is a passenger detection system for detecting a specific passenger in a predetermined detection area from the entrance to at least one boarding gate, and is installed at the entrance. An information reading means for reading boarding pass information of a boarding pass possessed by a passenger, a reading camera for photographing the passenger, boarding pass information acquired by the information reading device, The passenger feature database obtained by associating and storing the passenger's initial feature data obtained from the passenger's initial appearance image photographed by the reading camera, and extracting the initial feature data of the specific passenger from the passenger database Obtained from a central processing unit, a detection camera installed in the detection area and photographing the detection area, and an appearance image of a person photographed by the detection camera Matching means for determining that the person photographed by the detection camera is the specific passenger when the feature quantity data matches the initial feature quantity data, and a plurality of passenger detection devices, respectively, An operation database for storing operation information relating to at least one means of transportation departing from the boarding gate, wherein the central processing unit includes information relating to means of transportation included in boarding pass information acquired from the passenger database, and the operation database. And a passenger detection device selection means for selecting at least one passenger detection device among the plurality of passenger detection devices based on the operation information on the transportation means acquired from the vehicle.

  The detection area is a closed area having an entrance and a boarding gate as an exit. An information reading device is installed at the entrance of the detection area, and a plurality of passenger detection devices are installed in the detection area. When the passenger enters the detection area, the information reading means of the information reading device reads boarding pass information including information on the transportation means from the boarding pass, and the reading camera of the information reading device takes an appearance image of the passenger. Information acquired by the information reading device is registered in the passenger database. When detecting a specific passenger, first, the central processing unit extracts information on the transportation means on which the specific passenger is boarded from the passenger database, and extracts operation information about the extracted transportation means from the operation database. Subsequently, the central processing unit selects a passenger detection device that detects a passenger from a plurality of passenger detection devices installed in the detection area, based on the extracted operation information about the transportation means. The passenger detection device selected by the central processing unit determines that the specific passenger has been found when the feature data extracted from the person photographed by the detection camera matches the initial feature data of the specific passenger. .

In the invention of claim 2, the boarding pass information includes a boarding flight name specifying a transportation means on which a passenger boardes, and the boarding flight name and the boarding gate are associated and registered in the operation database. The passenger detection device selection means acquires boarding gate information from the operation database based on the boarding flight name acquired from the passenger database, and detects passengers installed in an area from the entrance to the boarding gate. The device is selected.
In the invention of claim 2, the central processing unit acquires information on the boarding gate as a place where the transportation means departs from the boarding flight name included in the boarding pass information. Here, the passenger enters the detection area from the entrance, moves in the detection area, and enters from the boarding gate. The central processing unit selects a passenger detection device installed in an area from the entrance to the boarding gate as a device for detecting a passenger, and detects the passenger efficiently by causing the selected device to detect the passenger.
The invention of claim 3 is a terminal comprising: input means for inputting data of the specific passenger; and communication means for transmitting data indicating the specific passenger input from the input means to the central processing unit. The initial processing amount data acquired from the passenger database by causing the passenger database to search the initial feature amount data using data indicating a specific passenger transmitted from the terminal device as a key. Is transmitted to the passenger detection device selected by the passenger detection device selection means.
In invention of Claim 3, arbitrary passengers can be specified from a terminal device, and a passenger detection apparatus can be made to detect.

According to a fourth aspect of the present invention, the terminal device includes a microphone that collects an audio signal, and a communication unit that transmits the collected audio data to the passenger detection device. The passenger detection device includes the communication unit. A speaker is provided for outputting audio data received from the TV.
In the invention of claim 4, each passenger detection device is provided with a speaker, and the voice of the operator is broadcast from the speaker of the passenger detection device at a short distance from the specific passenger, thereby pinpointing the specific passenger. You can make a call at.
According to a fifth aspect of the present invention, the passenger detection device includes a passenger tracking unit that tracks the image of the specific passenger determined by the matching unit, and the passenger tracking unit adds a marker image to the person image being tracked. It is characterized by combining.
In the invention of claim 5, since the marker image is synthesized with the image of the specific passenger found, the specific passenger and other passengers can be visually distinguished.
The invention according to claim 6 is characterized in that the entrance is a security inspection site entrance and the predetermined detection area is a restricted area.
The invention of claim 6 detects a passenger in a restricted area from the security checkpoint entrance to the boarding gate.

According to the present invention, when it is necessary to detect a passenger, the passenger detection device detects the passenger, so the load on the devices constituting the system can be reduced.
Further, the present invention stores boarding pass information necessary for passenger detection and initial feature data obtained from the passenger appearance image in the passenger database, so that the database is compared with the case where the position of the passenger is sequentially registered in the database. Capacity can be reduced.
Further, according to the present invention, there is provided an apparatus for detecting a passenger from a plurality of passenger detection devices based on information on transportation means included in boarding pass information and operation information on transportation means registered in the operation database. Since the selection is made, passengers can be found efficiently.

It is a schematic diagram for demonstrating the flow of the various procedures from a passenger's check-in to boarding in an airport. 1 is a schematic diagram of a passenger detection system according to an embodiment of the present invention. It is a functional block diagram of an information reading apparatus. It is a block diagram which shows the outline | summary of a central processing unit and a database apparatus. It is a functional block diagram of a passenger detection device. It is a functional block diagram of a terminal device. It is a figure which shows the example of a screen displayed on the display part of a terminal device, (a) shows an information input screen, (b) shows a screen for detection confirmation, (c) is a figure which shows a detection result screen. . It is a flowchart which shows the process of an information reading apparatus. It is a flowchart which shows the registration process to the database apparatus of the passenger data in a central processing unit. It is a flowchart which shows the registration process to the database apparatus of the flight information in a central processing unit. It is a flowchart which shows the process of a terminal device. It is a flowchart which shows the process in connection with the passenger detection of a central processing unit. It is a flowchart which shows the process of a passenger detection apparatus.

[First embodiment]
Hereinafter, the present invention will be described in detail with reference to embodiments shown in the drawings. However, the components, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only unless otherwise specified. .
The passenger detection system according to the present invention is specified in a predetermined detection area from an entrance such as a security checkpoint entrance (detection area entrance) to at least one boarding gate (departure gate) from which a transportation means such as an aircraft departs. Is a system to detect passengers. In particular, the passenger detection system according to the present invention includes a plurality of passenger detection devices in the detection area for detecting a specific passenger based on the appearance image of the passenger, and the boarding gate of the transportation means used by the specific passenger from the entrance It is characterized in that a passenger detection device installed in an area leading to detects a passenger.

Hereinafter, the passenger detection system will be described based on an embodiment in the case where it is installed at an airport. It is possible to install.
[Passenger flow]
FIG. 1 is a schematic diagram for explaining the flow of various procedures from check-in to boarding for passengers (passengers) at an airport.
After arriving at the airport terminal building 1001, the passenger 1000 presents an air ticket, performs boarding procedures at the check-in counter 1002 (or an automatic check-in machine), and receives the boarding ticket 1003. If there is checked baggage, check it in at the check-in counter 1002 and go to the security checkpoint 1004.
At the security checkpoint 1004, the boarding pass 1003 and the passport are checked. In addition, in order to guarantee safe operation of the aircraft, security inspections of passengers and carry-on baggage will be conducted. The area after the security inspection is a restricted area in which safety is guaranteed in the same manner as in an aircraft, and entry of general customers is restricted. Passengers who have passed the security checkpoint 1004 are subjected to an immigration check at the immigration checkpoint 1005. Thereafter, the passenger moves to the boarding port 1006a where the boarding aircraft departs from among the plurality of boarding ports 1006, and board the aircraft from the boarding port 1006a.

[Basic system configuration]
A system configuration of a passenger detection system according to an embodiment of the present invention will be described. A passenger detection system is a system for efficiently finding passengers in a restricted area when a passenger in a predetermined detection area such as a restricted area has not boarded at the departure time of an aircraft scheduled to be boarded. It is.
The “predetermined detection area” is not necessarily limited to a restricted area where only the boarding pass holder can enter.

FIG. 2 is a schematic diagram of a passenger detection system according to an embodiment of the present invention. The passenger detection system 1 according to the present embodiment includes an information reading device 10 installed at an entrance of a security checkpoint 1004 (entrance of a detection area) serving as an entrance of a restricted area, and a plurality of devices installed in various places within the restricted area. A passenger detection device 30, a terminal device 50 for causing the passenger detection device 30 to detect a passenger, a central processing unit 100 that mediates each device constituting the passenger detection system, such as the information reading device 10 and the passenger detection device 30, and the like. And a database device (DB device 120) that accumulates information about passengers. The devices are connected to each other via a network N such as a LAN (Local Area Network) or a WAN (Wide Area Network). The passenger detection system 1 is connected to a flight information system (FIS 130), receives flight information from the FIS 130, and registers it in the DB device 120.
As shown in FIG. 1, the restricted area is a closed area having an entrance for security inspection as an entrance and a boarding entrance as an exit, and passengers enter the restricted area from the entrance to the security inspection area. Move to enter the boarding gate.

[Information reading device]
An information reading apparatus will be described. FIG. 3 is a functional block diagram of the information reading apparatus.
The information reading device 10 includes a barcode reader 11 (information reading means) that reads information (boarding ticket information) described in a boarding pass carried by a passenger, and a reading camera 15 that takes a picture of the passenger. The information reading device 10 generates passenger data from the boarding pass information and the passenger's face image, and transmits the generated passenger data to the central processing unit 100.
The information reading device 10 is installed at a position where the passenger can surely pass when entering the restricted area. Here, the restricted area is an area where passengers can be detected by the passenger detection system 1, and the information reading device 10 is installed at a position where the passenger passes when entering the restricted area.
The information reading device 10 includes a bar code reader 11 that reads information (bar code) from a boarding pass carried by a passenger, and a bar code that controls the bar code reader 11 and extracts numerical / symbol information from the read information. A reader controller 13, a reading camera 15 that captures an initial appearance image (initial face image: still image) of a passenger, and a facial feature amount (initial feature amount data) from the passenger's initial face image captured by the reading camera 15 Passenger data is generated from the image processing unit 17 for calculating the image, the numerical value / symbol information extracted by the barcode reader controller 13, the initial face image taken by the reading camera 15, and the initial feature amount data calculated by the image processing unit 17. A passenger data generation unit 19 that transmits the passenger data generated by the passenger data generation unit 19 to the central processing unit 100. .

The boarding pass includes the boarding date, flight number, passenger name, and check-in sequence number. The check-in sequence number is a unique (unique) number assigned to each boarding pass. Such information is printed on the boarding pass by, for example, a bar code (one-dimensional code). Note that the method of printing or recording information on the boarding pass is not limited to the above-described method, and information may be recorded on the boarding pass using another method such as a two-dimensional code or magnetism. The boarding date and the flight name (boarding flight name) are information for specifying the aircraft on which the passenger is boarding.
The barcode reader 11 reads information from the barcode printed on the boarding pass and outputs the read information to the barcode reader controller 13. The bar code reader controller 13 extracts the information described in the boarding pass by converting the read information from the bar code reader 11 into numerical value / symbol information, and outputs the information to the passenger data generation unit 19.

The reading camera 15 captures the face image of the passenger passing through the entrance gate of the security checkpoint at the timing when the barcode reader 11 reads the barcode on the boarding pass, and outputs the captured face image to the image processing unit 17. . The reading camera 15 is installed at a position where the faces of passengers can be photographed individually and reliably. One information reading device 10 may include a plurality of reading cameras 15. The reading camera 15 may be a video camera that captures a moving image (captures a plurality of still images continuously).
The image processing unit 17 extracts a person's face area from the passenger's face image photographed by the reading camera 15, performs normalization processing on the extracted face area, and based on a predetermined algorithm, features of the passenger's face Is calculated. That is, the image processing unit 17 detects position information such as a face outline, eyes, nose, mouth, and the like from a change in brightness, a change in color tone, and the like of the face image. Further, the image processing unit 17 calculates a facial feature amount from the positional information of each detected part of the face.

Here, when the information reading apparatus 10 includes a plurality of reading cameras 15 or when the reading camera 15 is a camera that captures a moving image, a plurality of still images can be obtained for one passenger. The unit 17 selects a still image from which a passenger's face area can be easily extracted from a plurality of acquired still images according to a predetermined algorithm, and performs image processing.
The image processing unit 17 outputs the facial feature amount data calculated from the face image and the face image from which the feature amount data is calculated to the passenger data generation unit 19.
The passenger data generation unit 19 includes information on the boarding pass read by the barcode reader 11, the initial facial image of the passenger photographed by the reading camera 15, and the initial feature data of the passenger calculated by the image processing unit 17. Is generated by the check-in sequence number and is transmitted to the central processing unit 100 via the communication unit 21. The passenger data includes a check-in sequence number, a boarding date, a flight name, a passenger name, a passenger initial face image, and passenger face feature data.

The communication unit 21 is a part that enables communication with the terminal device 50 and the like, and transmits and receives data, and is realized by a NIC (Network Interface Card), a transmission / reception circuit, or the like. The communication unit 21 transmits the passenger data generated by the passenger data generation unit 19 to the central processing unit.
The bar code reader controller 13, the image processing unit 17, and the passenger data generation unit 19 can be configured from a known computer device including a CPU, a ROM, a RAM, and the like. In this case, the barcode reader controller 13, the image processing unit 17, and the passenger data generation unit 19 are realized by the CPU executing a program stored in a predetermined memory.

[Central processing unit]
The central processing unit will be described with reference to FIG. FIG. 4 is a block diagram showing an overview of the central processing unit and the DB device. In FIG. 4, each function which comprises a central processing unit is shown with the block.
The central processing unit 100 includes a communication unit 101, a control unit 103, a flight information registration unit 105, a passenger data registration unit 107, a detection target person data extraction unit 109, a passenger detection unit selection unit 111, and a detection target person. A data generation unit 113, a detection result registration unit 115, and a transfer processing unit 117 are provided.
The communication unit 101 is a part that enables communication with the terminal device 50 and the like and transmits and receives data, and is realized by a NIC (Network Interface Card), a transmission / reception circuit, or the like.
Control unit 103, flight information registration unit 105, passenger data registration unit 107, detection target person data extraction unit 109, passenger detection device selection unit 111, detection target person data generation unit 113, detection result registration unit 115, and transfer processing unit 117 can be configured from a well-known computer device including a CPU (Central Processing Unit), a ROM, a RAM, and the like. Each of the above units is realized by the CPU executing a program stored in a predetermined memory.

The control unit 103 distributes data received via the network N to each unit of the central processing unit 100 and controls each unit constituting the central processing unit 100.
The flight information registration unit 105 registers the flight information received from the FIS 130 in the flight DB 121 of the DB device 120. The specific configuration of the flight information will be described later together with the configuration of the DB device.
The passenger data registration unit 107 registers the passenger data received from the information reading device 10 in the passenger DB 123 of the DB device 120.

When the detection target data extraction unit 109 receives a detection request from the terminal device 50, the detection target person data extraction unit 109 extracts all passenger data that matches data (detection target person data) related to the detection target person (specific passenger) included in the detection request. Extracted from the passenger DB 123 of the DB device 120. That is, the detection target person data extraction unit 109 causes the passenger DB 123 to search for initial feature data using data (detection target person data) indicating a specific passenger transmitted from the terminal device 50 as a key.
Here, detection target person data does not necessarily need to be information that can individually identify passengers. The detection target person data only needs to include data that can identify one or a plurality of passengers to be detected. For example, the detection target person data may be data including only the flight number.
Further, the detection target person data extraction unit 109 refers to the flight name included in the passenger DB 123 of the DB device 120 and extracts flight information of the aircraft with the flight name from the flight DB 121 of the DB device 120. The detection target person data extraction unit 109 outputs the boarding gate number included in the flight information extracted from the DB device 120 to the passenger detection device selection unit 111. Further, the detection target person data extraction unit 109 outputs information other than the boarding gate number among the flight information extracted from the flight DB 121 of the DB device 120 to the detection target person data generation unit 113.

The passenger detection device selection unit 111 is means for selecting at least one passenger detection device that detects a detection target person from a plurality of passenger detection devices. The passenger detection device selection unit 111 selects the passenger detection device 30 that detects the detection target person based on the boarding gate number of the aircraft received from the detection target person data extraction unit 109. In addition, the passenger detection device selection unit 111 transmits information related to the selected passenger detection device 30 to the terminal device via the communication unit 101.
Specifically, the passenger detection device selection unit 111 includes a boarding port number of an aircraft on which the detection target person boardes, and passenger detection device installation position information (hereinafter simply referred to as “installation position information”) stored in the DB device. Then, the installation position and address number of the passenger detection device 30 included in the predicted movement range of the detection target person are extracted. And the installation position of the passenger detection apparatus 30 is transmitted to a terminal device. A method for predicting the movement range of the detection target person and a method for selecting the passenger detection device 30 will be described later.
Moreover, when there is a change (addition or reduction) instruction for the passenger detection device from the terminal device 50, the passenger detection device selection unit 111 changes the passenger detection device that detects the passenger and finally detects the passenger. The address number of the passenger detection device is output to the detection target person data generation unit 113.

The detection target person data generation unit 113 sends a detection start request to the passenger detection apparatus 30 having the address number received from the terminal device 50 together with the data related to the detection target person output from the detection target person data extraction unit 109. It transmits via the communication unit 101.
The detection result registration unit 115 writes a “detection flag” indicating the detection result of the passenger in the passenger data file in the DB device 120 based on the detection result received from the passenger detection device 30.
In addition, the detection result registration unit 115 generates a detection result screen, and transmits the generated detection result screen to the terminal device that transmitted the detection start request.
The transfer processing unit 117 transmits the moving image received from the passenger detection device 30 to the terminal device 50 that transmitted the detection start request, and transmits the audio data received from the terminal device to the passenger detection device 30. Operates as a relay device.

[DB device]
The DB device will be described with reference to FIG. FIG. 4 shows the data stored in the DB device and the structure of the data.
The DB device 120 is realized by a nonvolatile storage device such as a flash memory or a hard disk device. Data is stored (written), changed (updated), or read from the central processing unit 100 to the DB device 120. The DB device 120 includes a flight DB 121 (operation database) for registering flight information, a passenger DB 123 (passenger database) for registering passenger data, an installation location DB 125 (installation location database) for storing installation location information of the passenger detection device 30, and Have. The flight information and passenger data registered in the DB device 120 are data registered or updated in real time. The installation position information registered in the installation position DB 125 is fixed information, but is updated as necessary.

Hereinafter, the configuration of each data will be described.
In the flight DB 121, information related to flights of aircraft departing from the airport is registered. Each flight information includes a flight name, departure time (on-time or change), terminal name, boarding gate number, and time stamp, and is associated with the flight name. These pieces of information are data provided from the FIS.
The flight number is the flight number of the aircraft departing from the airport, and a unique number is given to each flight. The flight information of each flight is identified from the flight information of other flights based on the flight name.
The departure time is the scheduled departure time of the aircraft, and when the departure time is changed, the changed time is updated and registered. The “departure time” includes information on the departure date. The terminal name and boarding gate number are information on the terminal building and boarding gate used by the aircraft of the flight. The time stamp is information indicating the time when the flight information is newly registered in the FIS 130 or the time when the flight information is changed and registered.

In the passenger DB 123, passenger data that is information related to a person boarding the aircraft is registered. Each passenger data includes a check-in sequence number, a boarding date, a flight name, a name, a face image (initial face image), feature amount data (initial feature amount data), a registration time, and a detection flag. These are related by a check-in sequence number. The check-in sequence number, boarding date, flight name, name, face image, and feature amount data are data transmitted from the information reading device 10. The detection flag is data transmitted from the passenger detection device 30 when a passenger is detected.
Since the check-in sequence number is a unique (unique) number given to each passenger, the data of each passenger is identified and managed from the data of other passengers based on the check-in sequence number. The boarding date and flight number are data indicating the departure date and flight number of the aircraft on which the passenger is boarding. The face image is a still image obtained by photographing a passenger's face. The feature amount data is data relating to the feature amount of the passenger's face obtained from the passenger's face image. The registration time is information on the time when the passenger data is registered in the DB device. The detection flag is a flag indicating that a passenger can be detected by the passenger detection device 30.

In the installation position DB 125, installation position information indicating the installation position of the passenger detection device is registered. Each installation position information includes an address number of the passenger detection device and an installation position of the passenger detection device.
The address number is a unique number assigned to each passenger detection device in order to specify the passenger detection device, and each passenger detection device 30 is identified from the other passenger detection devices 30 by the address number. As the address number, for example, an IP address indicating a position on the network can be used. The installation position of the passenger detection device is information indicating a substantial position of the passenger detection device installed in the airport, and is registered in association with a corresponding place on the map together with the terminal map.

[Flight Information System (FIS)]
The FIS 130 is a system that manages flight information, specifically, aircraft departure time (on time, change), flight name, departure terminal name, boarding gate number, etc., information related to aircraft operation (operation information). . When the flight information is newly registered or changed, the FIS 130 transmits the newly registered or changed flight information to the central processing unit 100. The FIS 130 is realized by a server computer. The passenger detection system 1 and the FIS 130 are connected via a network. Both systems can communicate with each other via a communication interface.

[Passenger detection device]
A passenger detection device will be described. FIG. 5 is a functional block diagram of the passenger detection device.
The passenger detection device 30 is a device that detects a passenger based on a detection start request transmitted from the terminal device 50 via the central processing unit 100.
Passenger detection device 30 processes communication image 31 with central processing unit 100, detection camera 33 that images within a predetermined range, and images captured by detection camera 33 and displays them within the imaging range. A feature amount data extraction unit 35 that extracts feature amount data of a person, an initial feature amount data of a person to be detected received from the central processing unit 100 via the communication unit 31, and a photograph extracted by the feature amount data extraction unit 35 A matching unit 37 that performs matching processing with human feature amount data, and detection information generation that generates detection information as a matching result by the matching unit 37 and transmits the generated detection information to the central processing unit 100 via the communication unit 31 Unit 39, passenger tracking unit 41 for tracking a detected passenger in an image taken by detection camera 33, and analog from digital audio data A sound generating unit 43 for generating a voice signal, and a speaker 45 for outputting an audio signal sound generating unit 43 has generated, the.

The communication unit 31 is a part that enables communication with the central processing unit 100 and the like and transmits and receives data, and is realized by a NIC (Network Interface Card), a transmission / reception circuit, or the like.
The feature amount data extraction unit 35, the matching unit 37, the detection information generation unit 39, the passenger tracking unit 41, and the voice generation processing unit 43 can be configured by a known computer device including a CPU, a ROM, a RAM, and the like. Each unit is realized by the CPU executing a program stored in a predetermined memory.
The passenger detection device 30 performs a matching process between the feature amount data of the person to be detected received from the central processing unit 100 and the feature amount data obtained from the face image of the person included in the real-time image captured by the detection camera 33. When the two feature amount data match (match), the fact that the detection target person has been detected is transmitted to the central processing unit 100. In addition, the detection target person (detected passenger) in the image captured by the detection camera 33 is automatically tracked. In addition, the passenger detection device 30 outputs the audio data transmitted from the central processing unit from the speaker 45.
The detection camera 33 is a device that captures a moving image within a predetermined capturing range. The frame image in the moving image photographed by the detection camera 33 is transmitted to the subsequent feature amount data extraction unit 35 and the passenger tracking unit 41, respectively.

The feature amount data extraction unit 35 extracts a person's face area from the frame image captured by the detection camera 33, performs normalization processing on the extracted face area image, and based on a predetermined algorithm, the person's face area is extracted. The feature amount of the face is extracted. That is, the feature amount data extraction unit 35 detects position information such as a face outline, eyes, nose, and mouth from a change in luminance, a change in color tone, and the like for a face area of a person in a frame image. Further, the feature amount data extraction unit 35 calculates a face feature amount from the position of each part of the detected face.
The feature amount data extraction unit 35 associates the calculated feature amount of the face (feature amount data) with the position in the frame image where the face is detected, and outputs it to the matching unit 37. The feature amount data extraction unit 35 transmits image data (frame image) used for extracting the feature amount of the person's face to the detection information generation unit 39.

The matching unit 37 calculates a correlation value (similarity) between the feature amount data extracted by the feature amount data extraction unit 35 and the feature amount data of the detection target person transmitted from the central processing unit 100, and the correlation value is calculated. When the predetermined reference value or more is satisfied, it is determined that the person photographed by the detection camera 33 is a passenger to be detected. When a plurality of persons are captured in the detection camera 33, it is determined that a person having the highest correlation value and satisfying a correlation value equal to or higher than a predetermined reference value among the plurality of persons is a detection target person. .
When the detection target person is found, the matching unit 37 outputs the position (region) in the frame image where the face of the detection target person is detected and the feature amount data to the detection information generation unit 39 as a matching result. To do.
The matching unit 37 transmits the position in the frame image where the face of the detection target person is detected and the tracking start command to the passenger tracking unit 41.

When a passenger who is a detection target is detected, that is, when a matching result is received from the matching unit 37, the detection information generating unit 39 generates detection information indicating that fact. The detection information includes at least the address number of the own device (passenger detection device that found the passenger), the detection flag, the check-in sequence number or the passenger name, the detected face image (still image) that is the face image when matched, and the detected face image This information includes feature quantity data.
Here, the detected face image is obtained by cutting out an image of a predetermined size including the face area of the detection target person from the frame image. The detection information generation unit 39 matches the face image based on the image data received from the feature data extraction unit 35 and the position (region) in the frame image where the face of the detection target received from the matching unit 37 is detected. Cut out (detected face image).
The detection information generation unit 39 transmits the generated detection information to the central processing unit 100 via the communication unit 31.

The passenger tracking unit 41 tracks the detected detection target person when the detection target person is detected. The passenger tracking unit 41 tracks the passenger by detecting a change in the image data of the detection target person image including the face portion matched by the matching unit 37. For tracking, a known moving body monitoring method may be used.
That is, when the passenger tracking unit 41 receives a tracking start command from the matching unit 37, the passenger tracking unit 41 includes a body region of the passenger (including the face) from the position of the passenger's face in the first frame image in which the matching unit has detected the passenger. A region including a whole body or a part of the body). Also, the temporal movement of the passenger's body area is detected by taking the difference between the luminance image of the first frame image and the luminance image of the second frame image taken after taking the first frame image. To do. Similarly, a difference is taken between the second frame image and the third frame image shot after shooting the second frame image. By sequentially performing such processing between frame images, temporal movement of the passenger's body region is continuously detected.
Further, the passenger tracking unit 41 synthesizes the marker image with the person image being tracked. That is, the passenger tracking unit 41 generates a moving image in which a detection frame (marker image) surrounding the entire body region of the person image being tracked in each frame image is combined with the original frame image, and is transmitted to the central processing unit 100. To do. The marker image may be a graphic that can trace a person image. Thus, by synthesizing the marker images, the discovered passenger and other passengers can be visually distinguished in the moving image.
The voice generation processing unit 43 generates an analog voice signal by D / A converting digital voice data received from the terminal device 50 via the central processing unit 100.
The generated audio signal is output from the speaker 45.

[Terminal device]
The terminal device will be described. FIG. 6 is a functional block diagram of the terminal device.
The terminal device 50 is a device for causing the passenger detection device 30 to detect the current position of the passenger. That is, the terminal device 50 is a means for specifying a passenger to be detected by the passenger detection device 30. The terminal device 50 can be configured from a known computer device. By transmitting information about passengers to be detected from the terminal device 50 to the central processing unit 100, the passenger detection device 30 can detect the passengers.
The terminal device 50 may be a terminal that is fixedly installed beside the boarding gate, for example, or may be a mobile terminal that can be freely transported. In the case of fixed installation, for example, a desktop personal computer or a notebook personal computer can be used. In the case of a mobile terminal, a tablet computer or a smart phone (multifunctional mobile phone) can be used.

The terminal device 50 includes a display unit 51, a display control unit 53, an input unit 55, a processing unit 57, a microphone 59, an audio processing unit 61, and a communication unit 63.
The display unit 51 is realized by a liquid crystal display, a touch panel, or the like. In the case of a touch panel, the display unit 51 can also be used as the input unit 55. The display unit 51 displays an information input screen as an initial screen before the passenger detection device 30 detects a passenger, or various types including a detection result screen transmitted from the central processing unit 100 via the communication unit 63. Data is displayed or data input from the input unit 55 is displayed.
The content displayed on the display unit 51 is controlled by the display control unit 53.

The input unit 55 is a part that delivers the input data to the processing unit 57 when the operator inputs predetermined data. The input unit 55 can be realized by a keyboard, a pointing device such as a touch panel and a mouse, or the like.
The processing unit 57 outputs data received from the central processing unit to the display control unit 53 as data to be displayed on the display unit 51. Further, the processing unit 57 transmits data input from the input unit 55 to the central processing unit 100 via the communication unit 63. The processing unit 57 is realized by a CPU (Central Processing Unit) executing a program stored in a predetermined memory.
The communication unit 63 is a part that enables communication with the central processing unit 100 and the like and transmits and receives data, and is realized by a NIC (Network Interface Card), a transmission / reception circuit, or the like.
The audio processing unit 61 performs A / D conversion on the analog audio signal input from the microphone 59, and outputs the generated digital audio data to the communication unit 63.

FIG. 7 is a diagram showing an example of a screen displayed on the display unit of the terminal device, where (a) shows an information input screen, (b) shows a detection confirmation screen, and (c) shows a detection result screen. FIG.
A display screen 70 shown in the figure is displayed on the display unit 51 of the terminal device 50. On the display screen 70, an information input tab 71a for inputting passenger data to be detected by the passenger detection device 30, a detection confirmation tab 77a for determining the position of the passenger detection device 30 for detecting the passenger, and a passenger is found. In the case of detection, a detection result tab 87a for displaying the result is displayed. By selecting each tab, it is possible to transition to an information input screen 71, a detection confirmation screen 77, and a detection result screen 87, respectively.
An information input screen 71 shown in FIG. 7A is an initial screen displayed before the detection by the passenger detection device 30 is started. On the information input screen 71, a plurality of information input fields 73 for inputting passenger information to be detected by the passenger detection device 30 and a transmission button 75 are arranged. In each information input field 73, a flight number input field 73a, a name input field 73b, and a number input field 73c for inputting a check-in sequence number are arranged. Further, by selecting the transmission button 75 arranged at the lower right of the screen, the input information (detection target person data) and the detection request are transmitted to the central processing unit 100.
In this figure, the information input fields 73 for three persons are provided, but the number of information input fields may be larger or smaller.

A detection confirmation screen 77 shown in FIG. 7B is a screen that displays the response content transmitted from the central processing unit 100 after the transmission button 75 of the information input screen 71 is selected.
On the detection confirmation screen 77, a terminal map 79 and a detection start button 81 are displayed. The map 79 displays the boarding gate number (boarding port number 83). Further, the position of the passenger detection device is displayed by an icon 85 (85a, 85b).
Of the icons 85, the icon 85a is an icon indicating a selected passenger detection device, that is, an icon indicating a passenger detection device that detects a passenger. The icon 85b is an icon indicating a passenger detection device that has not been selected. The installation position and map of the passenger detection device may be held in the terminal device 50 in advance, or may be received from the central processing device 100 as a response to the detection request each time.
First, the detection confirmation screen 77 shows the passenger detection range (position of the passenger detection device that detects the passenger) determined by the central processing unit 100. Each time the icon 85 is selected, the icon 85a and the icon 85b are alternately displayed, and the selection or non-selection of the passenger detection device 30 can be switched. That is, the operator can arbitrarily change the passenger detection range.
When the detection start button 81 is selected, a detection start request is transmitted to the central processing unit 100, and passenger detection by the selected passenger detection device is started.

A detection result screen 87 shown in FIG. 7C is a screen for displaying a detection result transmitted from the central processing unit 100 after the detection start button 81 on the detection confirmation screen 77 is selected.
On the detection result screen 87, a terminal map 79 similar to the detection confirmation screen 77 and the position of the selected passenger detection device, in other words, the passenger detection device that performed the passenger detection are displayed as an icon 85a. Furthermore, among the passenger detection devices that have detected passengers, the passenger detection device 30 that has detected a passenger is displayed as an icon 85c.
Further, the detection result screen 87 has a passenger data display column 89 that displays data of discovered passengers, and a moving image display column 91 that displays the current status of passengers.
In the passenger data display field 89, a flight name 89a and a passenger's name 89b that the discovered passenger is scheduled to board are displayed. In addition, an initial face image 89c, which is a passenger's face image taken at a security checkpoint, and a detected face image 89d, which is a passenger's face image when detected, are displayed.

The moving image display column 91 includes a moving image display column 91a for displaying a moving image transmitted from the passenger detection device that has found the passenger, and a microphone button for broadcasting the operator's voice from the speaker of the passenger detection device 30 that is transmitting the moving image. 91b and a tracking end button 91c for turning off the distribution of the moving image transmitted from the passenger detection device. By selecting the microphone button 91b, the voice data of the operator is transmitted via the central processing unit 100 to the passenger detection device 30 that is transmitting a moving image. The passenger detection device 30 that is transmitting the moving image that has received the audio data broadcasts the audio from the speaker. By selecting the microphone button 91b again during the audio broadcast, a broadcast stop request is transmitted to the passenger detection device 30 via the central processing unit 100, and the broadcast is stopped. That is, the microphone button 91b is a button for turning on / off broadcasting.
The passenger detection device 30 that is transmitting a moving image is the passenger detection device 30 located at a short distance from the passenger. Accordingly, by broadcasting the operator's voice from the speaker 45 of the passenger detection device 30 that is transmitting a moving image, it is possible to call the passenger pinpointly.

[Selection method of detection device]
A method for predicting the movement range of the passenger will be described. The passenger detection device selection unit 111 of the central processing unit 100 causes the passenger detection device 30 installed within the passenger's predicted movement range to detect the passenger.
Passengers (persons to be detected) must move to the boarding gate of the aircraft that they plan to board after passing the security checkpoint. Therefore, the predicted movement range is set so as to include at least a route (area) from the security inspection station exit to the boarding gate of the aircraft on which the person to be detected is to board. That is, the passenger detection device selection unit 111 selects the passenger detection device 30 installed in the range from the security inspection station exit to the boarding gate as a device for detecting a passenger. Thus, a passenger can be detected efficiently by selecting a passenger detection device. Here, the passenger detection device 30 installed within the predicted movement range of the passenger is a passenger detection device capable of photographing the passenger's predicted action range.

FIG. 7B is a diagram showing a detection confirmation screen displayed on the terminal device. This figure is an example in which a passenger detecting device detects a passenger boarding an aircraft using the boarding gate No. 13. The lower side of the map 79 in the figure is a security inspection area exit, and the area from the security inspection area to the boarding gate No. 13 is a detection range by the passenger detection device 30. The passenger detection device selection unit 111 causes a passenger detection device that can capture a passenger's action range predicted from the position of the boarding gate used by the aircraft on which the passenger is boarding to detect the passenger.
Note that this method of determining the detection range is an example. For example, the detection range by the passenger detection device 30 may be changed according to the time difference between the passage time of the security checkpoint (passenger data registration time) and the detection start time.
For example, if there is no significant time difference between the passage time of the security checkpoint (passenger data registration time) and the detection start time (when the time difference is less than or equal to a predetermined threshold value), from the security checkpoint to the boarding gate Detect passengers in the limited range. On the other hand, when there is a large time difference between the passage time of the security checkpoint and the detection start time (when the time difference exceeds a predetermined threshold), the detection target is a slightly wide range including the vicinity of the boarding gate. You may do it.
Furthermore, even if the location exceeds the range from the security checkpoint to the boarding gate, the area around the facility (for example, a restroom) that passengers may use regardless of age or gender must be included in the detection range. Good. In addition, when the age and gender are known, it may be detected including the vicinity of the facility where passengers of that age and sex are likely to stop and between the facility and the boarding gate.
Thus, since the passenger is detected by the passenger detection device installed in the range where the passenger is supposed to pass (or the range considered to pass) can be photographed, the passenger can be found efficiently.

In addition, since the transit passenger always passes through the transit inspection area in the arrival terminal of the aircraft, an information reading device is installed at the entrance of the transit inspection area to acquire passenger data. If the arrival terminal and departure terminal of the aircraft are the same, the detection range from the exit of the transit inspection area to the boarding gate of the next boarding aircraft may be set.
If the arrival terminal and departure terminal of the aircraft are different, after passing through the normal transit inspection area, take a connecting bus from the dedicated bus gate and move to the dedicated bus gate of the departure terminal. Transit passengers travel from a dedicated bus gate at the departure terminal to the gate of the next boarding aircraft. Accordingly, the detection range may be set between the exit of the transit inspection site and the bus gate of the arrival terminal, and between the bus gate of the departure terminal and the boarding gate of the next boarding aircraft.

The processing of the passenger detection system will be described based on the flow.
[Processing flow of information reading device]
FIG. 8 is a flowchart showing processing of the information reading apparatus.
In step S <b> 5, the information reading device 10 reads the barcode of the boarding pass presented by the passenger, and extracts information (boarding date / flight name / passenger name / check-in sequence number) described in the boarding pass. That is, the bar code reader 11 reads the bar code of the boarding pass provided by the passenger. The barcode reader 11 outputs the reading information read from the barcode to the barcode reader controller 13. The bar code reader controller 13 extracts the information described in the boarding pass by converting the read information into numerical value / symbol information, and outputs the information to the passenger data generation unit 19.

In step S <b> 10, the information reading device 10 captures a face image (initial face image) of a passenger passing through the entrance gate of the security checkpoint with the reading camera 15 at the timing of reading the bar code of the boarding pass. The reading camera 15 outputs the captured face image to the image processing unit 17. The image processing unit 17 extracts a human face area from the face image sent from the reading camera 15 according to a predetermined algorithm, performs normalization processing on the face area portion, and based on the predetermined algorithm, the face of the passenger Extract feature values. The image processing unit 17 outputs the passenger's initial face image and facial feature data to the passenger data generation unit 19.
In step S15, the information reading device 10 associates the boarding pass information output from the barcode reader controller 13, the passenger's initial face image output from the image processing unit 17, and facial feature data. Data is generated, and the generated passenger data is transmitted to the central processing unit 100 together with a registration request.

[Processing flow of central processing unit (registration processing of passenger data to DB)]
FIG. 9 is a flowchart showing registration processing of passenger data in the DB device in the central processing unit.
The control unit 103 of the central processing unit 100 constantly monitors whether there is a passenger data registration request from the information reading device 10 via the network N (No in step S20). When there is a passenger data registration request from the information reading device 10 (Yes in step S <b> 20), the control unit 103 outputs the received passenger data to the passenger data registration unit 107.
In step S25, the passenger data registration unit 107 confirms whether the received passenger data is new passenger data. That is, the passenger data registration unit 107 confirms whether the check-in sequence number included in the received passenger data is already registered in the passenger DB 123 in the DB device 120. If the check-in sequence number has already been registered in the passenger DB 123, the passenger data is not new passenger data (No in step S25), so the process is terminated without registering the passenger data.
On the other hand, when the check-in sequence number included in the received passenger data is not registered in the passenger DB 123, the passenger data is new passenger data (Yes in step S25). The passenger data and the registration time are stored in the passenger DB 123 (step S30).

[Processing flow of central processing unit (flight information registration process in DB)]
FIG. 10 is a flowchart showing a process of registering flight information in the DB device in the central processing unit.
The control unit 103 of the central processing unit 100 constantly monitors whether there is a flight information registration request from the FIS 130 via the network N (No in step S35). When there is a flight information registration request from the FIS 130 (Yes in step S <b> 35), the control unit 103 outputs the received flight information to the flight information registration unit 105.
In step S40, the flight information registration unit 105 confirms whether or not the received flight information is new flight information. That is, the flight information registration unit 105 checks whether or not the flight name included in the received flight information is already registered in the flight DB 121 in the DB device 120.

When the flight name is already registered in the flight DB (No in step S40), the time stamp of the flight information registered in the flight DB is compared with the time stamp of the flight information received from the FIS 130. (Step S42).
If the time stamp of the flight information received from the FIS 130 is newer than the time stamp of the flight information registered in the flight DB (Yes in step S42), the flight information is overwritten on the existing flight information. Processing is performed (step S45). If the time stamp of the flight information received from the FIS 130 is older than the time stamp of the flight information registered in the flight DB (No in step S42), the flight information is not registered. The process ends.
On the other hand, when the flight name is not registered in the flight DB, since the received flight information is new flight information (Yes in step S40), the received flight information is newly registered in the flight DB ( Step S50).

[Processing flow of terminal equipment]
FIG. 11 is a flowchart showing processing of the terminal device.
The processing unit 57 of the terminal device 50 constantly monitors whether or not the transmission button has been selected (No in step S55). When the information related to the passenger to be detected is input to the information input field of the terminal device 50 and the transmission button is selected (Yes in step S55), the processing unit 57 receives the information related to the passenger input via the communication unit 63 (detection target). Person data) and a detection request are transmitted to the central processing unit 100 (step S60).
In step S65, the processing unit 57 of the terminal device 50 monitors whether or not the “detection confirmation screen” and “position information of the passenger detection device” for detecting the passenger are received from the central processing unit 100 (step S65). No in S65). When the “detection confirmation screen” and the passenger “position information of the passenger detection device” are received (Yes in step S65), the processing unit 57 outputs these to the display control unit 53.

In step S <b> 70, the display control unit 53 controls the display unit 51 to display the “detection confirmation screen” and the “position information of the passenger detection device”. That is, the display control unit 53 causes the display unit 51 to display a detection confirmation screen (see FIG. 7B) on the display unit 51, and displays the position of the passenger detection device that detects the passenger on the map of the detection confirmation screen. Displayed at 85a.
In step S75, the processing unit 57 monitors whether or not the detection start button has been selected (No in step S75). In this step, an input for selecting / deselecting a passenger detection device that detects a passenger is accepted. If the detection start button is selected, the process of step S80 is performed.
In step S80, the processing unit 57 transmits a detection start request to the central processing unit 100 together with the address number of the selected passenger detection device.

In step S85, the processing unit 57 monitors whether or not the “detection result screen” is received from the central processing unit 100 (No in step S85). When the “detection result screen” is received from the central processing unit 100 (Yes in step S85), the processing unit 57 outputs these to the display control unit 53.
In step S <b> 90, the display control unit 53 controls the display unit 51 to display a “detection result screen”. That is, the display control unit 53 displays the detection result screen (see FIG. 7C) on the display unit 51, and displays the position of the passenger detection device that detected the passenger on the map of the detection result screen as an icon 85c. The detected passenger information and a video showing the current passenger status are displayed. Note that. This video is displayed until a tracking end request is sent.
In step S95, the processing unit 57 monitors whether or not the microphone button has been selected (No in step S95). If the microphone button is not selected, the process of step S115 is performed. When the microphone button is selected (Yes in step S95), the process of step S100 is performed.

In step S100, the voice processing unit 61 collects the voice signal of the operator with the microphone 59 and transmits it to the central processing unit 100 together with the broadcast command. That is, the audio processing unit 61 A / D converts the audio signal collected by the microphone 59 to generate audio data, and transmits the audio data and the broadcast command to the central processing unit 100 via the communication unit 63. To do.
In step S105, the processing unit 57 monitors whether or not the microphone button has been selected (No in step S105). That is, until the microphone button is selected again, the voice processing unit 61 continues to transmit the operator's voice to the central processing unit (the processes of steps S100 and S105 are performed). When the microphone button is selected (Yes in step S105), the process of step S110 is performed.
In step S <b> 110, the voice processing unit 61 stops collecting the operator's voice signal by the microphone 59 and transmits a broadcast stop request to the central processing unit 100. The broadcast stop request is a request transmitted to the central processing unit so as to stop the broadcast from the speaker of the passenger detection device.

In step S115, the processing unit 57 monitors whether or not the tracking end button has been selected (No in step S115). When the tracking end button is selected (Yes in step S115), the process of step S120 is performed.
In step S <b> 120, the processing unit 57 transmits a tracking end request to the central processing unit 100. That is, when the tracking end button is selected, the processing unit 57 transmits a tracking end request to the central processing unit 100 so as to stop moving image distribution from the passenger detection device.

[Processing flow of the central processing unit (processing related to passenger detection)]
FIG. 12 is a flowchart showing processing related to passenger detection of the central processing unit.
The control unit 103 of the central processing unit 100 constantly monitors whether or not a detection request including passenger data (detection target person data) is transmitted from the terminal device 50 (No in step S125). When the detection request is received (Yes in step S125), the control unit 103 outputs the detection request or the like to the detection target person data extraction unit 109.
In step S130, the detection target person data extraction unit 109 extracts the passenger's name from the DB device based on the flight name and the passenger name (or check-in sequence number) information (detection target person data) transmitted together with the detection request. Extract data necessary for detection. Specifically, the boarding gate number from which the aircraft on which the detection target person plans to board is departing from the flight DB 121 of the DB device 120 is extracted. Further, the face image and feature quantity data of the detection target person are extracted from the passenger DB 123 of the DB device 120.

The detection target person data extraction unit 109 outputs the extracted boarding gate number of the aircraft to the passenger detection device selection unit 111, and outputs data related to the detection target person other than the boarding gate number to the detection target person data generation unit 113.
In step S <b> 135, the passenger detection device selection unit 111 determines a passenger detection device candidate for detecting a passenger based on the boarding gate number information received from the detection subject data extraction unit 109. That is, the passenger detection device selection unit 111 is a position that can photograph within the predicted movement range of the detection target person among the passenger detection devices 30 registered in the installation position DB 125 of the DB device 120 based on the information of the boarding gate number. The passenger detection device 30 installed in (or within the predicted movement range) is determined as a candidate for a passenger detection device that detects a passenger.

In step S140, the passenger detection device selection unit 111 generates a detection confirmation screen 77 (image data: see FIG. 7B) and transmits it to the terminal device 50 that has transmitted the detection request together with information related to the passenger detection device. To do.
First, the passenger detection device selection unit 111 extracts the installation position and the address number of the passenger detection device 30 that are candidates in step S135 from the installation position DB 125, and displays them on the detection confirmation screen 77 as the icon 85a.
Furthermore, among the passenger detection devices 30 displayed on the map 79 of the detection confirmation screen 77, the passenger detection device selection unit 111 also sets the installation position and address number of the passenger detection device 30 that is not a candidate for detecting passengers. Extracted from the installation position DB 125 and displayed on the detection confirmation screen 77 as an icon 85b.
And the produced | generated detection confirmation screen 77 and the information regarding the passenger detection apparatus 30 extracted from installation position DB125 are transmitted to the terminal device which transmitted the detection request.

As described above, the passenger detection device 30 (non-candidate passenger detection device) that has not become a candidate for detecting passengers also extracts the address number when receiving the detection request and transmits the address number to the terminal device. Even when the passenger detection device 30 for detecting a passenger in the device 50 is changed, it is not necessary to extract the information of the passenger detection device 30 from the installation position DB 125 again, so that the passenger detection device 30 for detecting a passenger can be detected quickly. Target person data can be transmitted.
In step S145, the control unit 103 constantly monitors whether or not a “detection start request” has been received from the terminal device 50 (No in step S145). When the “detection start request” is received from the terminal device 50 (Yes in step S145), the process of step S150 is performed.
In step S150, the passenger detection device selection unit 111 outputs the address number of the “selected passenger detection device 30” transmitted from the terminal device 50 together with the “detection start request” to the detection target person data generation unit 113.

In step S152, the person-to-be-detected data generation unit 113, together with the data about the person to be detected output from the person-to-be-detected data extraction unit 109, with respect to the passenger detection apparatus 30 having the address number received from the passenger detection apparatus selection unit 111 , “Detection start request” is transmitted via the communication unit 101. Here, the detection target person data is data including at least the name (or check-in sequence number) of the detection target person and the feature amount data of the detection target person.
In step S155, the control unit 103 receives detection information from the passenger detection device. The detection information includes at least the address number of the passenger detection device that has detected the passenger, the detection flag, the check-in sequence number or the passenger name, the detected face image (still image) that is a face image when matched, and the features of the detected face image Includes quantity data. In addition, the control unit 103 includes, in the detection information, the address number of the passenger detection device that detected the detection target person, the detection flag indicating that the detection target person has been detected, and the face image (still image) of the passenger when detected. And feature amount data generated from the face image is output to the detection result registration unit 115. In addition, the control unit 103 outputs the moving image transmitted from the passenger detection device 30 to the transfer processing unit 117.

In step S160, the detection result registration unit 115 records a detection flag indicating that the detection target person has been detected in the record of the passenger in the passenger DB 123. The detection result registration unit 115 generates a detection result screen based on the detection information and the detection target person data, and transmits the detection result screen to the terminal device 50 that has transmitted the detection start request. The detection result screen generated by the detection result registration unit 115 is as shown in FIG.
The moving image displayed in the moving image display field 91a of the detection result screen 87 is received by the transfer processing unit 117 from the passenger detection device 30 and sequentially transferred to the terminal device that transmitted the detection start request. .

In step S165, the transfer processing unit 117 confirms whether a broadcast command has been received (No in step S165). If a broadcast command has been received (Yes in step S165), the process of step S170 is performed.
In step S170, the transfer processing unit 117 transfers the voice data of the operator received from the terminal device 50 together with the broadcast command to the passenger detection device 30 that has detected the passenger.
In step S175, the transfer processing unit 117 confirms whether or not a broadcast stop request has been received (No in step S175). When the broadcast stop request is received (Yes in step S175), the broadcast stop request is transmitted to the passenger detection device 30 that is broadcasting the operator's voice (step S180).
In step S185, the transfer processing unit 117 confirms whether or not a tracking end request has been received (No in step S185). When the tracking end request is received (Yes in step S185), the tracking end request is transmitted to the passenger detection device that is tracking the passenger (step S190).

[Processing flow of passenger detection device]
FIG. 13 is a flowchart showing the processing of the passenger detection device.
In step S195, the passenger detection device 30 confirms whether or not detection target person data has been received from the central processing unit 100 (No in step S195). When the detection target person data is received (Yes in step S195), the process of step S200 is performed.
In step S200, the matching unit 37 extracts feature amount data included in the detection target person data. Further, the detection information generation unit 39 extracts data (name of the detection target person or check-in sequence number) other than the feature amount data included in the detection target person data.

In step S <b> 205, the feature amount data extraction unit 35 extracts feature amount data of a person shown on the detection camera 33. That is, by extracting a face area of a person included in a moving image (a plurality of frame images) photographed by a detection camera and performing a predetermined algorithm processing on the face area, feature value data of the photographed person's face Is generated. The feature amount data extraction unit 35 outputs the generated feature amount data to the matching unit 37. The feature amount data extraction unit 35 outputs the frame image (still image) from which the feature amount data is extracted to the detection information generation unit 39.
In step S210, the matching unit 37 sequentially performs matching between each feature amount data received from the feature amount data extraction unit 35 and the feature amount data of the person to be detected received from the central processing unit. That is, the matching unit 37 calculates a correlation value (similarity) between the feature amount data received from the feature amount data extraction unit 35 and the feature amount data of the detection target person.

In step S215, the matching unit 37 determines whether or not the two feature amount data match (whether the correlation value satisfies a predetermined reference value or more). The matching unit 37 detects information indicating matching when the two feature quantity data match (Yes in step S215), that is, when the correlation value between the two feature quantity data satisfies a predetermined reference value or more. While outputting to the information generation part 39, a tracking start command is output to the passenger tracking part 41 so that the discovered person may be tracked. If the two feature amount data do not match (when the correlation value does not satisfy the predetermined reference value: No in step S215), the matching unit 37 repeats the processes in steps S210 and S215.
In step S220, the passenger tracking unit 41 starts tracking the detected person to be detected based on the tracking start command received from the matching unit 37. In other words, the passenger tracking unit 41 regards the detection target person shown in each frame image received from the detection camera 33 as a moving object, and continuously detects temporal movement of the body area of the detection target person. Further, a “frame image” surrounding the detected body region is generated, and the “frame image” is combined with each frame image and transmitted to the central processing unit.

  In step S225, the detection information generation unit 39 generates detection information and transmits it to the central processing unit 100. That is, the detection information generation unit 39 first performs a cut-out process of a detected face image (still image) that is a face image at the time of matching from the image data received from the feature amount data extraction unit 35. The detection information generating unit 39 detects the detected face image, the feature amount data of the detected face image received from the matching unit 37 in step S215, the name or check-in sequence number of the detection target person extracted in step S200, and the own device ( Information including an address number of a passenger detection device that has detected a passenger) and a detection flag is generated as detection information and transmitted to the central processing unit 100 as a piece of information.

In step S230, the sound generation processing unit 43 confirms whether a broadcast command has been received (No in step S230). If a broadcast command is received (Yes in step S230), the process of step S235 is performed.
In step S235, the voice generation processing unit 43 demodulates the voice data received from the central processing unit 100 and broadcasts the operator's voice from the speaker 45.
In step S240, the sound generation processing unit 43 confirms whether a broadcast stop request has been received (No in step S240). If a broadcast stop request is received (Yes in step S240), the broadcast is stopped in step S245.
In step S250, the passenger tracking unit 41 confirms whether or not a tracking end request has been received (No in step S250). When the tracking end request is received (Yes in step S250), the passenger tracking is ended (step S255). That is, in step S255, the distribution of the moving image is stopped.

As described above, according to the passenger detection system, when the passenger needs to be detected, the passenger detection device detects the passenger, so that the load on the devices constituting the system can be reduced. In addition, since the distributed processing is performed by each device constituting the system, the processing load is not concentrated on the central processing unit.
In the present embodiment, the boarding pass information necessary for passenger detection and the initial feature value data obtained from the passenger's face image are stored in the passenger database, so that the position of the passenger is sequentially registered in the database. Database capacity can be greatly reduced.
Furthermore, according to this embodiment, based on the flight name contained in boarding pass information, and the information of the boarding gate in the flight information registered into flight DB, a passenger is detected from several passenger detection apparatuses. Select equipment. By this selection, it is possible to limit the passenger detection range to the passenger's predicted movement range, so that the passenger can be detected efficiently in a short time.

[Second Embodiment]
The passenger data registered in the passenger DB 123 may store a boarding flag indicating completion of boarding. Here, the boarding flag is a flag indicating that the passenger has boarded. Whether or not the boarding is completed is recorded by setting a boarding flag in the data of passengers who have already boarded, and setting no boarding flag in the data of passengers who have not boarded.
For example, the detection target person data extraction unit 109 extracts all passenger data that matches the detection target person data from the DB device 120. When the boarding flag is set, a message to that effect is transmitted to the terminal device, and a process of excluding the passenger from the detection target by the passenger detection device 30 is performed. Passengers who do not have a boarding flag are subject to detection by the passenger detection device 30.
By registering the boarding flag in the passenger DB 123, when only the flight name is specified as the detection target person data, all the passengers who have not boarded among all the passengers boarding the flight are to be detected. It can be.

  DESCRIPTION OF SYMBOLS 1 ... Passenger detection system, 10 ... Information reading apparatus, 11 ... Bar code reader, 13 ... Bar code reader controller, 15 ... Reading camera, 17 ... Image processing part, 19 ... Passenger data generation part, 21 ... Communication part, DESCRIPTION OF SYMBOLS 30 ... Passenger detection apparatus, 31 ... Communication part, 33 ... Detection camera, 35 ... Feature-value data extraction part, 37 ... Matching part, 39 ... Detection information generation part, 41 ... Passenger tracking part, 43 ... Voice generation process part, 45 ... Speaker, 50 ... Terminal device, 51 ... Display unit, 53 ... Display control unit, 55 ... Input unit, 57 ... Processing unit, 59 ... Microphone, 61 ... Audio processing unit, 63 ... Communication unit, 70 ... Display screen, 71 ... Information input screen, 73 ... Information input field, 75 ... Send button, 77 ... Detection confirmation screen, 79 ... Map, 81 ... Detection start button, 83 ... Boarding gate number, 87 ... Detection result screen, 89 ... Passenger data display Field, DESCRIPTION OF SYMBOLS 1 ... Movie display column, 100 ... Central processing unit, 101 ... Communication part, 103 ... Control part, 105 ... Flight information registration part, 107 ... Passenger data registration part, 109 ... Detection object person data extraction part, 111 ... Passenger detection apparatus Selection part 113 ... Detection subject data generation part 115 ... Detection result registration part 117 ... Transfer processing part 120 ... DB device 121 ... Flight DB 123 ... Passenger DB 125 ... Installation position DB 130 ... FIS 1000 ... Passenger, 1001 ... Airport terminal building, 1002 ... Check-in counter, 1003 ... Boarding pass, 1004 ... Security checkpoint, 1005 ... Immigration checkpoint, 1006 ... Boarding gate

In order to solve the above-mentioned problem, the invention according to claim 1 is a passenger detection system for detecting a specific passenger in a predetermined detection area from the entrance to at least one boarding gate, and is installed at the entrance. Information reading means for reading boarding pass information including a boarding flight name for specifying a transportation means on which the passenger is boarded from a boarding pass possessed by the passenger, and a reading camera for photographing the passenger A passenger database for storing the reading device, boarding pass information acquired by the information reading device, and the initial feature data of the passenger obtained from the initial appearance image of the passenger photographed by the reading camera; A central processing unit that extracts initial feature data of the specific passenger from the passenger database, and a detection camera that is installed in the detection area and images the detection area And matching that determines that the person photographed by the detection camera is the specific passenger when the feature data obtained from the appearance image of the person photographed by the detection camera matches the initial feature data. A plurality of passenger detection devices each having a means, and an operation database for storing the boarding flight name and the boarding gate in association with each other as operation information relating to at least one means of transportation departing from the boarding gate, The central processing unit is at least one of the plurality of passenger detection devices based on information on transportation means included in boarding pass information acquired from the passenger database and operation information on transportation means obtained from the operation database. includes a passenger detection system selecting means for selecting one of the passenger detecting device, wherein the passenger detection system selecting means, the It acquires information boarding gate from the operation database based on the flight name obtained from the customer database, characterized by selecting the installed passenger detection system in an area leading to the boarding gate from said inlet.

The detection area is a closed area having an entrance and a boarding gate as an exit. An information reading device is installed at the entrance of the detection area, and a plurality of passenger detection devices are installed in the detection area. When the passenger enters the detection area, the information reading means of the information reading device reads boarding pass information including information on the transportation means from the boarding pass, and the reading camera of the information reading device takes an appearance image of the passenger. Information acquired by the information reading device is registered in the passenger database. When detecting a specific passenger, first, the central processing unit extracts information on the transportation means on which the specific passenger is boarded from the passenger database, and extracts operation information about the extracted transportation means from the operation database. Subsequently, the central processing unit selects a passenger detection device that detects a passenger from a plurality of passenger detection devices installed in the detection area, based on the extracted operation information about the transportation means. The passenger detection device selected by the central processing unit determines that the specific passenger has been found when the feature data extracted from the person photographed by the detection camera matches the initial feature data of the specific passenger. .
Further, the central processing unit acquires information on a boarding gate as a place where the transportation means departs from the boarding flight name included in the boarding pass information. Here, the passenger enters the detection area from the entrance, moves in the detection area, and enters from the boarding gate. The central processing unit selects a passenger detection device installed in the area from the entrance to the boarding gate as a device for detecting passengers.
By detecting it, passengers can be detected efficiently.
According to a second aspect of the present invention, the passenger detection device includes a passenger tracking unit that tracks the image of the specific passenger determined by the matching unit, and the passenger tracking unit adds a marker image to the person image being tracked. It is characterized by combining.
In the invention of claim 2, since the marker image is synthesized with the image of the specific passenger found, the specific passenger and other passengers can be visually distinguished.

The invention of claim 3 is a passenger detection system for detecting a specific passenger in a predetermined detection area from the entrance to at least one boarding gate, and is installed at the entrance and boarding a boarding pass carried by the passenger An information reading device comprising information reading means for reading ticket information, a reading camera for photographing the passenger, boarding pass information acquired by the information reading device, and an initial appearance image of the passenger photographed by the reading camera A passenger database that stores the initial feature value data of the passenger obtained in association with each other, a central processing unit that extracts the initial feature value data of the specific passenger from the passenger database, and installed in the detection area A detection camera that captures the inside of the detection area, feature amount data obtained from an appearance image of a person captured by the detection camera, and the initial feature amount data. A matching unit that determines that the person photographed by the detection camera is the specific passenger when the image matches the image, and a marker image is synthesized with the image of the specific passenger determined by the matching unit Passenger tracking means for tracking the specific passenger thereby, a plurality of passenger detection devices, respectively, and an operation database for storing operation information on at least one means of transportation departing from the boarding gate, The central processing unit is at least one of the plurality of passenger detection devices based on the information on the transportation means included in the boarding pass information acquired from the passenger database and the operation information on the transportation means acquired from the operation database. Passenger detection device selection means for selecting two passenger detection devices is provided.
The detection area is a closed area having an entrance and a boarding gate as an exit. An information reading device is installed at the entrance of the detection area, and a plurality of passenger detection devices are installed in the detection area. When the passenger enters the detection area, the information reading means of the information reading device reads boarding pass information including information on the transportation means from the boarding pass, and the reading camera of the information reading device takes an appearance image of the passenger. Information acquired by the information reading device is registered in the passenger database. When detecting a specific passenger, first, the central processing unit extracts information on the transportation means on which the specific passenger is boarded from the passenger database, and extracts operation information about the extracted transportation means from the operation database. Subsequently, the central processing unit selects a passenger detection device that detects a passenger from a plurality of passenger detection devices installed in the detection area, based on the extracted operation information about the transportation means. The passenger detection device selected by the central processing unit determines that the specific passenger has been found when the feature data extracted from the person photographed by the detection camera matches the initial feature data of the specific passenger. .
In addition, since the marker image is synthesized with the image of the specific passenger found, the specific passenger and other passengers can be visually distinguished.
The invention according to claim 4 is a terminal comprising: input means for inputting the data of the specific passenger; and communication means for transmitting data indicating the specific passenger input from the input means to the central processing unit. The initial processing amount data acquired from the passenger database by causing the passenger database to search the initial feature amount data using data indicating a specific passenger transmitted from the terminal device as a key. Is transmitted to the passenger detection device selected by the passenger detection device selection means.
In the invention of claim 4, an arbitrary passenger can be identified from the terminal device and can be detected by the passenger detection device.

According to a fifth aspect of the present invention, the terminal device includes a microphone that collects an audio signal, and a communication unit that transmits the collected audio data to the passenger detection device. The passenger detection device includes the communication unit. A speaker is provided for outputting audio data received from the TV.
In the invention of claim 5 , each passenger detection device is provided with a speaker, and the voice of the operator is broadcast from the speaker of the passenger detection device at a short distance from the specific passenger, thereby pinpointing the specific passenger. You can make a call at .
Invention 請 Motomeko 6, wherein the inlet is a security checkpoint inlet, wherein the predetermined detection area is restricted area.
The invention of claim 6 detects a passenger in a restricted area from the security checkpoint entrance to the boarding gate.

Claims (6)

A passenger detection system for detecting a specific passenger in a predetermined detection area from an entrance to at least one boarding gate,
An information reading device provided with information reading means that is installed at the entrance and reads boarding pass information of a boarding pass carried by a passenger, and a reading camera that photographs the passenger,
Passenger database that stores the boarding pass information acquired by the information reading device and the initial feature data of the passenger obtained from the initial appearance image of the passenger taken by the reading camera in association with each other;
A central processing unit for extracting initial feature data of the specific passenger from the passenger database;
A detection camera that is installed in the detection area and images the inside of the detection area, and when the feature amount data obtained from the appearance image of the person imaged by the detection camera matches the initial feature amount data. A plurality of passenger detection devices each comprising a matching means for determining that the person photographed by the detection camera is the specific passenger;
An operation database storing operation information related to at least one means of transportation departing from the boarding gate, and
The central processing unit is at least one of the plurality of passenger detection devices based on information on transportation means included in boarding pass information acquired from the passenger database and operation information on transportation means obtained from the operation database. A passenger detection system comprising a passenger detection device selection means for selecting one passenger detection device. The boarding pass information includes a boarding flight name that identifies the means of transportation on which the passenger is boarded,
In the operation database, the boarding flight name and the boarding gate are registered in association with each other,
The passenger detection device selection means acquires boarding gate information from the operation database based on the boarding flight name acquired from the passenger database, and is installed in an area from the entrance to the boarding gate. The passenger detection system according to claim 1, wherein the passenger detection system is selected. An input unit that inputs data of the specific passenger; and a communication unit that transmits data indicating the specific passenger input from the input unit to the central processing unit.
The central processing unit causes the passenger database to search for the initial feature value data using the data indicating the specific passenger transmitted from the terminal device as a key, and the initial feature value data acquired from the passenger database 3. The passenger detection system according to claim 1, wherein the transmission is transmitted to a passenger detection device selected by a detection device selection means. The terminal device includes a microphone that collects an audio signal, and a communication unit that transmits the collected audio data to the passenger detection device,
The passenger detection system according to claim 3, wherein the passenger detection device includes a speaker that outputs audio data received from the communication unit. The passenger detection device includes a passenger tracking unit that tracks an image of the specific passenger determined by the matching unit,
The passenger detection system according to any one of claims 1 to 4, wherein the passenger tracking unit synthesizes a marker image with a person image being tracked.   The passenger detection system according to any one of claims 1 to 5, wherein the entrance is a security checkpoint entrance, and the predetermined detection area is a restricted area.

Images