JP2009284442A - Apparatus and method for person identification - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically identify a correspondence between a person and a portable terminal to properly provide data and functions possessed by the portable terminal to a corresponding person. <P>SOLUTION: Cameras C1 to Cn photograph the inside and outside of a vehicle, and a rider detection section 20 detects positions of riders by image recognition of the photographed results of the cameras. A portable terminal detection section 30 detects positions of portable terminals from the reception intensity at antennas A1 to An. A correspondence determination section 15 determines corresponding relations using the positions of the persons and the positions of the portable terminals. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a person authentication apparatus and person authentication method for person authentication, and more particularly to a person authentication apparatus and person authentication method for recognizing a correspondence relationship between a communication terminal and a person.

  In recent years, car navigation systems have become widespread, and it has become common for automobiles to be equipped with in-vehicle devices capable of viewing music and video, displaying navigation information, and the like. Further, such an in-vehicle device communicates with a portable communication terminal device such as a mobile phone or a PDA (Personal Digital Assistant) carried by a passenger, and performs data transmission / reception between the portable communication terminal device / the in-vehicle device. It has become.

  Thus, when communicating between communication devices, a wireless communication standard called Bluetooth (registered trademark) is well known. Bluetooth (registered trademark) is a wireless communication standard that uses a frequency band of 2.4 GHz, and wireless communication within a radius of several tens of meters is possible.

  By the way, with the spread of portable communication terminal devices, a plurality of communication terminal devices are often brought into the vehicle. However, in this case, the correspondence relationship between the plurality of communication terminal devices and the person who is the owner is unclear, and it cannot be said that it is convenient for the user.

  As a conventional technique that considers the correspondence between a person in a vehicle and a communication terminal device, Patent Document 1 detects that a mobile phone is in use by a radio wave detector or the like, and is used by a driver from shooting with a camera. A technique for outputting a warning when it is detected is disclosed.

JP 2007-249478 A

  However, in the related art, for a person who does not have a communication terminal in his / her hand, the communication terminal corresponding to that person (owned by that person) cannot be specified. For this reason, there is a problem that even when information useful for vehicle setting or content that can be used while riding is stored in the communication terminal, such information and content cannot be used effectively.

  Also, in recent years, hands-free calling functions and e-mail reading functions have been realized so that the functions of mobile phones can be used within the range that does not hinder driving operations, but when multiple mobile phones exist in the car In order to determine which mobile phone is the target of the above function, it has been required to recognize the correspondence between the mobile phone (communication terminal) and the occupant (person).

  The present invention has been made to solve the above-described problems in the prior art and to solve the problems. The present invention automatically recognizes correspondence between a person and a communication terminal and supports data and functions of the communication terminal. It is an object of the present invention to provide a person authentication device and a person authentication method that can be appropriately provided to a person who performs a job.

  In order to solve the above-described problems and achieve the object, a person authentication device and a person authentication method according to the present invention detect a person by image recognition, detect a communication terminal by wireless communication, and detect the position of the person and the communication terminal. The correspondence between the person and the communication terminal is determined by comparing the position.

  According to the present invention, it is possible to obtain a person authentication device and a person authentication method capable of automatically recognizing correspondence between a person and a communication terminal and appropriately providing data and functions of the communication terminal to the corresponding person. There is an effect that can be.

  Hereinafter, a person authentication device and a person authentication method according to the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic configuration diagram showing a schematic configuration of an occupant authentication device according to an embodiment of the present invention. The occupant authentication device 10 shown in the figure is an in-vehicle device mounted on a vehicle, and includes a plurality of cameras C1 to Cn, a plurality of antennas A1 to An, a navigation device 41, an audio device 42, an air conditioner 43, and a vehicle body control device 44. Connected.

  The cameras C1 to Cn are mounted on the vehicle and are shooting means for shooting inside and outside the vehicle. The antennas A1 to An are communication means that are also mounted on the vehicle and communicate with a communication terminal (for example, a mobile phone) inside or outside the vehicle by near field communication (for example, Bluetooth (registered trademark)). Here, only one camera may be used as long as the inside and outside of the vehicle can be photographed, but a plurality of antennas are provided for the communication means. This is because the position of the communication terminal is specified by comparing the reception strengths of the positions at a plurality of antenna positions.

  The occupant authentication device 10 includes a video reception unit 11, a video analysis unit 12, a radio wave reception unit 13, an occupant detection unit 20, a radio wave intensity analysis unit 14, a mobile terminal detection unit 30, a correspondence determination unit 15, and a service control unit 16. Have

  The video reception unit 11 receives video captured by the cameras C <b> 1 to Cn and passes the video to the video analysis unit 12. The video analysis unit 12 analyzes the video captured by the video reception unit 11, specifies an image area where an object exists, and outputs the image region to the occupant detection unit 20. The occupant detection unit 20 performs image recognition processing on the output of the video analysis unit 12 to detect a person.

  Specifically, the occupant detection unit 20 includes a person recognition unit 21, a person position detection unit 22, a person motion detection unit 23, and a tracking processing unit 24 therein. The person recognizing unit 21 performs a process of identifying an area that is an image of a person among the image areas output from the video analysis unit 12. At this time, if it is possible to calculate the feature amount on the image that can identify the person, it is desirable to calculate (the use of the feature amount of the person will be described later).

  The person position detection unit 22 is a processing unit that detects the position of the person recognized by the person recognition unit 21. The position of the person can be detected by comparing the installation position and shooting direction of the camera that shot the person image, the position of the person image in the image, and the shooting results of other cameras.

  The human motion detection unit 23 is a processing unit that detects a human motion by recognizing a shape change of a human image from a continuous image. The tracking processing unit 24 is a processing unit that detects the movement of the person by recognizing a change in the position of the person's image from the continuous images. The result of this tracking is fed back to the person position detection unit 22.

  The occupant detection unit 20 outputs the result of the above process, that is, the result of person recognition, the result of position detection, the result of motion detection, and the result of tracking processing (result of movement detection) to the correspondence determination unit 15.

  The radio wave receiver 13 receives the radio waves received by the antennas A <b> 1 to An and passes them to the radio wave intensity analyzer 14. The radio wave intensity analysis unit 14 analyzes the reception intensity of each antenna acquired from the radio wave reception unit 13 and outputs the analysis result to the mobile terminal detection unit 30. The mobile terminal detection unit 30 detects a mobile terminal using the analysis result of the radio wave intensity analysis unit 14.

  Specifically, the mobile terminal detection unit 30 includes an antenna determination unit 31, a terminal position detection unit 32, and a terminal movement detection unit 33 therein. The antenna discriminating unit 31 discriminates each antenna and obtains the reception intensity for each antenna.

  The terminal position detection unit 32 is a processing unit that compares the reception intensity for each antenna and detects the position of the mobile terminal. The detection of the position of the terminal can be performed by comparison in consideration of the installation position of the antenna that receives the radio wave from the terminal and the reception direction.

  The terminal movement detection unit 33 is a processing unit that detects the movement of the terminal from the transition of the reception intensity at each antenna.

  The portable terminal detection unit 30 outputs the result of the above processing, that is, the detection result correspondence determination unit 15 of the terminal position and the movement state.

  The correspondence determination unit 15 compares the position of the person output from the occupant detection unit 20 with the position of the mobile terminal output from the mobile terminal detection unit 30 to determine the correspondence between the person and the mobile terminal. It is a determination means.

  An outline of determination by the correspondence determination unit 15 will be described with reference to FIGS. 2 and 3. Here, an explanation will be given taking as an example the situation from when the occupant approaches from outside the vehicle, gets into the driver's seat, and places the mobile phone in the passenger seat.

  First, as shown in FIG. 2, a person is recognized by image recognition from the imaging results of the cameras C1 to Cn that can capture the outside of the vehicle, and the person's movement (approach to the vehicle) using an optical flow or the like. ) Is detected.

  When the occupant enters the vehicle, an operation of the person placing the mobile phone on the front passenger seat is detected using an optical flow or the like with respect to a captured image of the camera that can shoot the vehicle among the cameras C1 to Cn.

  At the same time, as shown in FIG. 3, the antennas A1 to An communicate with the mobile terminal at their respective positions. Since the reception intensity can vary depending on the distance between the mobile terminal and the antenna, the position of the mobile phone can be known from the reception radio wave intensity of the antenna. Further, the moving direction of the mobile phone can be obtained from the change in the radio wave intensity depending on the position of the mobile phone.

  For example, as shown in FIG. 3, when an occupant carrying a mobile phone approaches the vehicle, only the nearest antenna reacts with a weak radio wave intensity, and then receives each antenna as the occupant approaches the vehicle. Strength increases.

  Next, after the occupant gets into the vehicle, the reception strength at the antenna on the driver's seat has changed from strong to weak, and the reception strength at the antenna on the passenger's seat has changed from weak to strong. It can be seen that the driver's seat has moved to the passenger seat.

  It is assumed that the positions of the mobile phone and the person are approximately the same until the mobile phone is separated from the person who is the owner. Accordingly, the determination handling unit 15 sets the person closest to the mobile phone and the person whose mobile phone movement direction matches the person's movement direction as the owner of the mobile phone.

  After determining the owner of the mobile phone, the determination handling unit 15 associates the mobile phone with the person and stores the information in the occupant information holding unit 15a. Thereafter, as long as the associated person (owner of the mobile phone) is within the range that can be photographed, the tracking is performed.

  Therefore, for example, after leaving the back seat door and placing the mobile phone on the back seat, the person's position is separated from the mobile phone position and the person's position Even if only changes, the position of the person who is the owner of the mobile phone can always be monitored, and the correspondence between the person and the mobile phone can be maintained.

  The service control unit 16 uses the determination result of the correspondence determination unit 15, that is, the correspondence relationship between the person and the mobile terminal, and uses the information held by the mobile terminal for the seating position of the person corresponding to the mobile terminal. Service control means to be executed.

  Here, the navigation device 41, the audio device 42, the air conditioner 43, and the vehicle body control device 44 are illustrated as devices that are controlled by the service control unit 16.

  The navigation device 41 is a device that specifies position information of the own vehicle using a GPS artificial satellite or the like, provides route guidance to the destination, and provides information related to the area for traffic. The audio device 42 is an in-vehicle device that performs reproduction of music data and video data, reception of radio broadcasts, television broadcasts, and the like. The air conditioner 43 is an in-vehicle device that performs an air conditioning process such as cooling and heating in the passenger compartment. The vehicle body control unit 44 is a device that controls various in-vehicle devices such as a seat position, an angle, a seat surface height, an angle adjustment of a rearview mirror and a side mirror, and a handle height and an angle.

The service control unit 16 controls these in-vehicle devices and provides the service shown in FIG.
As shown in FIG. 4, the services provided to the driver's seat occupants include setting / control of the air conditioner 43 using setting information stored in the portable terminal, and audio devices such as music stored in the portable terminal. Content playback, music and video playback settings change, reading of incoming mail to mobile terminal device, hands-free call function, mirror position by control of body control unit 44 using setting information stored in mobile terminal Control, seat position (including angle) control, provision of the home position stored in the portable terminal to the navigation device 41, and the like can be executed.

  The services provided to passengers other than the driver's seat include remote operation of the portable terminal, setting / control of the air conditioner 43 using setting information stored in the portable terminal, and setting information stored in the portable terminal apparatus. The seat position (including the angle) control by the control of the vehicle body control unit 34 using can be executed. It is also possible to execute content reproduction by an audio device such as music stored in the mobile terminal device, and change settings in music and video reproduction.

  Next, the processing operation of the occupant authentication device 10 will be described with reference to FIG. The flowchart shown in FIG. 5 is a main flow of the occupant authentication device 10 and is a process that is repeatedly executed. It is particularly preferable that the occupant authentication device 10 be operated even when the vehicle is parked so that occupant detection can be performed when the occupant approaches.

  First, the occupant detection unit 20 executes an occupant detection process (step S101), and the portable terminal detection unit 30 executes a portable terminal detection process (step S102).

  When an occupant is detected by the occupant detection process and a portable terminal is detected by the portable terminal detection process, the correspondence determination unit 15 performs the correspondence determination process (step S103). Furthermore, the occupant tracking determination process is performed for the occupant for which the correspondence relationship is set (step S104), and the occupant is continuously tracked until it is determined that the tracking is finished (step S105, Yes).

  FIG. 6 is a flowchart for explaining details of the occupant detection process. As shown in FIG. 6, the occupant detection unit 20 first detects a person by the person recognition unit 21 (step S201), and then detects the position of the person by the person position detection unit 22 (step S202). Then, the human motion detection unit 23 detects the human motion and its direction (step S203), and the occupant detection process is terminated.

  FIG. 7 is a flowchart for explaining the details of the portable terminal detection process. As shown in FIG. 7, the mobile terminal detection unit 30 first recognizes the antenna by the antenna discrimination unit 31 (step S301), and then the terminal position detection unit 32 detects the position of the terminal from the radio field intensity of each antenna. (Step S302), the terminal movement detection unit 33 detects the moving direction of the terminal from the transition of the radio field intensity (Step S303), and the terminal detection process is terminated.

  FIG. 8 is a flowchart for explaining the details of the correspondence determination process. First, the correspondence determination unit 15 compares the position of the person with the position of the mobile terminal (step S401). As a result, when the distance from the position of the person to the portable terminal is equal to or less than the threshold (Yes in step S402), the movement direction (operation direction) of the person is compared with the movement direction of the portable terminal (step S403).

  If the movement direction of the person matches the movement direction of the mobile terminal (Yes in step S404), the person and the mobile terminal are associated with each other (step S405), and the correspondence determination process ends. If the distance from the person to the portable terminal is larger than the threshold (No at Step S402), or if the movement direction of the person and the moving direction of the portable terminal do not match (No at Step S404), the correspondence determination process is performed as it is. finish.

  FIG. 9 is a flowchart illustrating details of the occupant tracking determination process. The occupant detection unit 30 determines “tracking” when the correspondence is set by the correspondence determination unit 15 (step S501, Yes) and a person is captured in the camera (step S502, Yes) (step S502). S503).

  On the other hand, when the correspondence relationship is not set (step S501, No), or when the person for whom the correspondence is set is not captured by the camera (out of the camera shooting range) (step S502, No), “tracking” “End” is determined (step S504).

  In addition, the state reflected in the camera may be reflected in any of the plurality of cameras, and when it is out of the shooting range of one camera and is in the shooting range of another camera (for example, a camera that takes a picture outside the vehicle when riding) If the camera goes out of the shooting range and enters the shooting range of the camera that shoots the inside of the car), the camera is switched and tracking is continued.

  In this way, tracking of a person ends when the person goes out of the shooting range of the camera, but once the person's feature value is calculated and held as described above, the person is once out of the shooting range of the camera. When the person returns again, the person can be recognized based on the feature amount and tracking can be continued. From the next time, the person can be personally authenticated by image recognition and used for service control and security operation control.

  Further, in the configuration in which the person is tracked until the person disappears from the camera, the tracking is continued even when the vehicle is running after getting on. However, once the vehicle has started running, it is rare for a person to move, so the position of the person may be determined at the start of running, and tracking may be temporarily suspended or the tracking interval may be extended.

  As described above, in the occupant authentication device 10 according to the present embodiment, the occupant detection unit 20 detects the position of the occupant by image recognition, and the mobile terminal detection unit 30 detects the position of the mobile terminal from the reception strength at the plurality of antennas. The position is detected, and the correspondence is set using the position of the person and the position of the mobile terminal. Therefore, it is possible to automatically recognize the correspondence between the person and the mobile terminal and appropriately provide the data and functions of the communication terminal to the corresponding person.

  As described above, the person authentication device and the person authentication method according to the present invention are useful for person authentication, and are particularly suitable for associating a mobile terminal with a user.

1 is a schematic configuration diagram illustrating a schematic configuration of an occupant authentication device according to an embodiment of the present invention. It is explanatory drawing explaining the detection of the passenger | crew by image recognition. It is explanatory drawing explaining the position detection of a portable terminal by the comparison of radio wave reception intensity. It is explanatory drawing explaining the service control based on a response | compatibility of a person and a portable terminal. It is a flowchart explaining the processing operation of the passenger | crew authentication apparatus concerning the Example of this invention. It is a flowchart explaining the detail of a passenger | crew detection process. It is a flowchart explaining the detail of a portable terminal detection process. It is a flowchart explaining the detail of a corresponding | compatible determination process. It is a flowchart explaining the detail of a passenger | crew tracking determination process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Passenger authentication apparatus 11 Image | video reception part 12 Image | video analysis part 13 Radio wave reception part 14 Radio wave intensity analysis part 15 Correspondence determination part 15a Passenger information holding part 16 Service control part 20 Passenger detection part 21 Person recognition part 22 Person position detection part 23 Human motion Detection unit 24 Tracking processing unit 30 Portable terminal detection unit 31 Antenna discrimination unit 32 Terminal position detection unit 33 Terminal movement detection unit 41 Navigation device 42 Audio device 43 Air conditioner 44 Car body control device C1 to Cn camera A1 to An antenna

Claims (7)

Person detecting means for detecting a person by image recognition;
Communication terminal detection means for detecting a communication terminal by wireless communication;
A correspondence determination unit that compares the position of the person detected by the person detection unit with the position of the communication terminal detected by the communication terminal detection unit to determine the correspondence between the person and the communication terminal;
A person authentication device comprising:   Service control means for executing a service using information held by the communication terminal with respect to a position where a person corresponding to the communication terminal exists, using a determination result by the correspondence determination means. The person authentication device according to claim 1.   The person is an occupant of the vehicle, and the service control means is configured to change a vehicle setting with respect to the seating position of the person, perform remote operation using an in-vehicle input / output device of the communication terminal, and reproduce and output data held by the communication terminal. The person authentication device according to claim 2, wherein at least one of them is controlled.   The person detection means further detects movement and / or movement of the person, the communication terminal detection means further detects movement of the communication terminal, and a correspondence determination means includes movement and / or movement of the person. The person authentication apparatus according to claim 1, wherein the correspondence relationship is determined by further comparing the movement of the communication terminal.   5. The person detection unit according to claim 1, wherein the person detection unit tracks the movement of the person even after being associated with the communication terminal by the correspondence determination unit, and updates the position where the person exists. The person authentication device according to any one of the above.   The person authentication apparatus according to claim 1, wherein the correspondence determination unit retains characteristics of an image of a person associated with the communication terminal. A person detection step of detecting a person by image recognition;
A communication terminal detection step of detecting a communication terminal by wireless communication;
A correspondence determination step of comparing the position of the person detected in the person detection step with the position of the communication terminal detected in the communication terminal detection step to determine the correspondence between the person and the communication terminal;
The person authentication method characterized by including.

Images