JP2019153908A - Image authentication system, image authentication method and image authentication program - Google Patents

Abstract

To provide an image authentication system, an image authentication method and an image authentication program which allow a monitoring person to visually recognize a monitoring object immediately and easily.SOLUTION: An image authentication system 100 includes: an imaging terminal 2 for acquiring a monitoring image; an imaging side storage section 26 for storing the monitoring image; a clip section 22 for clipping as an input feature image included in the monitoring image, and generating input image data containing the input feature image, image identification information for identifying the monitoring image, and positional information for specifying the clip position of the input feature image in the monitoring image; a collation section 12 for collating the input image data and a prestored registration image database of a monitoring object, and detecting an input image data corresponding to the registration image database as the detection image data; and a monitor 35 for displaying a position of the input feature image and at least a part including the input feature image of the monitoring image identified based on the detection image data, when the detection image data is detected as a result of collation by the collation section 12.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image authentication system, an image authentication method, and an image authentication program.

  In facilities where a large number of people enter and exit, such as commercial facilities, airports, banks, office buildings, and condominiums, an image captured by an imaging terminal such as a surveillance camera (hereinafter referred to as a surveillance image) is used to ensure security. Systems that use it to monitor the entry and exit of people are used. The monitoring image is transmitted from the imaging terminal to the server as needed, and information is extracted by image analysis. For example, a face image is cut out as a feature image from the monitoring image, and stored as input image data including a feature amount of the image, for example, a feature amount obtained by quantifying the face image or the feature of the face. This input image data is used, for example, for searching for a face image (for example, Patent Document 1 and Patent Document 2). A monitoring image is transmitted to a user terminal as necessary, and may be confirmed by a person (for example, a monitoring person) through a display unit of the user terminal (for example, Patent Document 3). Note that the monitoring image is handled as a still image or a moving image in which the still images are arranged in time series.

  Patent Document 1 discloses an invention relating to a face image collation system (an example of an image authentication system) that extracts a corresponding face image candidate from face images registered or stored in a face image data server. In the face image collation system, a face image to be searched is acquired from the face image registration device, and the registered face image data is searched for specific face image parts (eyes, nose, mouth, contour, etc.) in the registration unit of the face image data server. Are registered in the database as data combined with the original face image. In this face image matching system, an image of the entire face corresponding to the part of the face image to be searched for by the face image search device can be searched and transmitted to the face image search device for display.

  Patent Document 2 discloses an invention related to a personal authentication system or the like (an example of an image authentication system) that can efficiently perform personal collation for more users. In this personal authentication system, the control server divides the user's face image data captured by the terminal device into partial face image data, and then encrypts the data and transmits it to one of a plurality of authentication servers. Each authentication server decrypts the encrypted partial face image data received from the control server and then performs a matching process with the registered face image data registered in advance in the face image DB, and transmits the matching result to the control server. Then, the control server performs an authentication result determination process based on the collation result received from each authentication server and notifies the terminal device of the determination result.

  Patent Document 3 discloses an invention relating to a monitoring system (an example of an image authentication system) that can properly monitor a monitoring area even when a large number of monitoring cameras are connected, and can prevent a reduction in security. In this monitoring system, a captured image of a monitoring area captured by a monitoring camera is stored in a moving image storage unit. If a person is captured in the captured image, the face image of the person is cut out, and the face image cut out as a still image is transmitted to the host device. Furthermore, in response to a request from the host device, the moving image stored in the moving image storage unit can be transmitted to the host device.

JP 2003-331264 A JP 2006-293712 A JP 2005-323046 A

  In the conventional image authentication system described above, when the original monitoring image is specified based on the input image data generated from the monitoring image, and the monitoring image is displayed on the user terminal or the like, the feature image corresponding to the input image data However, there are cases where it is not known (visually unidentifiable) depending on the display which image portion of the original monitoring image is. For example, when a face image of a theft regular crime as a monitoring target is detected in a monitoring image captured by the imaging terminal, the face image of the theft regular crime is immediately recognized even if the monitor views the monitoring image on his / her terminal. It is difficult. The reason is that even if the original monitoring image is specified and displayed based on the input image data generated from the monitoring image, the feature image corresponding to the input image data is the monitoring image corresponding to the input image data or This is because which image portion of the partial images cannot be easily identified visually.

  Therefore, it is desirable to provide an image authentication system, an image authentication method, and an image authentication program that enable a monitor to visually recognize a monitoring target immediately and easily.

  The present invention has been made in view of such a situation, and an object of the present invention is to provide an image authentication system, an image authentication method, and an image authentication system that allows a monitor to visually recognize a monitoring target immediately and easily. It is to provide an image authentication program.

In order to achieve the above object, the characteristic configuration of the image authentication system according to the present invention is as follows:
An imaging unit that captures a monitoring target area and obtains a monitoring image;
A monitoring image storage unit for storing the monitoring image;
A feature image of an imaging target included in the monitoring image is cut out as an input feature image, and the input feature image, image specifying information for specifying the monitoring image, and a cutting position of the input feature image in the monitoring image are specified. A feature image cutout unit that generates input image data including positional information
A collation unit that collates the input image data with a registered image database to be monitored stored in advance, and detects the input image data corresponding to the registered image database as detected image data;
When the detected image data is detected as a result of collation by the collation unit, at least a part including the input feature image of the monitoring image specified based on the detected image data, and a position of the input feature image And a display unit for displaying.

  According to the above configuration, a monitoring image of the monitoring target area is acquired by the imaging unit. Then, when the feature image is included in the monitoring image, the feature image cutting unit cuts out the feature image from the monitoring image. Further, when the feature image is cut out from the monitoring image, the feature image cutout unit generates input image data including the feature image and position information for specifying the position of the feature image in the monitoring image.

  When the input image data is generated, the collation unit can collate the input image data with a registered image database to be monitored stored in advance. At this time, since the input image data is a cut out feature image, collation with the registered image database is quickly executed. In addition, when the feature image cutout unit and the collation unit are provided in different terminals, the input image data is the cut out feature image, so the communication speed is extremely faster than when the entire monitoring image is transmitted and received. . If it is detected by the comparison that the input image data corresponds to the registered image database, the input image data can be specified (selected) as detected image data in which the monitoring target is imaged.

  When the detection image data is specified, the display unit specifies the monitoring image specified based on the detection image data, or at least a part of the monitoring image including the input feature image (hereinafter, the entire monitoring image). Or, it may be referred to as a monitoring image or the like including any of the partial images). In other words, the display unit displays a monitoring image including a monitoring target image. When performing the display, the display unit displays the position of the input feature image. Thereby, the person who browsed the display unit (for example, a monitor who monitors the monitoring target area using the image authentication system, for example, shoplifting G men at a retail store) can display the displayed image (monitoring image or the like). In the displayed image (hereinafter referred to as a display image), it is possible to easily identify visually which image portion of the display image the feature image corresponding to the input image data is. Therefore, the monitor who has viewed the display image can immediately recognize the monitoring target. In addition, the monitoring burden on the monitoring target of the supervisor can be reduced. This enables early initial response to the monitoring purpose of the monitoring target (for example, ensuring security such as crime prevention and accident prevention).

A further characteristic configuration of the image authentication system according to the present invention is as follows.
The monitoring image is a moving image.

  According to the above configuration, in the monitoring moving image specified based on the detected image data, at least a portion of the moving image portion including the input feature image or a part of the frame (frame image) including the input feature image in the moving image. The position of the input feature image is displayed. As a result, the supervisor can grasp the behavior to be monitored in time series.

A further characteristic configuration of the image authentication system according to the present invention is as follows.
The image specifying information is an image capturing time or an image frame number.

  According to the above configuration, the monitoring image including the input feature image is specified by the imaging time or the image frame number. As a result, the monitoring image can be identified quickly and without error.

A further characteristic configuration of the image authentication system according to the present invention is as follows.
The position information is at a point that is coordinate information in the monitoring image.

  According to the above configuration, the detected image data includes the coordinate information of the input feature image in the monitoring image. Thereby, the display unit can specify the position of the input feature image in the monitoring image based on the coordinate information. For example, it is possible to display the position of the monitoring target in the display screen by surrounding it with a frame, and the monitor who sees the display screen can immediately recognize the monitoring target.

A further characteristic configuration of the image authentication system according to the present invention is as follows.
The input feature image includes a face image;
The position information is a point that is coordinate information of an eye part in the face image.

  According to the above configuration, the face image is included in the input feature image, and the coordinate information of the eye portion of the face included in the face image in the monitoring image is handled as coordinate information representing the position of the face image in the monitoring image. Thereby, the display unit can specify the position of the face image from the coordinate information of the eye part and display the specified position. Therefore, the monitor who has viewed the display image can immediately recognize the monitoring target.

A further characteristic configuration of the image authentication system according to the present invention is as follows.
The collation unit generates authentication result information that is information on a degree of coincidence between the imaging target included in the input feature image and the monitoring target included in the registered feature image,
The said display part exists in the point which acquires and displays the said authentication result information.

  According to the above configuration, the display unit displays the authentication result information generated by the verification unit in the display image. Thereby, the monitor can grasp | ascertain how much the input characteristic image displayed as a display image is likely to be a monitoring object as information of a coincidence degree. As a result, the necessary initial response can be made according to the monitoring purpose of the monitoring target.

  For example, when a theft regular offender is to be monitored, if the degree of coincidence between the theft regular offender displayed on the display unit and the face of the authenticated person (input feature image) is sufficiently high, the supervisor will actively authenticate as the theft regular offender. It is possible to act as a criminal deterrence by calling out to the person who was made. On the other hand, if the degree of coincidence is not so high, the observer detected the theft criminals by carefully monitoring the behavior of persons who were certified as theft criminals, taking into account the possibility of misperception (misplacement). This is preferable because the initial movement can be appropriately selected.

The characteristic configuration of the image authentication method according to the present invention for achieving the above object is as follows:
An imaging step of imaging a monitoring target area to obtain a monitoring image;
A monitoring image storing step for storing the monitoring image;
A feature image of an imaging target included in the monitoring image is cut out as an input feature image, and the input feature image, image specifying information for specifying the monitoring image, and a cutting position of the input feature image in the monitoring image are specified. A feature image cut-out step for generating input image data including position information,
A collation step of collating the input image data with a registered image database to be monitored stored in advance, and detecting the input image data corresponding to the registered image database as detected image data;
When the detected image data is detected as a result of the collation in the collating step, at least a part including the input feature image of the monitoring image specified based on the detected image data, and a position of the input feature image And a display step for displaying.

  According to the above configuration, it is possible to obtain the operational effects related to the above-described image authentication system.

In order to achieve the above object, the characteristic configuration of the image authentication program according to the present invention is as follows:
An image authentication program for causing a computer to execute information processing between an imaging terminal storing a monitoring image acquired by imaging a monitoring target area and a display terminal for displaying the monitoring image,
The feature image of the imaging target included in the monitoring image of the imaging terminal is cut out as an input feature image, the input feature image, image specifying information for specifying the monitoring image, and the input feature image in the monitoring image A feature image cutting process for generating input image data including position information for specifying a cutting position;
Collating the input image data with a pre-stored registered image database to be monitored, and collating processing for detecting the input image data corresponding to the registered image database as detected image data;
When the detected image data is detected as a result of collation by the collation processing, at least a part including the input feature image of the monitoring image specified based on the detected image data, and the position of the input feature image Display processing on the display terminal.

  According to the above configuration, it is possible to obtain the operational effects related to the above-described image authentication system.

  According to the image authentication system, the image authentication method, and the image authentication program described above, the monitor can visually recognize the monitoring target immediately and easily.

Illustration of the configuration of the image authentication system Illustration of operation of image authentication system Explanatory drawing of display mode of display terminal Explanatory drawing of display mode of display terminal Flow chart explaining operation of imaging terminal Flow chart explaining operation of authentication server Flowchart explaining operation of display terminal Flowchart explaining operation of display terminal Flow chart explaining operation of authentication server Flowchart explaining operation of display terminal Explanatory drawing of the highlight range on the display terminal Explanatory drawing of the highlight range on the display terminal Data structure of input image data and detected image data Illustration of verification and authentication method Illustration of verification and authentication method Illustration of verification and authentication method Explanatory drawing of operation | movement of the image authentication system which concerns on another embodiment Flowchart explaining operation of authentication unit Flow chart explaining operation of imaging terminal

  Based on FIGS. 1 to 19, an image authentication system according to an embodiment of the present invention, an image authentication method for realizing the system, and an image authentication program will be described.

[First embodiment]
[Overview]
As illustrated in FIG. 2, the image authentication system 100 according to the present embodiment includes a camera 24 (an example of an imaging unit) that captures a monitoring target area R and acquires a monitoring image P, and an imaging target included in the monitoring image P. The server 1 is an image authentication server that authenticates (for example, a visitor H who is a person entering the store), and a display terminal 3 that displays an authentication result and the like authenticated by the server 1. The server 1 includes a registration information storage unit 51 in which registration information F (an example of registration authentication information, see FIGS. 14 to 16), which is information on a monitoring target (for example, theft regular crime T), is stored as a registration image database. ing.

  The image authentication system 100 can target an object such as a person or a car as an imaging target. In the following, the image authentication system 100 detects a theft regular crime T from a visitor H who is a person who enters a store, a building, etc., and notifies the store supervisor A (see FIGS. 3 and 4) to perform a warning action. The case of an intelligent security system that prompts and prevents theft by the theft regular offender T will be described as an example. Further, in the following, a case where the imaging target is the visitor H and the monitoring target is the theft regular crime T will be described as an example.

  In the image authentication system 100, the server 1, the imaging terminal 2, and the display terminal 3 cooperate to capture an image of the monitoring target area R and acquire the monitoring image P, and a monitoring image storage step of storing the monitoring image P The feature image of the visitor H included in the monitoring image P is cut out as the input feature image 43, the input feature image 43, the image specifying information for specifying the monitoring image P, and the input feature image 43 in the monitoring image P are cut out. A feature image cut-out step for generating input image data 43d including position information for specifying a position; a registered image database of the theft regular crime T stored in advance with the input image data 43d (see the registration information storage unit 51, see FIG. 1); ) To detect the input image data 43d corresponding to the registered image database as detected image data X; When the detected image data X is detected as a result of the collation, at least a part including the input feature image 43 of the monitoring image P specified based on the detected image data X and the position of the input feature image 43 are displayed. And a display step.

  The image authentication system 100 performs information processing between the imaging terminal 2 that stores the monitoring image P acquired by imaging the monitoring target area R and the display terminal 3 for displaying the monitoring image P. This is realized by executing an image authentication program to be executed by a computer.

  This image authentication program cuts out the feature image of the imaging target included in the monitoring image P of the imaging terminal 2 as the input feature image 43, the input feature image 43, image specifying information for specifying the monitoring image P, and the monitoring image The feature image cut-out process for generating the input image data 43d including the position information for specifying the cut-out position of the input feature image 43 in P, and the input image data 43d and the registered image database to be monitored stored in advance are collated When the detected image data X is detected as a result of the collation processing for detecting the input image data 43d corresponding to the registered image database as the detected image data X and the collation by the collation processing, based on the detected image data X At least a part including the input feature image 43 of the specified monitoring image P and the position of the input feature image 43 are displayed on the display terminal 3. And it includes a processing shows, the.

[Detailed explanation]
[Explanation of terms and definitions]
Stores in this embodiment include small retail stores such as supermarkets and convenience stores, general retail stores such as department stores and department stores, and commercial areas such as shopping malls and shopping centers. The building includes public facilities such as company facilities, city halls, elementary schools, and kindergartens.

  As shown in FIG. 2, the visitor H is a person who enters the store and is authenticated by the image authentication system 100. The visitors H are candidates for monitoring by the image authentication system 100 or the monitoring person A (see FIGS. 3 and 4). Visitor H includes a theft regular crime T.

  Theft regular crime T is a person registered in the registration information storage unit 51 (see FIG. 1) of the server 1. Further, the theft regular offender T is a person who is a monitoring target of the image authentication system 100 or the monitor A.

  The face data mentioned below is image information (that is, informationized) of a face image or a feature amount thereof, and is included in the concept of the input feature image 43. The face image is an image obtained by photographing a person's face with a camera or the like. The feature amount is an amount obtained by quantifying a human face feature in a face image. Examples of feature quantities include, for example, the shape of eyes, nose and mouth, the distance between eyes, the distance between eyes and nose, contour information, the presence or absence of glasses, the hairstyle, and gender information that can be determined from these values. It has become. Hereinafter, the image information of the face image and / or the feature amount thereof is simply referred to as face data.

[Description of system configuration]
As described above, the image authentication system 100 includes the server 1, the imaging terminal 2, and the display terminal 3, as shown in FIGS. The server 1, the imaging terminal 2, and the display terminal 3 are connected so as to be capable of bidirectional communication via a network N (for example, a local area network or the Internet) of an electric communication line so that they can operate in an integrated manner. It has become. The server 1 is connected to a plurality of imaging terminals 2 via the network N. The server 1 is connected to a plurality of display terminals 3 via the network N. The display terminal 3 is connected to a plurality of imaging terminals 2 via the network N. The image authentication system 100 further includes a notification device 9.

  In the image authentication system 100, a plurality of imaging terminals 2, a plurality of display terminals 3, and one server 1 form one domain, and the server 1, the imaging terminal 2, the display terminal 3, and the like in the domain Can communicate with each other. In the following, a case where one imaging terminal 2, one display terminal 3, and one server 1 communicate will be described as an example, but the imaging terminal 2 and the display terminal 3 are not limited to one.

  As the image authentication system 100, a case where a plurality of imaging terminals 2, a plurality of display terminals 3, and a notification device 9 are connected to an intranet as a network N will be described as an example. Further, the server 1 will be described as an example in which the server 1 is connected to the Internet as the network N. Further, the case where the intranet to which the display terminal 3 or the like is connected is described as an example in which it is connected to the Internet so as to be capable of bidirectional communication. In addition, the intranet communication is described by taking as an example a case where large-capacity communication is possible more easily than communication via the Internet and the communication cost is low.

[Description of imaging terminal]
The imaging terminal 2 is a device that captures and records still images and moving images that include a camera 24 and the like. As shown in FIG. 2, the imaging terminal 2 is installed at a position where the monitoring target area R is imaged. The imaging terminal 2 is installed so as to capture the vicinity of the entrance of the store entrance door set as the monitoring target area R. Thereby, the imaging terminal 2 images the face etc. of the visitor H who enters the store from the entrance door. The imaging terminal 2 is connected to the same intranet as the intranet (part of the network N) to which the display terminal 3 is connected. As described above, this intranet is connected to the Internet (a part of the network N) so as to be capable of bidirectional communication.

  As illustrated in FIG. 1, the imaging terminal 2 includes an imaging-side central control unit 20, a camera 24 that captures the monitoring target area R and acquires the monitoring image P, and an imaging-side storage unit 26 ( An example of a monitoring image storage unit) and an imaging-side communication unit 29 that communicates with the network N.

  As shown in FIGS. 1 and 2, the imaging terminal 2 connects a camera 24 to an image input interface of a hard disk recorder (imaging side storage unit 26) having a network function, and further connects the hard disk recorder to the imaging side communication. A series of systems connected to the network N via the unit 29.

  The imaging side communication unit 29 is a communication interface that connects the imaging terminal 2 to the network N. The imaging-side communication unit 29 may perform communication protocol control along with a role as a physical or electrical connection interface with the network N. The functions of the imaging-side communication unit 29 can include communication control functions such as routers and gateways and protocol conversion functions as well as interface functions with the network N such as so-called LAN cards.

  The camera 24 is an imaging unit having a CCD sensor or a CMOS sensor. The camera 24 captures the face of the visitor H passing through the monitoring target area R of the imaging terminal 2 and acquires the monitoring image P. The camera 24 continuously captures the monitoring image P as a still image of the monitoring target area R at regular intervals. The camera 24 stores the moving image data in which the monitoring images P are arranged in time series in the imaging side storage unit 26. That is, the monitoring image P is one frame of a moving image captured by the camera 24.

  Hereinafter, one frame (still image at a certain point in time) of the moving image captured by the camera 24 will be described as the monitoring image P. In the image authentication system 100, processes and operations performed on the monitoring image P described below are applied in time series to the monitoring images P arranged in time series. The camera 24 of the image authentication system 100 is not limited to capturing moving images.

  The imaging-side central control unit 20 is a functional unit that controls the operation of the imaging terminal 2. The imaging-side central control unit 20 includes, for example, a software program for realizing various processes, a CPU that executes the software program, and various hardware that is controlled by the CPU.

  The imaging-side central control unit 20 outputs operation instructions to the camera 24, the imaging-side storage unit 26, the imaging-side communication unit 29, and the like, and receives information from the camera 24, the imaging-side storage unit 26, the imaging-side communication unit 29, and the like. get. When the imaging-side central control unit 20 acquires and accepts the monitoring image P or the moving image transmission request including the monitoring image P via the imaging-side communication unit 29, the monitoring image P or the monitoring image P corresponding to the transmission request and The images before and after the monitoring image P are read from the imaging side storage unit 26, and the monitoring image P is transmitted via the imaging side communication unit 29. In the following, as an example of the case where the monitoring image P and a moving image of the predetermined time before and after the monitoring image P (hereinafter referred to as the monitoring moving image 45) are transmitted as the monitoring moving image data 45d as the monitoring image P and the images before and after the monitoring image P. To explain.

  The imaging-side central control unit 20 receives a request from the server 1 to transmit the monitoring image P or the monitoring moving image 45 to the display terminal 3, for example. The imaging-side central control unit 20 may receive a request for transmitting the monitoring image P to the display terminal 3 from the display terminal 3. Hereinafter, a case where the imaging-side central control unit 20 receives a request for transmitting the monitoring video 45 from the display terminal 3 will be described. Hereinafter, the request for transmitting the monitoring image P or the monitoring moving image 45 may be referred to as a request for transmitting the monitoring image. The monitoring image P and the monitoring moving image 45 may be collectively referred to simply as a monitoring image.

  The imaging side central control unit 20 includes an imaging side control unit 21 and a cutout unit 22 (an example of a feature image cutout unit).

  The imaging side control unit 21 is a functional unit that detects a face image of the visitor H from the monitoring image P. The imaging-side control unit 21 detects the face image of the visitor H in the monitoring image P based on the structural features of the human face (for example, the distance between both eyes, the distance between the eyes, the nose, and the mouth). The imaging side control unit 21 is realized by executing a program for detecting a face in the imaging side central control unit 20.

  The cutout unit 22 cuts out the face image of the visitor H included in the monitoring image P. When the imaging side control unit 21 detects the face of the visitor H, the cutout unit 22 cuts out an image range including the face of the visitor H from the monitoring image P, and sets the input feature image 43 as an image of the input feature image 43. Input image data 43d including information is generated.

  The cutout unit 22 is realized by executing a program for cutting out an image range including a face in the imaging-side central control unit 20. In the program, the feature image of the imaging target included in the monitoring image P of the imaging terminal 2 is cut out as the input feature image 43, the input feature image 43, image specifying information for specifying the monitoring image P, and the monitoring image P And a feature image cutout process for generating input image data 43d including position information for specifying a cutout position of the input feature image 43.

  Note that the cutout unit 22 cuts out the face image of the visitor H, so that the information amount of the input image data 43d (a so-called amount in units of bytes or the like) can be reduced. Thereby, communication via the Internet with the server 1 to be described later can be realized at high speed and low cost.

  The cutout unit 22 includes information (referred to as image specifying information) for specifying the monitoring image P in the input image data 43d. In addition, the cutout unit 22 includes, in the input image data 43d, position information that specifies which image range the input feature image 43 is in the monitoring image P.

  Further, since the input image data 43d includes position information specifying which image range in the monitoring image P, the input feature image 43 is also included in the detected image data X that is the input image data 43d corresponding to the registered image database. However, position information specifying which image range in the monitoring image P is included is included. Therefore, based on the position information included in the detected image data X, it is possible to identify which image range in the monitoring image P the input feature image 43 is.

  Examples of the image specifying information included in the input image data 43d by the cutout unit 22 include information on the time (imaging time) when the monitoring image P was captured. The time information is, for example, date and time information. Further, the information included in the image specifying information may be an elapsed time since the start of moving image capturing including the monitoring image P. In addition, since the moving image is a sequence of still images with time, it may be a sequential number (so-called frame number) from the top image. When there are a plurality of imaging terminals 2, information for identifying the imaging terminal 2 (for example, information relating to an ID number unique to the imaging terminal 2 or an installation location of the camera 24) may be included. Hereinafter, a case will be described as an example in which the image specifying information includes information related to the imaging time and the installation location of the camera 24.

  As the position information included in the input image data 43d by the cutting unit 22, for example, coordinate information on the pixels of the monitoring image P can be included. The position information included in the input image data 43d by the cutout unit 22 includes the coordinates of both eyes E in the face image of the visitor H as shown in FIG. 11 and the contour of the face K as shown in FIG. For example, coordinate information of four corners of a rectangle including the range to be performed (for example, coordinates from the position p1 to the position p4), and the like. A case where the coordinates of both eyes E in the face image are included as coordinate information is described.

  The dimension of a person's face is usually determined at a predetermined ratio. Therefore, as shown in FIG. 11, if the eye position (coordinates) in the image is specified, the range of the face image in the image can be estimated. Compared with the case where the position of the face image (the range of the face image) is expressed by the coordinate information (for example, p1 to p4 shown in FIG. 12) of the four corners of the rectangle in order to specify the position inside the rectangle, for example, When using information, it is sufficient to specify the coordinates of two points, so the amount of information can be reduced.

  When the cutout unit 22 generates the input image data 43d, the cutout unit 22 issues a transmission command to the imaging side communication unit 29, and causes the server 1 to transmit the input image data 43d. Since the input image data 43d has a reduced amount of information as described above, transmission to the server 1 can be realized at high speed and low cost.

  When transmitting the input image data 43d to the server 1, the imaging-side communication unit 29 transmits the input image data 43d as one information group that cannot be separated. For example, the input image data 43d is assumed to be image data in JPEG (Joint Photographic Experts Group) format. As one of the metadata of the image data, image specifying information for specifying the monitoring image P, which of the monitoring images P Include location information that identifies the image range. As a result, the information included in the input image data 43d can be managed as a single unit to prevent information gaps and information loss. As a result, the information security of the image authentication system 100 is also increased, which is beneficial.

  For example, as shown in FIG. 13, the input image data 43d is a group of information packages (format) that starts with the header information of the information section i01 and ends with the footer information of the information section in (where n is an integer of 3 or more). Can be included and sent.

  In the example illustrated in FIG. 13, the image information of the input feature image 43 is included in the information section in−1. The information classification in-2 includes image specifying information. The information section in-2 includes position information. Note that the image specifying information and the position information may be included in different information categories.

  The operation of the imaging terminal 2 will be described with reference to the flowchart illustrated in FIG. Hereinafter, each step is described as # 123 or the like. The number immediately after the # symbol corresponds to the serial number of each step. For example, when describing as # 123, it indicates the 123rd step (step # 123).

  # 101 is a step (an example of an imaging step) in which the camera 24 starts capturing the monitoring image P. Also, # 102 is a step (an example of a monitoring image storage step) in which the monitoring image P is recorded in the imaging side storage unit 26. When imaging starts (# 101), the camera 24 proceeds to # 102, starts recording the monitoring image P as a moving image in the imaging-side storage unit 26, and proceeds to # 103.

  In step # 103, the imaging side control unit 21 waits until the face image of the visitor H is detected from the monitoring image P (No in # 103), and when the face image of the visitor H is detected from the monitoring image P (# 103). , Yes), a step of moving to # 104. After the recording of the monitoring image P is started in # 101, the imaging-side control unit 21 detects the face image of the visitor H from the monitoring image P while continuing the recording (# 103, Yes), and proceeds to # 104. Transition.

  # 104 is a step of cutting out the input feature image 43 (an example of a feature image cutting out step). In # 104, the cutout unit 22 cuts out the face image of the visitor H included in the monitoring image P, and generates the input feature image 43. When the input feature image 43 is generated in # 104, the cutout unit 22 generates input image data 43d in # 105. When the input image data 43d is generated in # 105, the clipping unit 22 transmits the input image data 43d to the server 1 (# 106). The transmission is performed by the imaging-side communication unit 29 based on a transmission instruction from the cutout unit 22. After the input image data 43d is transmitted by the imaging side communication unit 29, the process proceeds to # 107.

  # 107 waits until the imaging terminal 2 receives a request for transmitting a monitoring image from the server 1 or the display terminal 3 to the display terminal 3 (# 107, No), and receives the request (# 107, Yes). ), A step of moving to # 108.

  In step # 108, the imaging terminal 2 transmits a monitoring image to the display terminal 3. The case where the monitoring image is the monitoring moving image 45 including a range of a predetermined time before and after the monitoring image P (for example, 30 seconds before and after) is described as an example.

  In # 108, the imaging-side central control unit 20 reads the monitoring movie 45 from the imaging-side storage unit 26, and transmits the monitoring movie 45 to the display terminal 3 via the imaging-side communication unit 29. Thereafter, the process proceeds to # 109 and the series of operations is terminated. When the process proceeds to # 109 and the series of operations is completed, the process returns to # 101. In addition, although the case where the monitoring image is the monitoring moving image 45 is illustrated, the same applies to the case where the monitoring image is the monitoring image P. Moreover, the monitoring image P and the monitoring moving image 45 may be transmitted as the monitoring image.

[Description of server]
[Explanation of server configuration and operation]
The server 1 provides a function of authenticating the visitor H as a theft regular crime T. Note that the concept of authenticating that the visitor H is the theft regular crime T described below includes determining that the visitor H is the theft regular crime T as described later. In addition, the concept of authentication includes determining that the probability that the visitor H is a theft regular crime T is high. Furthermore, the concept of authentication includes determining that the presumption that the visitor H is the theft habitual offender T is not denied.

  As shown in FIGS. 1 and 2, the server 1 includes a server-side central control unit 10, a server-side storage unit 50 that stores information handled by the server 1, and a server-side communication unit 19 that communicates with the network N. ing. The server 1 is connected to the imaging terminal 2 and the display terminal 3 via the Internet as the network N.

  The server side communication unit 19 is a communication interface for communicating with the network N outside the server 1 according to an operation command from the server side central control unit 10. The server side communication unit 19 communicates with the imaging terminal 2 and the display terminal 3 via the network N.

  The server-side storage unit 50 includes a registration information storage unit 51, a collation information storage unit 52, and a temporary storage unit 53. The server-side storage unit 50 includes a storage device such as an HDD (Hard Disk Drive) or a nonvolatile RAM (Random Access Memory). The server-side storage unit 50 may be a physically separate storage device or may be two storage areas provided in one storage device. The server-side storage unit 50 may be provided in a place (for example, a server room) where the image authentication system 100 is provided, or may be a storage area on a so-called cloud.

  The registration information storage unit 51 stores registration information F as shown in FIGS. The registration information F includes a face image of the theft regular offender T as a registration image 44 (see FIG. 2). The registration information F including the face data of the person H1 or the like who is the theft regular crime T may be referred to as registration information F1 or the like with the same number at the end. In addition, a plurality of pieces of registration information F may be stored for the same person (for example, person H1), and the registration information F is referred to as registration information F11, F12, F13, etc. with a serial number added to the end. There is. In this case, the number immediately after the alphabet corresponds to the number such as the person H1, and the next number represents the serial number in the plurality of face data such as the person H1.

  Returning to FIGS. 1 and 2, the collation information storage unit 52 stores a program (hereinafter referred to as a collation program) that collates the input image data 43 d and the registration information F to calculate the degree of coincidence. The degree of coincidence is calculated in the range of 0 to 100. The degree of coincidence is 100 when there is a complete coincidence. The degree of coincidence is 0 when there is a complete disagreement.

  Information temporarily acquired from the network N by the server 1 is temporarily stored in the temporary storage unit 53. The server side communication unit 19 stores the input image data 43d received from the imaging terminal 2 via the network N and the result of collation by the collation unit 12.

  The server-side central control unit 10 is a functional unit that controls the overall operation of the server 1. The server-side central control unit 10 outputs an operation instruction to the server-side storage unit 50, the server-side communication unit 19, and the like, and acquires information from the server-side storage unit 50 and the server-side communication unit 19.

  The server-side central control unit 10 compares the central control unit 11, the collation unit 12 that collates the input image data 43d and the registration information F to detect the detected image data X, and the collation unit 12 collates the results. The authentication part 13 which authenticates that the visitor H is the theft regular crime T, and the process part 14 which adds the information which concerns on the result of the authentication which the authentication part 13 performed to the detected image data X are provided. Here, the detected image data X refers to the input image data 43d in which it is detected that the matching unit 12 matches the registered information F among the plurality of input image data 43d.

  FIG. 2 illustrates a case where a plurality of input image data 43 d corresponding to each of a plurality of visitors H detected from the monitoring image P is transmitted to the server 1. Then, the case where the input image data 43d that matches the registration information F among the plurality of input image data 43d transmitted to the server 1 is transmitted as the detected image data X from the server 1 to the display terminal 3 is illustrated. Yes.

  The server-side central control unit 10 includes, for example, a software program for realizing various processes, a CPU that executes the software program, and various hardware that is controlled by the CPU. Software programs and data necessary for the operation of the server-side central control unit 10 are stored in the server-side storage unit 50, but the storage destination is not particularly limited, and is stored in a dedicated storage device such as a disk or flash memory. The mode may be stored, or may be stored in a server or a storage device connected so as to be communicable.

  The central control unit 11 is a functional unit that controls transmission / reception of information received by the server 1 and information transmitted by the server 1. When the server-side communication unit 19 receives the input image data 43d from the network N, the central control unit 11 stores the input image data 43d in the temporary storage unit 53.

  When the detected image data X is stored in the temporary storage unit 53, the central control unit 11 sends an instruction to the server side communication unit 19 to transmit the detected image data X to the display terminal 3. At this time, the central control unit 11 can instruct the notification device 9 to notify. In response to the instruction, the notification device 9 notifies the monitor A (see FIGS. 3 and 4) that the display terminal 3 has received the detected image data X. Thus, the supervisor A is prompted to confirm the display terminal 3 and can immediately notice the reception of the detected image data X by the display terminal 3. Since the detected image data X has a reduced amount of information in the same manner as the input image data 43d, transmission of the detected image data X from the server 1 to the display terminal 3 is realized at high speed and at low cost.

  The collation unit 12 collates the input image data 43d with the registration information F (see FIGS. 14 to 16), and the degree of coincidence between the attendee H and the person H1 registered as the theft regular crime T in the registration information F Is calculated.

  The collation unit 12 calls the collation program stored in the collation information storage unit 52 and executes the collation program. The collation unit 12 collates the input image data 43d and the registration information F by executing the collation program. The collation program includes collation processing in which the input image data 43d is collated with a registered image database to be monitored stored in advance, and the input image data 43d corresponding to the registered image database is detected as detected image data X.

  When collating the input image data 43d and the registration information F, the collation unit 12 can collate based on the face data included in the input image data 43d, in particular, the feature amount. It is also possible to newly calculate the feature amount of the face image based on the input image data 43d and collate it.

  The collation unit 12 detects the input image data 43d as detected image data X based on the degree of coincidence obtained by collation. The collation unit 12 determines that the monitoring target is detected when the degree of coincidence obtained by collation by the collation unit 12 is equal to or greater than a predetermined threshold (for example, the first determination value V1, 60), and determines the input image data 43d. Detected as detected image data X. The matching unit 12 does not detect the input image data 43d as the detected image data X when the degree of coincidence is less than a predetermined threshold (for example, the first determination value V1). Since the image specifying information is included in the input image data 43d, the detected image data X, which is the input image data 43d corresponding to the registered image database, also includes the image specifying information.

  When the verification unit 12 detects the detected image data X, the authentication unit 13 authenticates the visitor H corresponding to the detected image data X as the theft regular crime T under a predetermined condition as will be described later. At the time of authentication, the authentication unit 13 refers to the information on the degree of coincidence obtained by the verification unit 12 and outputs information indicating that authentication has been performed and information to be described later as information related to the authentication result, and temporarily outputs these as authentication results. Store in the storage unit 53. Hereinafter, the authentication score and the information indicating the authentication are included and simply referred to as an authentication result.

  When the authentication unit 13 authenticates the visitor H corresponding to the detected image data X as the theft regular crime T, that is, when the authentication result is stored in the temporary storage unit 53, the processing unit 14 detects the detected image data X Are added to the detected input image data 43d, the degree of coincidence information obtained by the collation unit 12, the authentication result obtained by the authentication unit 13, and the image data of the registered image 44. Hereinafter, the degree of coincidence information obtained by the collation unit 12, the information related to the authentication result performed by the authentication unit 13, and the image data of the registered image 44 may be referred to simply as supplementary information.

  As shown in FIG. 13, the input image data 43d detected as the detected image data X starts from the header information of the information section i01 and ends with the footer information of the information section in (where n is an integer of 3 or more). It is handled as a group of information packages. The processing unit 14 records the incidental information in the information section in-2 of the detected image data X, updates the detected image data X, and stores it in the temporary storage unit 53. That is, the temporary storage unit 53 stores detected image data X (hereinafter, simply referred to as detected image data X) that is updated with the accompanying information.

[Description of authentication performed by the server]
The collation performed by the collation unit 12 will be described. The collation unit 12 collates one input image data 43d and a plurality of registration information F to obtain a degree of coincidence, and selects the registration information F having the highest degree of coincidence among the plurality of registration information F. At this time, if the largest matching degree is equal to or greater than the first determination value V1 (for example, 60), the input image data 43d is detected as the detected image data X.

  The authentication unit 13 authenticates the visitor H as a theft regular crime T when the degree of coincidence of the registration information F (the registration information F having the largest degree of coincidence) detected by the verification unit 12 is greater than a predetermined threshold. . Hereinafter, the threshold value of the degree of coincidence for authenticating as a specific theft regular crime T may be referred to as a second determination value V2. Hereinafter, the degree of coincidence when the registration information F having the largest degree of coincidence among the registration information F is selected may be simply referred to as the degree of coincidence.

  The second determination value V2 is set equal to the first determination value V1. The second determination value V2 may be set larger than the first determination value V1.

  Hereinafter, the execution of authentication in the server 1 will be described with a specific example. First, the registration information F of the theft regular offender T will be described.

  As shown in FIGS. 14 to 16, the registration information storage unit 51 stores registration information F of the theft regular offender T in advance. In this example, a plurality of pieces of registration information F are stored in the registration information storage unit 51 for each of the three persons H1, H2, and H3 who are theft criminals T.

  Specifically, four pieces of registration information F11, F12, F13, and F14 are stored for the person H1. Two pieces of registration information F21 and F22 are stored for the person H2. Three pieces of registration information F31, F32, and F33 are stored for the person H3. In this way, a plurality of registration information F is associated with one theft regular offender T.

  In the example of FIG. 14, the degree of coincidence calculated by the collation unit 12 is shown below the registration information F. For example, the degree of coincidence between the input image data 43d related to the person H1 and the registration information F21 to F24 related to the person H2 is calculated as 18, 21, 22, and 19.

  In the example of FIG. 14, the registration information F32 related to the person H3 has the highest matching score of 97, and therefore the input image data 43d corresponding to the registration information F32 is detected as the detected image data X. In the example of FIG. 15, the registration information F32 related to the person H3 has the highest matching score of 67, and therefore the input image data 43d corresponding to the registration information F32 is detected as the detected image data X. In the example of FIG. 16, the registration information F32 related to the person H3 has the highest matching score of 27, and thus the input image data 43d corresponding to the registration information F32 is detected as the detected image data X.

  The operation of the server 1 will be described with reference to the flowchart shown in FIG.

  When the server 1 is in an authenticable state (# 111), the central control unit 11 waits until the server side communication unit 19 receives the input image data 43d from the imaging terminal 2 (# 112, No). When the server-side communication unit 19 receives the input image data 43d from the imaging terminal 2 (# 112, Yes), the central control unit 11 stores the input image data 43d in the temporary storage unit 53 and proceeds to # 113. .

  # 113 is a step of collation (an example of a collation step). In # 113, the collation unit 12 collates the input image data 43d with the registration information F. The collation unit 12 reads the input image data 43 d from the temporary storage unit 53. The collation unit 12 reads the registration information F from the registration information storage unit 51. The collation unit 12 collates the input image data 43d with the registration information F. If the degree of coincidence is less than the first determination value V1 (60) (# 114, No), the process proceeds to # 119 and ends. If the degree of coincidence is equal to or higher than the first determination value V1 (60) (# 114, Yes), the collation unit 12 determines that the monitoring target has been detected, and proceeds to # 115. At this time, the collation unit 12 detects the input image data 43d as detected image data X.

  In step # 115, authentication is performed. In # 115, the authentication unit 13 authenticates the visitor H corresponding to the input image data 43d detected as the detected image data X as the theft regular crime T, and proceeds to # 116.

  In the present embodiment, since the second determination value V2 is equal to the first determination value V1, all visitors H corresponding to the input image data 43d detected as the detected image data X are authenticated as the theft regular crime T.

  In step # 115, the authentication unit 13 generates an authentication score upon authentication. The authentication score classifies the authentication probability into a predetermined level (for example, “high” and “low”, etc.), displays the authentication probability in a numerical value (for example, in the range of 0 to 100, 100 is the highest probability of authentication). When the authentication unit 13 authenticates, the authentication result is stored in the temporary storage unit 53. The operation in which the authentication unit 13 generates the authentication score will be described later.

  # 116 is a step of adding incidental information to the detected image data X. In # 116, the processing unit 14 adds the degree-of-match information obtained by the matching unit 12 and the authentication result performed by the authentication unit 13 to the input image data 43d detected as the detected image data X, and newly The detected image data X is stored in the temporary storage unit 53. Then, the process proceeds to # 117.

  # 117 is a step of transmitting the detected image data X to the display terminal 3. In # 117, the central control unit 11 reads the detected image data X from the temporary storage unit 53. Then, the central control unit 11 transmits the detected image data X to the display terminal 3 via the server side communication unit 19. Then, the process proceeds to # 118.

  # 118 is a step of transmitting a notification instruction to the notification device 9. In # 118, the central control unit 11 transmits to the notification device 9 an instruction to notify through the server side communication unit 19. Then, the process proceeds to # 119 and the series of operations is terminated. When the process proceeds to # 119 and the series of operations is completed, the process returns to # 111.

[Description of the operation of the authentication unit to generate the authentication score]
An operation in which the authentication unit 13 generates an authentication score will be described with reference to flowcharts shown in FIGS.

  In the authentication in # 115 shown in FIG. 6, the operation of the flowchart shown in FIG. 18 is performed.

  When the authentication is started in # 121, the authentication unit 13 determines whether or not the degree of coincidence is a third determination value V3 (for example, 80) or more (# 122). The third determination value V3 is set as a value larger than the second determination value V2. If the degree of coincidence is greater than or equal to the third determination value V3 (# 122, Yes), the process proceeds to # 123. If the degree of coincidence is less than the third determination value V3 (# 122, No), the process proceeds to # 124.

  # 123 and # 124 are steps for generating and outputting an authentication score.

  In # 123, “high” is output as the authentication score. In addition, this authentication score is displayed on the monitor 35 (an example of a display part) of the display terminal 3 so that it may mention later. Since the degree of coincidence is equal to or higher than the third determination value V3, by outputting “high” as the authentication score and displaying it, the supervisor A (see FIG. 3 and FIG. 4) makes the attendee H This is to intuitively understand that it is determined to be. Thus, the supervisor A can act as a crime deterrent by speaking to the visitor H who is authenticated as the theft regular crime T with a certain confidence.

  In # 124, “low” is output as the authentication score. The authentication score is displayed on the monitor 35 of the display terminal 3 as will be described later. Since the degree of coincidence is greater than or equal to the second determination value V2 and less than the third determination value V3, by outputting “low” as the authentication score, the supervisor A is performed and the visitor H is the theft regular crime T This is to intuitively understand that it is determined that the probability is high. This makes it possible for the observer A to prepare for crime prevention by carefully monitoring the behavior of the visitor H who has been authenticated as a theft regular crime T while taking into account the possibility of misperception (incorrectness). .

  When the authentication score is output in # 123 or # 124, the process proceeds to # 125 to end the authentication. Thereafter, the process proceeds to # 116 shown in FIG.

[Description of display terminal]
[Description of configuration and operation of display terminal]
The display terminal 3 is a terminal where the monitor A browses the monitoring image P and the like.

  As shown in FIGS. 1 and 2, the display terminal 3 communicates with a display-side central control unit 30, a monitor 35, a drawing unit 34 that displays an image on the monitor 35, a terminal-side storage unit 36, and a network N. And a terminal-side communication unit 39. The display terminal 3 is a so-called tablet terminal. The display terminal 3 is connected to the same intranet as the intranet to which the imaging terminal 2 is connected.

  The terminal side communication unit 39 is a communication interface for communicating with the network N outside the display terminal 3 according to an operation command of the display side central control unit 30. The terminal side communication unit 39 communicates with the server 1 and the imaging terminal 2 via the network N. The terminal-side communication unit 39 receives, for example, the detected image data X and the like from the server 1 and the imaging terminal 2, and transmits a request for transmitting a monitoring image to the imaging terminal 2.

  The drawing unit 34 is a drawing device that outputs an image to be displayed on the monitor 35. This is a so-called GPU (Graphics Processing Unit).

  The monitor 35 is a display device having a touch input function. The monitor 35 is a touch panel type liquid crystal monitor. The monitor 35 displays an image output from the drawing unit 34. Information displayed on the monitor 35 will be described later. The monitor 35 accepts an operation command input by the monitor A through a touch operation, and transmits the operation command to the terminal-side control unit 31 described later.

  The terminal-side storage unit 36 includes a storage device such as an HDD (Hard Disk Drive) or a nonvolatile RAM (Random Access Memory). The terminal-side storage unit 36 may be a physically separate storage device or two storage areas provided in one storage device.

  The terminal-side storage unit 36 stores the detected image data X, the monitoring image P, and the monitoring video 45 received by the terminal-side communication unit 39 of the display terminal 3 from the network N. The display terminal 3 receives the detected image data X in the intranet via the Internet. On the other hand, the monitoring image P and the monitoring video 45 are received from the imaging terminal 2 connected to the same intranet as the intranet to which the display terminal 3 is connected. Thereby, the display terminal 3 can receive the monitoring image P and the monitoring moving image 45 at high speed using a high-speed communication network in the intranet. In addition, the amount of data communication of Internet communication that is external communication is reduced to facilitate cost reduction. Further, since the display terminal 3 can receive the monitoring image P and the monitoring moving image 45 without going through the server 1, the data communication amount of the Internet communication is reduced and the cost reduction is facilitated.

  The display-side central control unit 30 is a functional unit that controls the overall operation of the display terminal 3. The display-side central control unit 30 outputs operation instructions to the monitor 35, the drawing unit 34, the terminal-side storage unit 36, the terminal-side communication unit 39, and the like, and the monitor 35, the drawing unit 34, the terminal-side storage unit 36, and the like. Information is acquired from the terminal side communication unit 39.

  The display side central control unit 30 includes a terminal side control unit 31 and a position selection unit 32. The display-side central control unit 30 includes, for example, a software program for realizing various processes, a CPU that executes the software program, and various hardware that is controlled by the CPU. Software programs and data necessary for the operation of the server-side central control unit 10 are stored in the server-side storage unit 50, but the storage destination is not particularly limited, and is stored in a dedicated storage device such as a disk or flash memory. The mode may be stored, or may be stored in a server or a storage device connected so as to be communicable.

  The terminal-side control unit 31 controls the operation of the display terminal 3 in accordance with the operation command of the monitor A (see FIGS. 3 and 4) received by the monitor 35, and the information received by the display terminal 3 and the display terminal 3 is a functional unit that controls transmission / reception of information to be transmitted.

  When the terminal-side communication unit 39 receives the detected image data X from the network N, the terminal-side control unit 31 stores the detected image data X in the terminal-side storage unit 36. When the detected image data X is stored in the terminal-side storage unit 36, the terminal-side control unit 31 instructs the terminal-side communication unit 39 to request the imaging terminal 2 to transmit a monitoring image corresponding to the detected image data X. To do. At this time, the terminal side communication unit 39 receives the monitoring image specified based on the image specifying information included in the detected image data X.

  When the terminal-side communication unit 39 receives the monitoring image P and the monitoring moving image 45 from the network N, the terminal-side control unit 31 stores them in the terminal-side storage unit 36.

  When the monitor 35 displays at least a part including the input feature image 43 of the monitoring image P specified based on the detected image data X and the position of the input feature image 43, the position selection unit 32. It is a function part which specifies the display range of the position containing.

  The drawing unit 34 causes the monitor 35 to display the position display 5 based on the display range specified by the position selection unit 32. The position selection unit 32 realizes the display of the position display 5 by calling and executing the display program stored in the terminal-side storage unit 36. The display program includes at least a part of the input feature image 43 of the monitoring image P specified based on the detected image data X when the detected image data X is detected as a result of the collation by the collation processing of the collation unit 12. And a display process for displaying the position of the input feature image 43 on the display terminal 3. The position display 5 will be described later.

  Information displayed on the monitor 35 will be described. The monitor 35 displays an image output from the drawing unit 34. As shown in FIGS. 3 and 4, the monitor 35 displays a part of the monitoring image P including the monitoring image P or the input feature image 43, a position display 5 described later, and the like. Further, other information display 46 described later is performed.

  As the information display 46 displayed on the monitor 35, various information including a first information display 46a and a second information display 46b described later can be displayed. An example in which the first information display 46a and the second information display 46b are displayed as the information display 46 will be described below. The information display 46 is displayed based on image specifying information and position information included in the detected image data X. Specifically, as the first information display 46a, information specifying the time (imaging time) when the input feature image 43 was captured by the imaging terminal 2 and the camera 24 (see FIGS. 1 and 2) of the imaging terminal 2 is displayed. The identity probability and the authentication score are displayed as the second information display 46b.

  FIG. 3 illustrates a case where the first display image 41 is displayed on the monitor 35. The first display image 41 displays an input feature image 43, a registered image 44, an information display 46, and an image showing the input unit 49 as a display that allows the monitor A to recognize a portion that accepts an input operation. The input unit 49 is an input interface that receives an instruction to display the monitoring image P and the monitoring moving image 45 by the monitoring person A.

  The input unit 49 receives an instruction for the monitor A to play the monitoring video 45. When the monitoring person A inputs a command to reproduce the monitoring video 45 via the input unit 49, the central control unit 11 tries to reproduce the monitoring video 45. This operation will be described later. Note that the input unit 49 may receive a command for the monitor A to display the monitor image P.

  As the information display 46, a first information display 46a and a second information display 46b are displayed. As the first information display 46a, the time when the input feature image 43 is captured by the imaging terminal 2 (imaging time) and information for specifying the camera 24 of the imaging terminal 2 are displayed. Further, as the second information display 46b, an identity probability and an authentication score are displayed. The identity probability is equal to the coincidence value in this embodiment.

  FIG. 4 illustrates a case where the second display image 42 is displayed on the monitor 35. In the second display image 42, an input feature image 43, a registered image 44, a monitoring video 45 including the monitoring image P, a first information display 46a, and an image showing the input unit 49 are displayed. Note that the second information display 46 b may also be displayed on the second display image 42. The monitoring moving image 45 including the monitoring image P is not limited to the case where the entire region of the frame included in the monitoring moving image 45 is displayed, and the input feature image 43 and an image portion including at least the surrounding image region are trimmed. It may be displayed (cut out).

  The position selection unit 32 and the position display 5 will be described. The position display 5 displays at least a part including the input feature image 43 of the monitoring image P specified based on the detected image data X and the position of the input feature image 43.

  The detected image data X includes the coordinates of both eyes E (see FIG. 11) of the face image in the input feature image 43 as position information, and the position selection unit 32 includes the input feature image 43 based on the coordinates of both eyes E. Specify the display range of the position. The position selection unit 32 issues a drawing command for the position display 5 to the drawing unit 34 based on the specified display range. When the drawing unit 34 displays a part of the monitoring image P including the monitoring image P or the input feature image 43 on the monitor 35, for example, a display surrounding the input feature image 43 in a rectangle is displayed on the monitor 35 as a position display 5. Let it draw. The drawing unit 34 can overlay display the position display 5 on the monitoring image P or the monitoring moving image 45.

  As shown in FIGS. 4 and 11, the position display 5 is exemplified as a symbol that displays an area including the contour range of the face K of the visitor H. In the example shown in FIGS. 4 and 11, the position display 5 shows a case where the position display 5 is drawn in a rectangle including a face image. In addition, the position display 5 may use a symbol such as an arrow indicating the face of the visitor H (input feature image 43). The position display 5 may be displayed as a numerical value indicating coordinates that specify the contour range of the face K of the visitor H. Thus, the position display 5 is a display for emphasizing the face image (input feature image 43) of the visitor H.

  The display mode of the position display 5 is not limited to the mode described above. The position display 5 can also be displayed in the form of a balloon starting from the face image (input feature image 43, face image) of the visitor H. And it is good to enlarge and display the face of the visitor H in the balloon part. In this case, the position display 5 is a form of a balloon and an enlarged display of the face of the visitor H. When the position display 5 includes such an enlarged display of the face of the visitor H, even if the face of the visitor H cannot be visually recognized if the monitoring image P is displayed as it is, the face of the visitor H is enlarged and displayed. It can be displayed visually.

  The position display 5 illustrated in FIGS. 4 and 11 will be described. The position display 5 is displayed as a rectangular symbol whose longitudinal direction extends in the vertical direction. In other words, the position display 5 is a display serving as a mark for easily recognizing the face image (input feature image 43) of the visitor H visually. The position display 5 allows the supervisor A to easily recognize the face image (input feature image 43) of the visitor H by visual observation, and to take necessary actions immediately when the visitor H is the theft regular crime T. It is beneficial because it can.

  For example, the position selection unit 32 can determine the display of the long side and the short side of the position display 5 as follows. As shown in FIG. 11, when the distance between the left eye E1 and the right eye E2 is the inter-eye distance Ed, the position selection unit 32 can determine the left and right range of the face with reference to the inter-eye distance Ed. In the case of FIG. 11, the position shifted to the left by the distance d1 from the left eye E1 and the position shifted to the right by the distance d1 from the right eye E2 are defined as the left and right range of the face, and the length of the position display 5 It is displayed as a side. In the illustration of FIG. 11, the distance d1 is set to 1.3 times the inter-eye distance Ed. Note that the relationship between the distance d1 and the inter-eye distance Ed described in this embodiment is merely an example.

  Further, as shown in FIG. 11, when the distance between the left eye E1 and the right eye E2 is the inter-eye distance Ed, the position selection unit 32 can determine the upper and lower range of the face on the basis of the inter-eye distance Ed. it can. In the case of FIG. 11, a position displaced by a distance d2 from the height of both eyes E upward and a position displaced by a distance d3 from the height of both eyes E downward are the upper and lower ranges of the face. Is illustrated as an example. In the illustration of FIG. 11, the distance d2 is 1.5 times the eye distance Ed. Further, the distance d3 is set to 2.0 times the inter-eye distance Ed. The relationship between the distance d2 and the distance d3 described in this embodiment and the inter-eye distance Ed is merely an example.

  The operation of the display terminal 3 will be described with reference to the flowchart shown in FIG.

  When the display terminal 3 becomes ready to receive the detected image data X (# 131), the terminal side control unit 31 waits until the terminal side communication unit 39 receives the detected image data X from the server 1 (# 132). , No). When the terminal side communication unit 39 receives the detected image data X from the server 1 (# 132, Yes), the terminal side control unit 31 stores the detected image data X in the terminal side storage unit 36, and proceeds to # 133. To do.

  # 133 is a step of displaying the first display image 41 on the monitor 35. In # 133, the terminal side control unit 31 reads the detected image data X from the terminal side storage unit 36. Then, the terminal-side control unit 31 instructs the drawing unit 34 to draw the first display image 41 based on the detected image data X. The drawing unit 34 displays the first display image 41 on the monitor 35 based on the command (see FIG. 3). Thereafter, the process proceeds to # 134.

  # 134 is a step of waiting for the input unit 49 to input a command to reproduce the monitoring video 45 by the monitoring person A. In # 134, it waits until there is a command input by the supervisor A in the input unit 49 (No in # 134). If there is a command input from the supervisor A in the input unit 49 (# 134, Yes), the process proceeds to # 135.

  In step # 135, a request for transmitting a monitoring image is made. In # 135, the terminal-side control unit 31 requests the imaging terminal 2 to transmit a monitoring image via the terminal-side communication unit 39. As a request for transmitting the monitoring image, a request for transmitting the monitoring moving image 45 is made, but there may be a request for transmitting the monitoring image P. Thereafter, the process proceeds to # 136.

  In step # 136, the terminal-side communication unit 39 waits for reception of a monitoring image. In # 136, the process waits until the terminal-side communication unit 39 completes reception of the monitoring image (# 136, No). When the terminal side communication unit 39 completes reception of the monitoring image (# 136, Yes), the terminal side control unit 31 stores the monitoring image in the terminal side storage unit 36, and proceeds to # 137. Although the monitoring video 45 is stored in the terminal-side storage unit 36 as the monitoring image, the monitoring image P may be stored as the monitoring image.

  # 137 is a step of displaying the second display image 42 on the monitor 35 (an example of a display step). In # 137, the terminal-side control unit 31 reads the detected image data X and the monitoring moving image 45 from the terminal-side storage unit 36. Then, the terminal-side control unit 31 instructs the drawing unit 34 to draw the second display image 42 based on the detected image data X and the monitoring moving image 45. The drawing unit 34 displays the second display image 42 on the monitor 35 based on the command (see FIG. 4). At this time, the position selection unit 32 instructs the drawing unit 34 to perform overlay display of the position display 5. As a result, the position of the face image (input feature image 43) of the visitor H (theft regular offense T) is displayed on the monitoring video 45 in the second display image 42 with emphasis. Then, the series of operations is finished (# 138). When the series of operations is finished in # 138, the process returns to # 131 again.

[Description of Modification of First Embodiment]
The display terminal 3 may be operated as in the flowchart shown in FIG. 8, # 231 to # 233 and # 235 to # 238 are the same as # 131 to # 133 and # 135 to # 138 in FIG. The case of FIG. 8 is different in that it does not have a step corresponding to # 134 of FIG.

  In the case of FIG. 8, when the first display image 41 is displayed on the monitor 35 in # 233, a request for transmitting a monitoring image is made in # 235. That is, even if there is no input of a command for reproducing the monitoring moving image 45 by the monitoring person A, the monitoring image is acquired. In this case, a monitoring image such as the monitoring video 45 can be displayed on the monitor 35 with the shortest lead time after the terminal-side communication unit 39 receives the detected image data X from the server 1. This is particularly useful when the monitoring image is not the monitoring image P but the monitoring moving image 45 having a large information capacity.

[Second Embodiment]
Below, 2nd embodiment is described. The second embodiment differs from the first embodiment in part of the operations of the server 1 and the display terminal 3.

  The operation of the server 1 will be described with reference to the flowcharts shown in FIGS. In the case of FIG. 9, the operation of # 318 is different from the operation of # 118 of FIG.

  In the case of FIG. 9, the server 1 transmits the detected image data X to the display terminal 3 at # 317 and requests to transmit the monitoring image corresponding to the detected image data X to the display terminal 3 at # 318. Is transmitted to the imaging terminal 2. Thereby, the display terminal 3 can receive the monitoring image with the shortest lead time. In addition, the server 1 may instruct | indicate the alerting | reporting to the alerting device 9, while performing # 318.

  The operation of the display terminal 3 will be described with reference to the flowcharts shown in FIGS. In the case of FIG. 10, in FIG. 10, # 331 to # 333 and # 336 to # 338 are the same as # 131 to # 133 and # 136 to # 138 in FIG. The case of FIG. 10 is different in that it does not have a step corresponding to # 135 of FIG.

  Since the server 1 transmits a request for transmitting a monitoring image corresponding to the detected image data X to the display terminal 3 to the imaging terminal 2, the display terminal 3 performs a request for transmitting a monitoring image like # 135. Can be received (so-called push reception).

  As described above, in the image authentication system 100, when the server 1 authenticates (detects) that the visitor H is the theft regular crime T using the input feature image 43 captured by the imaging terminal 2, the display terminal 3 An image including the input feature image 43 can be displayed and notified to the supervisor A to call attention. As a result, the monitor A can speak to the visitor H who has been authenticated as the theft regular crime T and can carefully monitor its behavior. Therefore, the image authentication system 100 can realize prevention of crimes such as theft.

  As described above, in the present embodiment, when the original monitoring image is specified and displayed based on the input image data generated from the monitoring image, the feature image corresponding to the input image data is displayed as the input image. It is an object to provide an image authentication system, an image authentication method, and an image authentication program that can easily visually identify which image portion of a monitoring image corresponding to data or a partial image thereof.

[Another embodiment]
(1) In the embodiment described above, the image authentication system 100 detects the theft regular crime T from the visitor H who enters the store, notifies the monitor A of the store, prompts a warning action, and prevents theft by the theft regular crime T. The case of an intelligent security system to prevent was illustrated.

  However, the image authentication system 100 is not limited to the case of detecting the theft regular crime T from the visitors H. For example, as shown in FIG. 17, a system for detecting a vehicle C of a theft regular offender T among vehicles C entering a parking lot in a store or the like or parking in a parking space may be used. In this case, the image authentication system 100 can pick up the license plate G of the vehicle C instead of the face image of the visitor H and use it as the input feature image 43.

(2) In the above-described embodiment, the case where the imaging terminal 2 starts recording in the imaging-side storage unit 26 using the monitoring image P as a moving image in # 102 has been described. In the flowchart of FIG. 5, in # 101, the camera 24 starts capturing the monitoring image P, in # 102, starts recording the monitoring image P as a moving image in the imaging side storage unit 26, and further in # 103. When the face image of the visitor H is detected from the monitoring image P (# 103, Yes), the case has been described where the input feature image 43 is generated by moving to # 104, # 105.

  However, the imaging terminal 2 is not limited to recording the monitoring image P as a moving image. For example, as shown in the flowchart of FIG. 19, in step # 401, the camera 24 starts capturing the monitoring image P. However, the monitoring image P is not recorded as a moving image. Waiting for the detection of the H face image (# 402, No), and detecting the face image of the visitor H from the monitoring image P (# 402, Yes), the process proceeds to # 403, and the monitoring image P is displayed as a still image. May be recorded in the imaging-side storage unit 26. In this case, the storage capacity of the imaging side storage unit 26 can be saved. Note that # 404 to # 409 in FIG. 19 can be the same as # 104 to # 109 in FIG.

(3) In the above embodiment, the authentication unit 13 of the server 1 outputs “high” or “low” as the authentication score, and the display terminal 3 displays the identity probability as the second information display 46 b on the monitor 35. The case where the authentication score is displayed has been described.

  However, the authentication unit 13 may not output the authentication score. The authentication unit 13 is not limited to the case where the authentication score is divided into two levels of “high” or “low”, and may be divided into three or more levels and output. For example, when the degree of coincidence is 40 or more and less than 60, the authentication score is “low”, when the degree of coincidence is 60 or more and less than 80, the authentication score is “medium”, and when the degree of coincidence is 80 or more, the authentication score is “ It can also be “high”. As described above, when “low” is output as the authentication score when the degree of coincidence is 40 or more and less than 60, the degree that the observer A cannot deny that the visitor H is the theft regular crime T It becomes possible to have attention recognition.

(4) In the embodiment described above, the display terminal 3 has described the case where the first information display 46 a and the second information display 46 b are displayed on the monitor 35 as the information display 46. And as the 1st information display 46a, the time (imaging time) when the input characteristic image 43 was imaged with the imaging terminal 2, and the information which specifies the camera 24 of the imaging terminal 2 are displayed, and the 2nd information display 46b Exemplifies the case where the identity probability and the authentication score are displayed.

  However, the display terminal 3 may display information other than the above on the monitor 35. For example, you may display the individual feature-value used when the collation part 12 collated the input image data 43d and the registration information F. FIG. For example, the presence / absence of glasses, hairstyle, and sex information may be displayed. Thereby, since the supervisor A can visually easily grasp the characteristics of the theft regular offender T, it is useful for crime prevention.

  Further, the monitor 35 does not necessarily have to display all of the information included in the information display 46. In some cases, only a part of the information included in the information display 46 described in the present embodiment is displayed. For example, it is not necessary to display the identity probability or the authentication score included in the information display 46. Both the identity probability and the authentication score may not be displayed, or the identity probability or the authentication score may be displayed.

(5) In the above embodiment, the case where the plurality of imaging terminals 2 and the plurality of display terminals 3 are connected to the intranet as the network N has been described as an example. Further, the server 1 has been described by way of example when connected to the Internet as the network N.

  However, the imaging terminal 2, the display terminal 3, and the server 1 may be connected within the same intranet. Moreover, the imaging terminal 2, the display terminal 3, and the server 1 may each be directly connected to the Internet.

  In addition, the imaging terminal 2, the display terminal 3, and the server 1 are not limited to being connected via the network N, and the functions of the imaging terminal 2, the display terminal 3, and the server 1 have one system (for example, a computer) A mode integrated with the system) may also be adopted. In this case, the imaging terminal 2, the display terminal 3, and the server 1 can be connected via a bus (data bus) inside the computer system. Even when a bus is connected instead of the network N, the amount of bus communication is reduced, so that high-speed processing can be realized.

(6) In the above embodiment, the case where the application target of the image authentication system 100 is the visitor H who is a person has been described. However, the application target of the image authentication system 100 is not limited to the visitors H.

  For example, it may be a person who leaves. In some cases, it is not a person but an animal such as a dog or a cat. In some cases, it is not a person but a bag, shoes, glasses, a hat, or the like (for example, personal belongings). The application target of the image authentication system 100 may be an object having a shape and shape.

(7) In the above embodiment, a store is exemplified as a place where the image authentication system 100 is used. However, the application range of the image authentication system 100 is not limited to this.

  In addition to stores, the present invention can be applied to facilities such as commercial facilities, airports, banks, office buildings, and condominiums where many people come and go.

  Note that the configurations disclosed in the above-described embodiments (including other embodiments, the same applies hereinafter) can be applied in combination with the configurations disclosed in the other embodiments as long as no contradiction arises. The embodiment disclosed in this specification is an exemplification, and the embodiment of the present invention is not limited to this. The embodiment can be appropriately modified without departing from the object of the present invention.

  The present invention can be applied to the provision of an image authentication system, an image authentication method, and an image authentication program that allow a monitor to visually recognize a monitoring target immediately and easily.

1: Server 2: Imaging terminal (imaging unit)
3: Display terminal (display unit)
5: Position display (display)
12: Verification unit 22: Cutout unit 24: Camera (imaging unit)
35: Monitor (display unit)
43: Input feature image 43d: Input image data 44: Registered image 45: Monitoring video (monitoring image)
100: Image authentication system P: Monitoring image R: Monitoring target area X: Detection image data

Claims (8)

An imaging unit that captures a monitoring target area and obtains a monitoring image;
A monitoring image storage unit for storing the monitoring image;
A feature image of an imaging target included in the monitoring image is cut out as an input feature image, and the input feature image, image specifying information for specifying the monitoring image, and a cutting position of the input feature image in the monitoring image are specified. A feature image cutout unit that generates input image data including positional information
A collation unit that collates the input image data with a registered image database to be monitored stored in advance, and detects the input image data corresponding to the registered image database as detected image data;
When the detected image data is detected as a result of collation by the collation unit, at least a part including the input feature image of the monitoring image specified based on the detected image data, and a position of the input feature image And an image authentication system.   The image authentication system according to claim 1, wherein the monitoring image is a moving image.   The image authentication system according to claim 1, wherein the image specifying information is an imaging time or an image frame number.   The image authentication system according to any one of claims 1 to 3, wherein the position information is coordinate information in the monitoring image. The input feature image includes a face image;
The image authentication system according to claim 4, wherein the position information is coordinate information of an eye part in the face image. The collation unit generates authentication result information that is information on a degree of coincidence between the imaging target included in the input feature image and the monitoring target included in the registered feature image,
The image authentication system according to claim 1, wherein the display unit acquires and displays the authentication result information. An imaging step of imaging a monitoring target area to obtain a monitoring image;
A monitoring image storing step for storing the monitoring image;
A feature image of an imaging target included in the monitoring image is cut out as an input feature image, and the input feature image, image specifying information for specifying the monitoring image, and a cutting position of the input feature image in the monitoring image are specified. A feature image cut-out step for generating input image data including position information,
A collation step of collating the input image data with a registered image database to be monitored stored in advance, and detecting the input image data corresponding to the registered image database as detected image data;
When the detected image data is detected as a result of the collation in the collating step, at least a part including the input feature image of the monitoring image specified based on the detected image data, and a position of the input feature image And a display step for displaying the image. An image authentication program for causing a computer to execute information processing between an imaging terminal storing a monitoring image acquired by imaging a monitoring target area and a display terminal for displaying the monitoring image,
The feature image of the imaging target included in the monitoring image of the imaging terminal is cut out as an input feature image, the input feature image, image specifying information for specifying the monitoring image, and the input feature image in the monitoring image A feature image cutting process for generating input image data including position information for specifying a cutting position;
Collating the input image data with a pre-stored registered image database to be monitored, and collating processing for detecting the input image data corresponding to the registered image database as detected image data;
When the detected image data is detected as a result of collation by the collation processing, at least a part including the input feature image of the monitoring image specified based on the detected image data, and the position of the input feature image Display processing for displaying the image on the display terminal.

Images