JP2001016579A - Image monitor method and image monitor device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and surely recognize a subject extracted from a moving image and a still image in accordance with each user's convenience or the like by comparing the feature value of the subject extracted from the image with that of each category registered in a dictionary, presenting the recognition result of a category of the subject, and updating the dictionary on the basis of the feature value of the subject. SOLUTION: A visitor monitor device is composed by an image input part 1, a sound input part 2, a human body detecting part 3, an information extracting part 4, a storage part 5, a monitor part 6 and a user information input part 7. The human body detection part 3, for example, detects a change from an image acquired by a camera or the like, detects the human body. The information extracting part 4 extracts information necessary for record or recognition from the inputted image or sound. The storage part 5 is composed of a storage medium for storing dictionary information, extracted image information, recorded sound information and the like. The user information input part 7 performs input or correction of a name and a category of a visitor and retrieval and arrangement of the recorded information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image monitoring apparatus which can be used for, for example, an intercom with a television, a video mail, a videophone, and the like, and more particularly, monitors a subject such as a person from a moving image including the person. The present invention relates to an image monitoring device.

[0002]

2. Description of the Related Art Telephone intercoms installed at the entrance of a condominium or at the entrance of a house allow visitors to check the status of visitors on a monitor. Is desired. In order to realize such a function, in order to save an image as a moving image, using a VTR device or the like makes the device large and cumbersome, and when a storage medium such as a hard disk is used, the storage capacity becomes large. Therefore, it has been difficult to reduce the size and cost.

On the other hand, in order to solve these problems,
When saving as a still image, an image that can identify a person is not always saved, and there is a possibility that the image cannot be identified when viewed later.

[0004] Even if an image that can be identified exists, it is not always easy to collect money other than an acquaintance, and to identify a delivery person or a suspicious person. In addition, it is expected that even an acquaintance will forget the name of a person in the future with the aging of society.

[0005] Recently, the price of personal computers with video cameras has been reduced, so that video mail, videophone, and the like using personal computers can be easily realized even at home. When moving images are routinely used on a personal computer in this way, a storage device such as a hard disk is limited, and a new problem arises in that it takes time to read and search once stored data.

[0006]

SUMMARY OF THE INVENTION Accordingly, the present invention has been made in view of the above problems, and it is easy to recognize a subject such as a person extracted from a moving image or a still image in accordance with the convenience and needs of each user. It is another object of the present invention to provide an image monitoring method and an image monitoring apparatus using the method, which can be performed more reliably.

[0007]

According to an image monitoring method of the present invention, a dictionary for registering a feature amount for each category for recognizing a subject extracted from an input image is generated.
By comparing the feature amount of the subject extracted from the input image with the feature amount of each category registered in the dictionary, the category of the subject is recognized, and the recognition result is presented,
By updating the dictionary based on the feature amount of the subject, it is possible to easily and reliably recognize the subject such as a person extracted from a moving image or a still image according to the convenience and needs of each user.

The image monitoring apparatus of the present invention generates a dictionary in which feature amounts for each category are registered for recognizing a subject extracted from an input image, and generates a dictionary of the subject extracted from the input image. The feature amount is compared with the feature amount of each category registered in the dictionary, the category of the subject is recognized, the recognition result is presented, and the dictionary is updated based on the feature amount of the subject. By storing the voice message in the storage unit in association with the recognition result or the dictionary, the recognition of a subject such as a person extracted from a moving image or a still image can be performed easily and reliably in accordance with the convenience and needs of each user. And more intelligent response to user needs.

Preferably, of the plurality of input time-series images, only an image in which a subject whose feature amount is most similar to one of the categories registered in the dictionary is extracted is stored in the storage means. As a result, only an image suitable for recognizing a subject such as a person can be selectively stored from a moving image and a still image, and the storage capacity can be reduced.

The image monitoring apparatus according to the present invention has a dictionary generating means for generating a dictionary in which feature amounts for each category are registered for recognizing a subject extracted from an input image; Recognizing means for recognizing the category of the subject by comparing the feature quantity of the subject with the feature quantity of each category registered in the dictionary, presentation means for presenting a recognition result by the recognizing means, And updating means for updating the dictionary on the basis of the feature amount of the moving image, the recognition of a subject such as a person extracted from a moving image or a still image can be easily and easily matched to the convenience and needs of each user. It can be done reliably.

Preferably, the apparatus further comprises storage means for storing only an image in which a subject whose feature amount is most similar to one of the categories registered in the dictionary is extracted from the plurality of input time-series images. I do. As a result, only an image suitable for recognizing a subject such as a person can be selected and stored from a moving image or a still image, so that the storage capacity can be reduced, and the size and cost of the apparatus can be reduced accordingly.

[0012]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) In the first embodiment,
Record messages and images of visitors while they are away by using an intercom with a TV camera installed at the entrance of a condominium or home, or display the names of visitors or whether they belong to a pre-registered category while at home. A visitor monitoring device that enables a smooth response is described.

FIG. 1 shows an example of the structure of a visitor monitoring apparatus according to the present embodiment. An image input unit 1, a voice input unit 2, a person detection unit 3, an information extraction unit 4, a storage unit 5, a monitor It comprises a unit 6 and a user information input unit 7.

The image input unit 1 is for inputting an image obtained by a monitoring camera such as a video camera. The voice input unit 2 is for inputting voice obtained from a microphone of an intercom, for example.

The person detecting unit 3 is a unit that a visitor such as a bell or a buzzer voluntarily notifies the visitor, or
There is no particular limitation on a device that detects a person by detecting a change from an image acquired from a camera or the like, a device that detects a physical operation such as knocking or calling, and the like.

The information extracting section 4 extracts information necessary for recording or recognition from an input image or sound.

The storage unit 5 includes a recording medium such as a magnetic tape, a magnetic disk, a magneto-optical disk, an optical disk, and a semiconductor memory for storing dictionary information, extracted image information, recorded audio information, and the like.

The monitor section 6 includes a display device, a speaker, and the like, and displays various extracted information and signals directly input from a camera or a microphone.

The user information input section 7 is for inputting or correcting the name and category of a visitor, and for searching and organizing recorded information.

FIG. 2 shows an example of the configuration of the information extracting unit 4, which comprises a face detecting unit 8, a dictionary generating unit 9, a face recognizing unit 10, and a recording image determining unit 11.

The face detecting section 8 detects a face area of a person from the input image. For example, a region having the highest correlation value with the template is cut out from the image using a face region template created in advance.

The dictionary generator 9 creates a dictionary for face recognition from a plurality of cut out face area images. For example, when performing face recognition using a conventional subspace method, similarly to the related art, feature points such as pupil and nostril feature points are detected from the face region image, and the detected feature points are used to detect the face. Performs region normalization. The normalized image information is converted into a feature vector in a feature space, a subspace is generated from the feature vectors, and this is used as a face recognition dictionary. Then, the subspace is classified into categories and registered in the storage unit 5 as a dictionary for face recognition.

However, this operation (new registration of a dictionary) is performed when the visitor does not belong to any of the categories registered in advance, and if so, it is desirable to update the dictionary of the category. When performing this dictionary update operation, it is desirable to save not only the subspace but also statistical information such as eigenvalues and correlation matrices of the subspace at the same time as the dictionary. Then, at the time of updating the dictionary, the statistical information of the subspace may be updated based on the feature amount extracted from the face area image of the visitor.

The face recognizing unit 10 calculates the similarity between an input image sequence and a dictionary of each category to determine whether the image belongs to a category registered in advance (for example, to obtain a dot product of a feature vector in each subspace, or the like). It may be determined by some method. If it does not belong to any of the registered categories, the dictionary described above is created.

In the recording image determining unit 11, the image input unit 1
Each time a moving image is input from, an image representing the feature of the visitor is selected from the input image sequence and stored in the storage unit 5. Specifically, the selection method selects an image having the highest similarity with the visitor's dictionary from a plurality of image frames of the input image sequence. If the input moving image is stored in the storage unit 5 as it is, a large storage capacity is required. For example, when only one image is left in the storage, it is troublesome for the user to select which image is left. It is. In addition, there is a high possibility that the face will not be correctly photographed if selected mechanically. However, even if the images in the input image series do not belong to any of the categories in the dictionary (the degree of similarity is too low to judge to belong to any of the categories), the most similarity among them is The above problem can be solved by selecting only one high-priced sheet and storing it in the storage unit 5.

Next, an example of the operation of the visitor handling apparatus shown in FIG. 1 will be described, assuming that a visitor comes during an absence.

The user sets in advance that the user is in the absence state. At this time, the user may be at home. This operation may be similar to a normal answering machine. When a visitor comes in front of the camera set at the entrance of the user's home and rings the bell, or in the case of a multi-family house, enters the room number, the operation triggers the image input from the camera and the microphone. The input voice and the input voice are started by the image input unit 1 and the voice input unit 2, respectively, and displayed on the monitor unit 6 installed indoors.

On the other hand, the information extraction unit 4 first detects a face area of a person from the image input by the face detection unit 8, and stores the feature vector extracted through the dictionary generation unit 9 and the storage unit 5. With reference to the face recognition dictionary, the face recognition unit 10 recognizes the visitor's face from the image input from the image input unit 1 and determines, for example, the category of the visitor. Then, the category name as the recognition result is displayed on the monitor 6 together with the image captured from the camera.

The category name is registered in advance, and is not limited to an individual name. For example, newspaper collection, home delivery, postal delivery, etc., are likely to have the same person every time, and the individual name is meaningless. Therefore, in this case, a category name that is not an individual name, for example, "newspaper collection", "home delivery", It is also possible to classify these recognition results by a category name such as “delivery”. Similarly, for categories that do not need to be classified,
The category name may be "suspicious person".

If the category does not belong to a category registered in advance, a dictionary is generated from the input image sequence as described above. If so, the existing dictionary is updated as described above. However, if the categorized category is incorrect, it is possible to create a dictionary only for the visitor and save the original dictionary of the category without deleting it so that the user can correct it later. desirable.

The recording image determining unit 11 includes an image input unit 1
One of the images having the highest similarity to the dictionary created in the input image sequence from is selected. Even if the image is a recognized category and an unrecognized category,
Save only one of them. Dictionaries are generated from videos,
Saving only one image saves storage capacity.
In order to further reduce the storage capacity, only an image obtained by cutting out only the face area may be left.

The voice message of the visitor input from the voice input unit 1 is recorded in the storage unit 5 together with the face image and the recognition result.

The message is indicated by a keyword spot.
Recognizing a keyword by voice recognition and leaving the result facilitates later naming of categories to new dictionary registrations. In order to simplify this, it is also effective to issue a prompt such as "Please name" as voice or character information.

The user checks the message at his / her favorite time, such as when returning home. At that time, the recognized category is corrected, a new category name is input, and unnecessary messages are deleted.

When the user is at home but wants to respond after confirming the visitor, an image is displayed on the monitor,
Since the recognition result is also displayed, it is possible to respond only to a visitor who wants to respond. At this time, when there is a visitor who does not want to meet the same, it is convenient because only the category name can be used without having to remember the face.

(Second Embodiment) In the second embodiment,
A video mail (for example, in general, a moving image may be transmitted using a predetermined communication means) on a personal computer with a camera or the like, and an image sent from a partner through a video conference may be used to transmit the video mail. It is used to reduce the storage capacity, confirm the other party by face recognition, and create a face recognition dictionary.

The visitor monitoring device described in the first embodiment is mainly used for security. In the second embodiment, for example, a personal computer or the like is used to easily check image contents. The following describes an image monitoring device that can also be used.

FIG. 3 shows an example of the configuration of an image monitoring apparatus according to the second embodiment. The image input unit 12, the face detection unit 13, the dictionary generation unit 14, the face recognition unit 15, the display unit 16,
The storage unit 17 includes a recording image determination unit 18. The visitor monitoring device shown in FIG. 3 may be configured on a personal computer (personal computer). That is, for example, a magnetic disk (floppy (registered trademark)) is used as a program that allows a computer to execute the functions of the above-described units using hardware included in a personal computer.
Disk, hard disk, etc.), optical disk (CD-
ROMs, DVDs, etc.), semiconductor media, and other storage media for distribution.

The image input unit 12 reads a moving image recorded in advance on a hard disk or the like of a personal computer,
It reads moving images sent through a communication line such as a LAN or a telephone.

The face detection unit 13, dictionary generation unit 14, face recognition unit 15, storage unit 17, and recording image determination unit 18
The face detection unit 8, the dictionary generation unit 9 described in the first embodiment,
It is the same as the face recognition unit 10, the storage unit 5, and the recording image determination unit 11.

Next, an actual operation example will be described by taking the case of video mail capacity reduction as an example.

The video mail sent from the other party is usually stored on the hard disk of the personal computer. Image input unit 1
2, the data is read, only the moving image portion is extracted, and the face detection unit 13 detects a face area. The face recognition unit 15 determines whether the detected face area belongs to a person category registered in advance. The determination result is displayed together with the image on a display device 16 such as a display.

At the same time or according to a user's instruction, a dictionary is created in the dictionary generation unit 14 from the moving image. At this time, in the case of a new category, the user can newly input a category name, but in a normal case, it is convenient to use the category name as an e-mail address. For categories that already exist, instead of creating a new dictionary,
Update existing dictionaries.

The recording image determining section 18 determines a recording image from the moving image. Specifically, as described in the first embodiment, an image having the highest similarity with the dictionary is selected from the input image sequence.

Since the video mail is moving picture information, if it is stored as it is, a large storage capacity is required. For example, when only one of them is left, it is troublesome for the user to select which image is left. Yes, there is a high possibility that the face will not be correctly captured if it is selected mechanically, but this will solve that problem.

In the first and second embodiments, the case where the monitoring target is a person has been described as an example.
It is not limited to this case, and the monitoring target may be anything,
In that case, it is the same as the above description.

Further, the present invention is not limited to these examples, and can be applied in various modifications.

[0049]

As described above, according to the present invention,
Recognition of a subject such as a person extracted from a moving image or a still image can be performed easily and reliably in accordance with the convenience and needs of each user.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration example of a visitor monitoring device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a configuration example of an information extraction unit.

FIG. 3 is a diagram showing a configuration example of an image monitoring apparatus according to a second embodiment of the present invention.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 image input unit 2 voice input unit 3 person detection unit 4 information extraction unit 5 storage unit 6 monitor unit 7 user information input unit 8 face detection unit 9 dictionary generation unit 10 face recognition unit 11 ... Recording image determination unit 12 ... Image input unit 13 ... Face detection unit 14 ... Dictionary generation unit 15 ... Face recognition unit 16 ... Display unit 17 ... Storage unit 18 ... Recording image determination unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat ゛ (Reference) H04M 9/00 H04M 11/00 301 H04N 7/14 11/00 301 7/16 Z H04N 5/765 G10L 3 / 00A 551A 5/781 551P 7/14 H04N 5/781 510E 7/16 F term (reference) 5C054 AA02 CA04 CA08 CC02 CD06 CE15 CG06 CG07 CG08 CH04 CH05 DA01 DA08 EA01 EA03 EA05 EA07 EJ00 FA01 FA02 FA05 FB16 FB02 FC12 FC12 FE23 FE26 FF03 GA01 GA02 GA04 GB02 GB15 GC01 GC03 GD03 GD09 HA18 HA22 HA24 5C064 AA01 AA06 AB04 AC04 AC06 AC12 AC16 AC18 AD06 AD13 BA01 BB10 BC23 BC25 BD03 BD08 5D015 AA02 BB02 KK02 5K038 AA08 KK03 NN DD03

Claims (8)

[Claims] 1. A dictionary for registering a feature amount for each category for recognizing a subject extracted from an input image is generated, and a feature amount of the subject extracted from the input image and the dictionary are stored in the dictionary. Recognizing the category of the subject by comparing the feature amount of each registered category and presenting the recognition result,
An image monitoring method, wherein the dictionary is updated based on a feature amount of the subject. 2. The image monitoring method according to claim 1, wherein the input voice message is stored in a storage unit in association with the recognition result or the dictionary. 3. The image processing apparatus, comprising: a plurality of input time-series images;
2. The image monitoring method according to claim 1, wherein only an image from which a subject whose feature amount is most similar to one of the categories registered in the dictionary is extracted is stored in a storage unit. 4. A dictionary generating means for generating a dictionary for registering a feature amount for each category for recognizing a subject extracted from an input image, and a feature amount of the subject extracted from the input image. Recognizing means for recognizing the category of the subject by comparing the feature amount of each category registered with the dictionary; presenting means for presenting a recognition result by the recognizing means; and An image monitoring apparatus, comprising: updating means for updating the dictionary. 5. The image monitoring apparatus according to claim 4, further comprising storage means for storing the input voice message in association with the recognition result or the dictionary. 6. A plurality of input time-series images,
5. The image monitoring apparatus according to claim 4, further comprising a storage unit configured to store only an image in which a subject whose feature amount is most similar to one of the categories registered in the dictionary is extracted. 7. A dictionary generating means for generating a dictionary in which a feature amount for each category is registered for recognizing a subject extracted from an input image, and a feature amount of the subject extracted from the input image. A recognition unit for recognizing the category of the subject by comparing the feature amount of each category registered with the dictionary; a presentation unit for presenting a recognition result by the recognition unit; and a feature amount of the subject based on the feature amount of the subject. An update unit that updates the dictionary; and a machine-readable recording medium that stores a program that executes the following. 8. A plurality of input time-series images,
8. The recording medium according to claim 7, further comprising a program executing storage means for storing only an image in which a subject whose feature amount is most similar to any of the categories registered in the dictionary is extracted.