JP2008009560A - Appearance determination device, appearance determination program, and recording medium with the same recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prepare desired types of rules (functions) of setting expressions representing features of parts from image data of a person. <P>SOLUTION: Data (feature data) obtained by digitizing features of parts such as the face according to a pickup image of a person is stored in an appearance feature database 8. Appearance words representing features of human appearances (such as round face and dark skin) are stored in an appearance determination database 12 together with reference values used to determine appearance words corresponding to the feature data. An appearance word generation engine 4 compares the feature data with the reference values to determine appearance words to be associated with the feature data, and records the appearance words to be associated with the feature data in an integrated database 13. The integrated database 13 stores the appearance words together with the human image data. A word registration engine 5 changes and deletes the appearance words stored in the appearance determination database 12 or adds words therto. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an appearance determination apparatus that easily determines a person's appearance and state by expressing the person's appearance and state with words.

  Conventionally, by extracting a feature of a person using a photographed person image (for example, a face image) and digitizing the feature, whether the feature matches a pre-registered person feature or not A method for determining whether or not (for example, Patent Document 1) is known. Such a method is widely used as a personal authentication technique.

In Patent Document 1, the feature amount of each part of the face is determined from the captured face image data and registered in the face image database as personal data, and by inputting the feature amount of the face component at the time of retrieval, It describes that image data having a feature amount is displayed. As the feature amount of the face part, for example, feature amounts such as “thin”, “large”, and “normal” are set for the eyes according to a predetermined rule.
JP 11-85992 A (published March 30, 1999)

  In the technique disclosed in the above-mentioned Patent Document 1, it is considered that the above-described rules for setting feature quantities are prepared in advance. For this reason, it is impossible to set and register a feature amount for a desired part as a face part. For example, if you want to prepare standard rules for setting special feature quantities such as beards and eyebrows that are not common as facial parts, it is necessary to prepare many types of rules and store the rules. The database capacity to be kept must be increased.

  The present invention has been made in view of the above-described problems, and an object thereof is to prepare only a desired type of rules (functions) for setting feature amounts.

  In order to solve the above-described problem, the appearance determination apparatus according to the present invention extracts a parameter representing a feature of each part of a person from a photographed person image and registers it in a feature database, A feature word database that registers a feature word that expresses a feature of a part and a condition that specifies the feature expressed by the feature word together with a determination function for determining whether or not the parameter satisfies the condition; A feature word extraction unit that extracts a feature word corresponding to the parameter from the feature word database when it is determined based on the determination function that the parameter registered in the feature database satisfies the condition registered in the feature word database; , A correspondence database in which person images and extracted feature words are registered in association with each other, and A feature word that matches each feature word is searched in the correspondence database, and an image extraction unit that extracts an image corresponding to the feature word that matches the input feature word and a new determination function are added to the feature word database. Or a decision function adding / subtracting unit that performs at least one of deleting the decision function registered in the feature word database.

  In the above configuration, parameters (numerical data) representing features of each part such as a face from the photographed person image are stored in the feature database. On the other hand, in the feature word database, feature words (round face, color black, etc.) representing the features of each part of the person and conditions for specifying the features expressed by the feature words are stored together with the judgment function. ing. As the above condition, for example, in the case of “round face”, the range of the ratio of the vertical and horizontal sizes of the face can be mentioned.

  Here, when the feature word extraction unit determines that the parameter registered in the feature database satisfies the condition registered in the feature word database based on the determination function, the feature word corresponding to the parameter is the feature word. Extracted from the database. The extracted feature words are registered in association with a person image in the correspondence database. As a result, the characteristic words that characterize the person are linked to the image of the person.

  In addition, the image extraction means searches the correspondence database for feature words that respectively match the input feature words, and extracts an image corresponding to the feature words that match the input feature words. As a result, an image associated with the appearance word can be extracted simply by inputting the appearance word.

  In addition, the decision function adding means adds a new decision function to the feature word database or deletes the designated decision function from the feature word database. As a result, the minimum necessary determination function can be registered in the feature word database.

  The appearance determining apparatus preferably includes a feature word changing unit that changes the feature words registered in the feature word database. Thereby, it is possible to prepare feature words desired by the user.

  In the appearance determination apparatus, the feature word database registers feature words that express distinctive features as feature words, and further records data of voices of photographed persons in association with images and feature words. The image extraction means searches the feature word database for feature words that respectively match the input feature words, and searches the corresponding database for the input voice data to obtain the feature words that match the input feature words. It is preferable to extract an image corresponding to voice data that corresponds to the input voice data.

  In this configuration, it is possible to extract an image of a person who has a stronger impression. Moreover, if feature words representing the features of celebrities are stored in the feature word database, images of persons similar to celebrities can be easily selected.

  Further, in the appearance determination apparatus, the feature extraction unit extracts, for each image, a parameter representing a feature of each part of the person from a plurality of images of the person photographed in time series, and registers the parameter in the feature database. The appearance determination device is further extracted by a state word database that registers a change condition that specifies a change in the state of each part of a person and a state word that represents an operation state that satisfies the change condition, and feature extraction means. When it is determined that the parameter for each image satisfies the change condition registered in the state word database, it is preferable to include state word extraction means for extracting a state word corresponding to the change condition from the state word database.

  In the above configuration, the parameters representing the features of each part of the person are extracted for each image from the plurality of images of the person photographed in time series by the feature extraction means, and registered in the feature database in advance. On the other hand, in the state word database, a change condition that specifies a change in the state of each part of the person and a state word that represents an operation state that satisfies the change condition are registered. For example, when an eye is used as the part, the changing condition includes opening / closing of the eye and a closed state of the eye. In addition, blinking and sleeping are associated with the operating state that fulfills these changing conditions.

  Here, when the parameter for each extracted image satisfies the change condition registered in the state word database, the state word corresponding to the change condition is extracted from the state word database.

  Accordingly, it is possible to authenticate the person together with detecting the movement of the person. Therefore, authentication of impersonation by a photograph can be rejected. In addition, it is possible to construct a risk avoidance system that issues a warning to the driver by determining the drowsiness state based on the driver's eye state or determining the yawning state of the mouth.

  The appearance determination program of the present invention causes a computer to function as each means of any of the above-described appearance determination apparatuses. The recording medium of the present invention is a computer-readable recording medium that records the appearance determination program. With the above-described appearance determination program and recording medium, the minimum necessary determination function can be registered in the feature word database as in the appearance determination apparatus.

  As described above, the appearance determination device according to the present invention adds at least one of a new determination function to the feature word database or deletes the determination function registered in the feature word database. Since the judgment function adding / subtracting means for performing the above is provided, the minimum necessary judgment function can be registered in the feature word database. Therefore, there is an effect that the capacity of the feature word database necessary for storing the determination function can be reduced.

  An embodiment of the present invention will be described with reference to FIGS. 1 to 9 as follows.

  As shown in FIG. 1, the appearance determination system 1 according to the present embodiment is configured by a computer device, and includes a CPU, memory (RAM, ROM, etc.), external storage device (hard disk drive, MO drive, etc.), and display device. And an input device (keyboard, mouse, etc.). In addition, the appearance determination system 1 performs a feature determination engine 2, a voiceprint data detection engine 3, a appearance word generation engine 4, a word registration engine 5, a state word generation engine 6, and a appearance word search engine 7 to perform the appearance determination described later. , The appearance feature database 8, the voiceprint database 9, the appearance determination database 10, the first impression database 11, the state determination database 12, and the integrated database 13.

  Image data of a person from the camera 14 is input to the feature extraction engine 2. The camera 14 takes an image of a person and outputs it as image data such as a bitmap format. The feature extraction engine 2 recognizes each part such as a person's face as a part based on image data (for example, face image data) input from the camera 2 and digitizes the position, size, color, etc. of each part. The feature data (parameter) representing the feature is written into the appearance feature database 8. The feature extraction engine 2 can apply a feature extraction technique used in conventional face authentication, and the feature data corresponds to, for example, feature amount data described in Patent Document 1 described above.

  The appearance feature database 8 collectively manages feature data input from the feature extraction engine 2 for each person together with person image data. Further, the appearance feature database 8 may manage the feature data for each person together with personal information (name, age, affiliation, etc.) of the person inputted separately.

  The voiceprint data detection engine 3 detects voiceprint data based on the voice signal of the person's voice input from the microphone 15 and writes it in the voiceprint database 9.

  The appearance word generation engine 4 generates appearance words (feature words) expressing the characteristics of each part of the person using the determination function and the appearance word table based on the feature data stored in the appearance feature database 8. Various determination functions and appearance word tables are stored in the appearance determination database 10 in association with each other.

  The appearance word table is a reference value range (determination standard maximum value and determination criterion minimum value) of feature data for determining whether or not various appearance words and feature data correspond to the appearance words, that is, the appearance words. Are stored in association with the conditions for identifying the features expressed in the above, and are prepared for each group of appearance words. For example, “round face”, “face length”, “thin face”, and the like are prepared as group appearance words for face shapes, and a reference numerical value range is set for each.

  As the determination function, a simple appearance determination function and a complex appearance determination function are prepared. The simple appearance determination function is a function for determining whether or not the feature data corresponds to the appearance word, that is, whether or not the feature data satisfies the above-described condition, and is a reference in which the feature data is stored in the appearance word table. If it is determined that the value is within the numerical range, the corresponding appearance word within the reference numerical range is extracted. The composite appearance determination function is a function for extracting other appearance words based on a plurality of determination appearance words stored in the integrated database described later, and is associated with a plurality of determination appearance words. The matched appearance words are extracted from the appearance word table among other appearance words.

  The appearance word generation engine 4 includes a simple appearance determination function execution unit 41 and a composite appearance determination function execution unit 42, as shown in FIG.

  The simple appearance determination function execution unit 41 loads a determination function for simple appearance determination (simple appearance determination function) from the appearance determination database 10, executes the determination function, and integrates the extracted appearance words as appearance word data. Store in database 13. The simple appearance determination function execution unit 41 extracts the face image data that is the basis of the feature data corresponding to the appearance word from the appearance feature database 8 and also extracts the voice print data corresponding to the face image data from the voice print database 9. In association with the appearance word data, it is written in the integrated database 13. In the integrated database 13, each data is managed in the form of integrated data formed by appearance word data, face image data, and voiceprint data relating to the same person.

  The composite appearance determination function execution unit 42 loads a composite appearance determination function (composite appearance determination function) from the appearance determination database 10, executes the determination function, and uses the extracted appearance words as appearance word data. The same is stored in the integrated database 13.

  IDs are assigned to the face screen data and the voiceprint data by the feature extraction engine 2 and the voiceprint data detection engine 3 when the person's camera 14 shoots and the microphone 15 collects the sound, respectively. Based on the ID, the simple appearance determination function execution unit 41 and the composite appearance determination function execution unit 42 retrieve the face screen data and the voiceprint data from the appearance feature database 8 and the voiceprint database 9 as the same person data, respectively.

  Further, the appearance word generation engine 4 uses the determination function and the appearance word table stored in the first impression determination database 11, and the appearance word similarly to the case where the determination function and the appearance word table of the appearance determination database 10 are used. In particular, the appearance words representing the outstanding features of the person that are likely to remain as the first impression are extracted.

  The appearance word table in the first impression determination database 11 stores appearance words that are likely to remain as the first impression when first meeting, such as “large eyes” and “gera”.

  The state word generation engine 6 generates a state word (feature word) representing a person's state (motion or expression) using a determination function and a state word table based on the feature data stored in the appearance feature database 8. To do. Various determination functions and state word tables for determining state words are stored in the state determination database 12 in association with each other.

  The state word table stores various state words and feature data change conditions for determining whether or not the feature data corresponds to the appearance words, and is prepared for each state word group. ing. For example, “sleep”, “blink”, etc. are prepared as appearance words of the eye state group, and a change condition for determination is set for each. For example, as the change condition for determining the state word “blink”, the vertical size of the opened eye and the vertical size of the closed eye are prepared. In addition, as a change condition for determining the state word “sleeping”, a vertical size of the closed eye is prepared.

  Since the state word generation engine 6 extracts a state word representing a person's state based on the feature data, the feature data is obtained from a plurality of face image data of a person photographed a plurality of times within a predetermined time by the camera 14. The obtained feature data is used. For example, when determining the blinking state, it is necessary to increase the number of times of shooting, such as 1000 times in 30 seconds, in order to capture the moment because the eye is closed for a moment. Conversely, when determining the sleep state, the number of times of photographing may be small, such as two or three times in 30 seconds.

  The state word generation engine 6 has a state determination function execution unit 61 as shown in FIG. 2 in order to determine the above state words. The state determination function execution unit 61 loads a determination function for state determination (state determination function) from the state determination database 12 based on feature data based on a plurality of shootings, executes the determination function, and extracts the result. The state word thus stored is stored in the integrated database 13 as state word data. In addition, the state determination function execution unit 61 extracts the face image data that is the basis of the feature data corresponding to the state word from the appearance feature database 8, and also extracts the voice print data corresponding to the face image data from the voice print database 9. , Write it in the integrated database 13 in association with the state word data. In the integrated database 13, each data is managed in the form of integrated data formed by state word data, face image data, and voiceprint data relating to the same person.

  The state word data may not be integrated with the voiceprint data as described above.

  The word registration engine 5 modifies appearance words and state words including changes, additions, and deletions of appearance words and state words. The word registration engine 5 includes a registration processing unit 51, a word table changing unit 52, a determination function deleting unit 53, and a determination function adding unit 54.

  The registration processing unit 51 operates any one of the word table changing unit 52, the determination function deleting unit 53, and the determination function adding unit 54 in accordance with a user instruction.

  Hereinafter, when the appearance word table and the state word table are not limited, they are collectively referred to as a word table.

  The word table changing unit 52 displays the word table instructed by the user to be changed on a display device (not shown) included in the appearance determination system 1 and accepts the change by the user. The changeable elements received by the word table changing unit 52 include at least appearance words, state words, reference numerical value ranges, and changing conditions. When the word table changing unit 52 receives a change instruction from the user, the word table changing unit 52 rewrites the word table in the appearance determination database 10 with the changed word table.

  The determination function deletion unit 53 displays a list of determination functions currently registered in the appearance determination database 10 and the corresponding word table, and accepts a determination function and a deletion input of the word table by the user. The determination function deletion unit 53 deletes the determination function and the word table in the appearance determination database 10 when receiving a deletion instruction from the user.

  When the determination function adding unit 54 receives an instruction to add a determination function by the user, the determination function adding unit 54 adds the determination function instructed to be added and the word table corresponding thereto to the appearance determination database 10. For this reason, the determination function adding unit 54 prepares several determination functions and word tables corresponding to them in advance, and displays a list of determination functions on the display device so that the user can select them.

  The appearance word search engine 7 searches the integrated database for one or more input appearance words, and displays all face image data corresponding to the appearance words matching the appearance words on the display device. Further, when performing speech authentication in the search mode, the appearance word search engine 7 searches the integrated database 13 for voiceprint data converted by the voiceprint data search engine 3 based on the voice recorded by the microphone 17. As a result of this search, the appearance word search engine 7 specifies face image data corresponding to the voice print data matching the voice print data in the face image data displayed corresponding to the appearance words.

  Next, the operation of the appearance determination system 1 configured as described above will be described.

  First, generation (extraction) of appearance words by the appearance word generation engine 4 will be described with reference to the flowchart of FIG.

  First, the simple appearance determination function execution unit 41 loads a simple appearance determination function corresponding to the feature data stored in the appearance feature database 8 from the appearance determination database 10 and executes it (S1).

  At this time, for example, when executing the face shape determination function shown in FIG. 4A, as shown in FIG. 4B, the face contour position and the vertical and horizontal sizes of the output data from the appearance feature database 8 are set. Obtain (S11). Then, the vertical size of the face outline is divided by the horizontal size (S12). Appearance words having a reference numerical value range between the determination criterion maximum value and the determination criterion minimum value to which this division result applies are searched in the appearance word table relating to the face shape as shown in FIG. 4C (S13). As a result, all appearance words having a reference numerical value range to which the division result applies are returned to the simple appearance determination function execution unit 41.

  When the skin color determination function shown in FIG. 5A is executed, the face contour position and the vertical and horizontal sizes are acquired from the output data from the appearance feature database 8 as in the case of executing the face shape determination function (S21). ). At this time, the average color in the face contour is calculated based on the face image data from the appearance feature database 8 (S22). Appearance words having a reference numerical value range between the determination reference maximum value and the determination reference minimum value to which this average value applies are searched in the appearance word table for skin color as shown in FIG. 5B (S23). As a result, all the appearance words having the reference numerical value range to which the average value applies are returned to the simple appearance determination function execution unit 41.

  In this way, when the simple appearance determination function execution unit 41 acquires the appearance word, it records the appearance word as a determination result in the integrated database 13 (S2). When all the necessary simple appearance determination functions corresponding to each feature data are executed, the process moves to the composite appearance determination function execution unit 42.

  The composite appearance determination function execution unit 42 loads the corresponding composite appearance determination function from the appearance determination database 10 and executes it with reference to the appearance words stored in the integrated database 13 (S4).

  At this time, for example, when the race determination function shown in FIG. 6A is executed, the hair color determination result (appearance word) and the skin color determination result (appearance word) are acquired from the integrated database 13 (S31). Then, for each determination type (hair color and skin color) of both determination results, the appearance words having the appearance words (judgment reference appearance words) to which both determination results apply are shown in FIG. 6B. Search in the table (S32). As a result, the appearance word corresponding to the determination reference appearance word is returned to the composite appearance determination function execution unit 42.

  Note that the “reference degree” in the appearance word table of FIG. 6B is provided to show the corresponding degree as a reference because not all of the determination reference appearance words are associated with the corresponding appearance words.

  In this way, when the composite appearance determination function execution unit 42 acquires the appearance word, it records the appearance word as a determination result in the integrated database 13 (S5). Then, when all necessary complex appearance determination functions are executed, the appearance word generation processing is completed.

  Next, the determination of the operation state by the state word generation engine 6 of the appearance determination system 1 will be described.

  First, the camera 14 continues to photograph a person for determining the state. The feature extraction engine 2 extracts feature data based on each captured face image data and stores it in the appearance feature database 8. The state determination function execution unit 61 of the state word generation engine 6 loads a state determination function corresponding to the feature data stored in the appearance feature database 8 from the state determination database 12 and executes it.

  At this time, for example, when the eye movement state determination function shown in FIG. 8 is executed, the right and left eye positions and the vertical and horizontal sizes in the normal state are acquired as feature data from the appearance feature database 8 (S51). Then, for the photographed face image, the left and right eye positions and the vertical and horizontal sizes as the feature data are acquired (S52). Furthermore, the presence / absence of a change in the vertical size from the normal time is detected by comparing the acquired vertical sizes of the eyes (S53). At this time, the vertical size of all photographed face images is compared with the normal vertical size. A state word having a change condition (closed state, open / closed change, etc.) to which the comparison result applies is searched in the state word table related to the eye movement state (S54). As a result, all state words having change conditions to which the comparison result applies are returned to the determination function execution unit 61. When the state word generation engine 6 acquires the state word in this manner, the state word generation engine 6 records the state word in the integrated database 13 as a determination result.

  Next, registration of appearance words and state words by the word registration engine 5 will be described with reference to the flowchart of FIG.

  First, the registration processing unit 51 determines whether or not the change of the appearance word table is selected by the mode selection by the user (S41). When the change of the appearance word table is selected (word table change mode), the word table changing unit 52 changes the word table according to the user's instruction (S42), and the process is finished.

  When the change of the appearance word table is not selected, the registration processing unit 51 determines whether or not deletion of the determination function is selected by the mode selection by the user (S43). When deletion of the determination function is selected (determination function deletion mode), the determination function deletion unit 53 deletes the target determination function and the corresponding word table according to the user's instruction (S44), and the process ends. .

  If deletion of the determination function is not selected, the registration processing unit 51 determines whether or not addition of the determination function is selected by the mode selection by the user (S45). When the addition of the determination function is selected (determination function addition mode), the determination function adding unit 54 adds the target determination function and the corresponding word table according to the user's instruction (S46), and ends the process. .

  As described above, the appearance determination system 1 extracts the appearance words expressing the characteristics based on the extracted characteristics of the person's appearance, and adds the appearance words together with the image data of the person to the integrated database 13. Thus, by searching the integrated database 13 based on the inputted appearance words, all image data corresponding to the appearance words that match the input appearance words can be extracted and displayed. Therefore, it is possible to select a target person image from a large number of person images by appearance words, and in a criminal investigation etc., it is possible to narrow down candidates in descending order of accuracy from the images of a large number of previous persons by the witness testimony It becomes.

  Further, by extracting the appearance words using the first impression database 11, it is possible to select a person image having a stronger impression. By storing appearance words representing the characteristics of celebrities in the first impression database 11, images of persons similar to celebrities can be easily selected.

  In addition, by registering voiceprint data together with appearance words and face image data in the integrated database 13, in addition to narrowing down images by appearance words, a person can be specified by voiceprints, and more accurate. High person authentication can be performed.

  In addition, by providing the state word generation engine 6 and the state determination database 11, it is possible to compare the feature data obtained from normal image data prepared in advance with the feature data obtained from captured image data. Can be extracted as state words. As a result, in the conventional face authentication technology, authentication is performed not only by the person but also by a photograph, but by authenticating the face in conjunction with detecting the movement of a person such as blinking, the authentication of impersonation by a photograph is rejected. Can do. In addition, it is possible to construct a risk avoidance system that issues a warning to the driver by determining the drowsiness state based on the driver's eye state or determining the yawning state of the mouth.

  Furthermore, the provision of the word registration engine 5 makes it easy to change, delete and add appearance words and state words, and store the minimum necessary judgment functions and word tables in the judgment databases 10-12. That's fine. Thereby, each determination database 10-12 can reduce the capacity | capacitance required for storage of a determination function and a word table.

  The appearance determination system 1 is configured by a single computer as illustrated in FIG. 1, but may be configured separately from the server apparatus 16 and the client apparatus 17 as illustrated in FIG. 9. Good. In this configuration, the client device 17 is provided with the feature extraction engine 2, the voice print data detection engine 3, the appearance feature database 8 and the voice print database 9, while the server device 16 has the other engines 5 to 7 and the databases 10. To 13 are provided. In addition, a communication process for exchanging files and data via the LAN 18 is executed by the server processing unit 19 provided in the server device 16 and the client processing unit 20 provided in the client device 17.

  With such a configuration, it is possible to input an image or sound at a position away from the server device 16. Further, the client device 17 is configured by an operation display device such as a programmable display or a panel computer, and the server device 16 is configured by a host computer, so that photographing of a face or voice can be performed at a production site where the programmable display is installed. Recording can be performed, and the degree of freedom of system configuration can be further increased.

  Finally, each of the engines 2 to 7 described above is realized by software using a CPU.

  That is, the appearance determination system 1 includes a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, and a RAM (random access memory) that expands the program. The present invention is realized by a computer including a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a recording medium in which program codes (execution format program, intermediate code program, source program) of a control program of the appearance determination system 1 which is software for realizing the above-described functions are recorded so as to be readable by a computer. This can also be achieved by supplying to the appearance determination system 1 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a disk system including a magnetic disk such as a flexible disk / hard disk and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R, and an IC card. A card system such as an optical card (including a memory card) or a semiconductor memory system such as a mask ROM / EPROM / EEPROM / flash ROM can be used.

  Moreover, the appearance determination system 1 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Further, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used.

  The appearance determination system 1 can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  The appearance determination apparatus according to the present invention stores an appearance word representing a characteristic of a person's appearance in association with the image of the person, and searches for an image based on the appearance word, thereby expressing the impression expressed by the appearance word. Since the image can be extracted only by this, for example, it can be applied to the use of selecting a desired person image from a large number of person images.

It is a block diagram which shows the structure of the appearance determination system which shows one Embodiment of this invention. It is a block diagram which shows the structure of the principal part of the said figure determination system. It is a flowchart which shows the procedure of the execution process of the determination function by the appearance word production | generation engine in the said appearance determination system. (A) is a flowchart which shows the process sequence by the simple appearance determination function which the said appearance word generation engine performs, (b) is a figure which shows the output data of the feature extraction engine referred at the time of processing of a simple appearance determination function (C) is a figure which shows the appearance word table referred at the time of the process of a simple appearance determination function. (A) is a flowchart which shows the process sequence by the other simple appearance determination function which the said appearance word generation engine performs, (b) is the output data of the feature extraction engine referred at the time of the process of another simple appearance determination function. (C) is a figure which shows the figure word table referred at the time of the process of another simple figure determination function. (A) is a flowchart which shows the processing procedure by the composite appearance determination function which the said appearance word generation engine performs, (b) is a figure which shows the appearance word table referred at the time of the process of a composite appearance determination function. It is a flowchart which shows the process sequence by the state determination function which the state word generation engine in the said appearance determination system performs. It is a flowchart which shows the process sequence of the change of the word table by the word registration engine in the said appearance determination system, and the deletion / addition of a determination function. It is a block diagram which shows the structure of the appearance determination system which shows the other form of implementation of this invention.

Explanation of symbols

1 Appearance determination system (appearance determination device)
2 Feature extraction engine (feature extraction means)
3 Voiceprint data detection engine 4 Appearance word generation engine (feature word extraction means)
5 Appearance word registration engine (judgment function addition / deletion means, characteristic word change means)
6 State word generation engine (state word extraction means)
7 Appearance word search engine (image extraction means)
8 Appearance feature database 9 Voiceprint database 10 Appearance determination database (feature word database)
11 First impression database (characteristic database)
12 State determination database (state word database)
13 Integrated database (corresponding database)
14 Camera 15 Microphone

Claims (6)

Feature extraction means for extracting parameters representing the characteristics of each part of the person from the photographed person image and registering them in the feature database;
Feature words that register a feature word that expresses a feature of each part of a person and a condition that identifies the feature expressed by the feature word together with a determination function for determining whether or not the parameter satisfies the condition A database,
A feature word extraction means for extracting a feature word corresponding to the parameter from the feature word database when determining that the parameter registered in the feature database satisfies the condition registered in the feature word database based on the determination function; ,
A correspondence database in which person images and extracted feature words are registered in association with each other;
An image extraction means for searching for feature words that respectively match the input feature words in a correspondence database and extracting an image corresponding to the feature words that match the input feature words;
A judgment function adding / subtracting unit that performs at least one of adding the new judgment function to the feature word database or deleting the judgment function registered in the feature word database. Appearance judging device characterized by that.   2. The appearance determination apparatus according to claim 1, further comprising a feature word changing unit that changes the feature words registered in the feature word database. The feature word database registers feature words that express distinctive features as feature words, and registers data of voices of photographed persons in further association with images and feature words,
The image extraction means searches the feature word database for feature words that respectively match the input feature words, and searches the corresponding database for the input voice data, and corresponds to the feature words that match the input feature words. 3. The appearance determination apparatus according to claim 1, wherein an image corresponding to voice data that matches the input voice data is extracted. The feature extraction means extracts, for each image, a parameter representing the characteristics of each part of the person from a plurality of images of the person photographed in time series, and registers them in the feature database.
Appearance determination device
A state word database that registers a change condition that identifies a change in the state of each part of the person and a state word that represents an operation state that satisfies the change condition;
When it is determined that the parameter for each image extracted by the feature extraction unit satisfies the change condition registered in the state word database, a state word extraction unit that extracts a state word corresponding to the change condition from the state word database; The appearance determining apparatus according to claim 1, further comprising:   A figure determination program for causing a computer to function as each means of the figure determination apparatus according to any one of claims 1 to 4.   A computer-readable recording medium on which the program according to claim 5 is recorded.

Images