JP2021083071A - Information processing device - Google Patents

Abstract

To provide an information processing device for easily managing a camera in each zone with the camera established.SOLUTION: An image processing device 1 includes an acquisition part 161 for acquiring feature information generated on the basis of an image photographed by a camera 2 capable of photographing a specific object, the feature information related to a feature of the object extracted from an image including the object, a storage part 150 for storing registration information 51 including feature information on a feature of a registered object, and a discrimination part 162 for collating the feature information acquired by the acquisition part 161 with the feature information included in the registration information 51, and discriminating the registered object corresponding to the object in the image with the feature information generated therefrom on the basis of a collation result about the feature information. The storage part 150 stores camera information including a postal code number of a place where a plurality of cameras 2 respectively perform photographing.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing method, a program, and a camera system that perform processing related to discrimination of an object photographed by a camera.

The person photographed by the camera is identified by extracting the facial features of the person included in the image taken by the camera and collating the face image of the person registered in the database in advance with the extracted facial features. The technique is known (see, for example, Patent Document 1). In recent years, advances in image recognition technology using AI (artificial intelligence) have made it possible to discriminate people and objects with high accuracy based on images taken by a camera.

Japanese Unexamined Patent Publication No. 2003-187229

Technology that automatically identifies a person from a face image taken by a camera is applied to, for example, checking visitors at the entrance of a facility, accepting registered users at the counter, counting the number of repeaters in a store, and so on. It is possible. However, when there are restrictions on the installation location of the camera, conditions such as the distance between the camera and the subject, the shooting direction of the camera, and the surrounding lighting conditions may not be suitable for face shooting. In such a case, it may not be possible to identify the person based on the face image with sufficient accuracy. Further, even when there are a plurality of persons having similar facial features, it may be difficult to distinguish the persons by the facial image.

The present invention has been made in view of such circumstances, and an object of the present invention is to resemble the shooting conditions of a camera and the appearance of an object when the object is discriminated by using the information obtained from the image taken by the camera. It is an object of the present invention to provide an information processing device, an information processing method and a program capable of suppressing a decrease in discrimination accuracy due to the cause, and to provide a camera system provided with such an information processing device.

The information processing apparatus according to the first aspect of the present invention is feature information generated based on an image taken by one or more cameras capable of photographing a specific object, and is derived from the image including the object. In the acquisition unit that acquires the feature information regarding the extracted features of the object, a storage unit that stores the registration information including the feature information regarding one or more registered features of the object, and the acquisition unit. The acquired feature information is collated with the feature information included in the registration information, and based on the collation result of the feature information, the feature information corresponds to the object in the generated image. It has a discriminating unit for discriminating the registered object, and the registration information includes a registered postal code which is a postal code of a place related to the registered object, and the discriminating unit includes the discriminating unit. Enter the postal code of the place related to the object in the image, collate the registered postal code included in the registration information with the entered postal code, and check the collation result for the feature information and the postal code. Based on the collation result of, the registered object corresponding to the object in the image is determined.

The information processing method according to the second aspect of the present invention is feature information generated based on an image taken by one or more cameras capable of photographing a specific object, and is obtained from the image including the object. An information processing method for processing the extracted feature information related to the feature of the object by a computer, and the storage unit of the computer includes the feature information related to one or more registered features of the object. The registration information is stored, and the acquisition step of acquiring the feature information is collated with the feature information acquired in the acquisition step and the feature information included in the registration information, and the feature information is collated. Based on the result, it has a determination step of discriminating the registered object corresponding to the object in the image in which the feature information is generated, and the registration information is applied to the registered object. The registered postal code, which is the postal code of the related place, is included, and in the determination step, the postal code of the place related to the object in the image is input, and the registered postal code included in the registration information is used. It is determined that the registered object corresponding to the object in the image is determined based on the collation result of the feature information and the collation result of the postal code by collating with the input postal code. Including.

The program according to the third aspect of the present invention is a program for causing a computer to execute the information processing method according to the second aspect.

The camera system according to the fourth aspect of the present invention is a camera system including the information processing device according to the first aspect and the plurality of the cameras.

The information processing apparatus according to the fifth aspect of the present invention is feature information generated based on an image taken by one or more cameras capable of photographing a specific object, and is derived from the image including the object. In the acquisition unit that acquires the feature information regarding the extracted features of the object, a storage unit that stores the registration information including the feature information regarding one or more registered features of the object, and the acquisition unit. The acquired feature information is collated with the feature information included in the registration information, and based on the collation result of the feature information, the feature information corresponds to the object in the generated image. It has a discriminating unit for discriminating the registered object, and the storage unit stores camera information including a postal code of a place where each of the plurality of cameras takes a picture.

According to the present invention, when an object is discriminated by using information obtained from an image taken by a camera, it is possible to suppress a decrease in discrimination accuracy due to similarities in the shooting conditions of the camera and the appearance of the object. ..

FIG. 1 is a diagram showing an example of the configuration of the camera system according to the first embodiment. FIG. 2 is a diagram showing an example of registration information. FIG. 3 is a diagram showing an example of camera information. FIG. 4 is a diagram showing an example of history information. FIG. 5 is a diagram showing an example of the configuration of the camera. FIG. 6 is a flowchart for explaining an example of processing in the information processing apparatus according to the first embodiment. FIG. 7 is a diagram showing an example of registration information used in the information processing apparatus according to the second embodiment. FIG. 8 is a first flowchart for explaining an example of processing in the information processing apparatus according to the second embodiment. FIG. 9 is a second flowchart for explaining an example of processing in the information processing apparatus according to the second embodiment.

<First Embodiment>
FIG. 1 is a diagram showing an example of the configuration of the camera system according to the first embodiment. The camera system shown in FIG. 1 includes a plurality of cameras 2 and an information processing device 1 that performs processing for discriminating an object such as a person or an object based on an image taken by each camera 2. The camera 2 is installed at a position where a specific object (for example, a person who has visited a specific place) can be photographed, and features of the specific object included in the photographed image (for example, features of a person's face). Is processed to extract. The information processing device 1 corresponds to an object photographed by the camera 2 by collating the information (characteristic information) about the characteristics of the object generated by the camera 2 with the characteristic information of the object registered in advance. Determine the registered object to be processed. In the example of FIG. 1, a plurality of cameras 2 are connected to a network 9 such as the Internet, and the information processing device 1 acquires information from each camera 2 via the network 9.

[Information processing device 1]
In the example of FIG. 1, the information processing device 1 includes a communication unit 110, an input unit 120, a display unit 130, an interface unit 140, a storage unit 150, and a processing unit 160.

The communication unit 110 is a device for communicating with another device (camera 2, etc.) via the network 9, and performs communication in accordance with a predetermined communication standard such as Ethernet (registered trademark) or wireless LAN. Includes equipment (such as network interface cards).

The input unit 120 is a device for inputting instructions and information according to a user's operation, and includes an arbitrary input device such as a keyboard, a mouse, a touch pad, and a touch panel.

The display unit 130 is a device that displays an image according to the image data input from the processing unit 160, and includes, for example, a liquid crystal display and an organic EL display.

The interface unit 140 is a device for inputting and outputting various data in the processing unit 160, and includes, for example, a general-purpose interface device such as USB, a reader / writer device for a recording medium (for example, a memory card), and the like.

The storage unit 150 stores a program executed by the processor of the processing unit 160, data temporarily stored in the process of the processing unit 160, data used for the processing of the processing unit 160, and the processing unit 160. The data obtained as a result of the processing of is stored. For example, the storage unit 150 stores the registration information 51 (FIG. 2), the camera information 52 (FIG. 3), and the history information 53 (FIG. 4), which will be described later.

The storage unit 150 includes, for example, a main storage device (ROM, RAM, etc.) and an auxiliary storage device (flash memory, hard disk, optical disk, etc.). The storage unit 150 may be composed of one storage device, or may be composed of a plurality of storage devices of one or more types. Each storage device constituting the storage unit 150 is connected to the processing unit 160 via a computer bus or other communication means.

The processing unit 160 controls the overall operation of the information processing device 1 and executes predetermined information processing. The processing unit 160 includes, for example, one or more processors (CPU, MPU, etc.) that execute processing according to the instruction code of one or more programs stored in the storage unit 150. The processing unit 160 operates as one or more computers by executing one or more programs of the storage unit 150 by one or more processors.

The processing unit 160 may include one or more dedicated hardware (ASIC, FPGA, etc.) configured to realize a specific function. In this case, the processing unit 160 may execute the processing related to the determination of the object on one or more computers described above, or may execute at least a part of the processing on dedicated hardware.

As a component that performs processing related to determination of an object, the processing unit 160 includes, for example, acquisition unit 161, determination unit 162, registration unit 163, history update unit 164, and camera management, as shown in FIG. Includes part 165.

The acquisition unit 161 acquires the feature information of the object generated based on the image captured by the camera 2 capable of photographing a specific object (person or the like). The feature information is information on the features (characteristics of a person's face, etc.) of the object extracted from the image including the object (image taken by the camera 2). In the present embodiment, as an example, it is assumed that feature information is generated in the camera 2 that captures an object. When the feature information is generated in the camera 2, the acquisition unit 161 may acquire the feature information directly from the camera 2 or indirectly via one or more server devices (cloud server or the like) connected to the network 9. Characteristic information may be acquired. The feature information acquired by the acquisition unit 161 may be the same as the feature information generated by the camera 2, or may be subjected to predetermined conversion by a server device or the like interposed between the camera 2 and the information processing device 1. Good.

The determination unit 162 includes the feature information acquired by the acquisition unit 161 (hereinafter, may be referred to as "acquired feature information") and the feature information included in the registration information 51 stored in the storage unit 150 (hereinafter, "" It may be described as "registered feature information"). The registered feature information included in the registration information 51 is feature information generated in advance for each object registered in the registration information 51. For example, the feature of the object is extracted from the image of the object photographed in advance. This is the information obtained by doing so. The determination unit 162 is an object (hereinafter, "object in the image") included in the image (image taken by the camera 2) from which the acquired feature information is extracted based on the collation result of the registered feature information and the acquired feature information. The object registered in the registration information 51 (hereinafter, may be described as "registered object") corresponding to "object") is determined.

FIG. 2 is a diagram showing an example of registration information 51 stored in the storage unit 150. The registration information 51 includes predetermined information about each registered object. In the example of FIG. 2, the object registered in the registration information 51 is a person. Reference numeral 510 in FIG. 2 represents a group of information registered for one person. The group of information 510 shown in the example of FIG. 2 includes a personal ID for identifying an individual person, a date of registration (registration date), personal information about the person (name, gender, date of birth, address, address). (Postal code, e-mail address), type assigned to a person (for example, member type), current status (whether registration is valid, etc.), person's face image data, extracted from the face image Includes feature information that indicates the features of a person's face. The registration information 51 includes a group of information 510 as shown in FIG. 2 for each of one or more registered persons.

Note that the discrimination unit 162 is related to these objects when executing a process of discriminating the registered objects corresponding to the objects in the image (hereinafter, may be referred to as “discrimination process”). We may check the postal code of the place. That is, the discriminating unit 162 inputs a postal code of a place related to the object in the image (for example, a postal code of the address of a person included in the image taken by the camera 2), and the input postal code and registration information. The postal code included in 51 (hereinafter, may be referred to as "registered postal code") is collated. In this case, the discrimination unit 162 discriminates the registered object corresponding to the object in the image based on the collation result for the feature information and the collation result for the zip code.

For example, when the discrimination unit 162 discriminates two or more candidates of the registered object corresponding to the object in the image in the collation result of the feature information (the result of collating the acquired feature information with the registered feature information). , The postal code of the place related to the object in the image (for example, the postal code of the address of the person included in the image taken by the camera 2) is input in the input unit 120. Then, the discrimination unit 162 collates the registered zip code included in the registration information 51 with the zip code input from the input unit 120 for the two or more candidates, and based on the result of this collation, the two or more candidates. From among, the registered object corresponding to the object in the image is determined. For example, the discrimination unit 162 corresponds to an object in the image in which the registration information 51 contains the same registered zip code as the zip code input by the input unit 120 from the two or more candidates. Determine as a registered object. As a result, even if the object cannot be discriminated only by collating the feature information, it is possible to accurately discriminate the object by using the collation result of the zip code.

The registration unit 163 performs a process of adding information on an object to be registered (for example, a group of information 510 shown in FIG. 2) to the registration information 51 of the storage unit 150. For example, the registration unit 163 displays a registration screen on the display unit 130 that prompts the input of information about the object. The registration unit 163 adds the information input by the input unit 120 to the registration information 51 according to the guidance on the registration screen. Further, the registration unit 163 adds the information of the object provided by another device capable of communicating in the communication unit 110 and the information of the object input from the recording medium or the like in the interface unit 140 to the registration information 51. May be good.

When the registered object corresponding to the object in the image is determined by the determination unit 162, the history update unit 164 stores the information regarding the shooting history of the registered object in the storage unit 150.

In the present embodiment, as an example, the camera information 52 including information about each camera 2 and the history information 53 regarding the shooting history of each object registered in the registration information 51 are stored in the storage unit 150. To do. The camera information 52 includes a postal code of a place where each camera 2 takes a picture (hereinafter, may be referred to as a “shooting place postal code”). The history information 53 includes the postal code of the shooting location of the camera 2 that has shot the object determined by the discrimination process. When the registered object corresponding to the object in the image is determined by the determination unit 162, the history update unit 164 sets the shooting location mail included in the camera information 52 for the camera 2 that has taken the determined object. The number is added to the history information 53 as a postal code related to the shooting history of the determined object. Since the postal code of the shooting location of the camera 2 is included in the history information 53 as the information regarding the shooting history of the registered object, the geographical and temporal movement pattern of each object registered in the registration information 51 (target). When an object is a person, the behavior pattern of the person) can be grasped.

FIG. 3 is a diagram showing an example of camera information 52 stored in the storage unit 150. Reference numeral 520 in FIG. 3 represents a group of information about one camera 2. The group of information 520 shown in the example of FIG. 3 includes a camera ID for identifying each camera 2, a name given to each camera 2, a model of the camera 2, and a function provided by the camera 2 (for example, a gender determination function, etc.). Age judgment function, facial expression judgment function, etc.), camera 2 address on network 9 (for example, IP address), current status (whether or not it is connected to network 9), address of the place where shooting is performed, mail of the address. Includes number (shooting location postal code). The camera information 52 includes a group of information 520 as shown in FIG. 3 for each of the one or more cameras 2 included in the camera system.

FIG. 4 is a diagram showing an example of the history information 53 stored in the storage unit 150. In the history information 53 shown in FIG. 4, the object for which the shooting history is recorded is a person (same as the registration information 51 shown in FIG. 2). Reference numeral 530 in FIG. 4 indicates a group of information recorded when one person is discriminated by the discriminating process. The group of information 530 shown in the example of FIG. 4 includes the personal ID of the person who took the picture, the date and time when the picture was taken, the camera ID of the camera 2 which took the picture, and the postal code of the address of the place where the picture was taken. The personal ID in the history information 53 is the same as the personal ID in the registration information 51, and the camera ID in the history information 53 is the same as the camera ID in the camera information 52.

Each time the object is determined by the determination process, the history update unit 164 sets the information (personal ID, etc.) included in the registration information 51 for the determined object and the camera 2 for photographing the determined object. Based on the information (camera ID, shooting location postal code, etc.) included in the information 52, a group of information 530 as shown in FIG. 4 is generated, and this is added to the history information 53.

The camera management unit 165 performs processing related to the management of the camera 2 included in the camera system. For example, when a new camera 2 is added to the camera system, the camera management unit 165 displays a camera addition screen prompting the input of information of the newly added camera 2 on the display unit 130. The camera management unit 165 adds the information input by the input unit 120 to the camera information 52 shown in FIG. 3 according to the guidance on the camera addition screen.

Further, when searching for a camera 2 satisfying a specific condition, the camera management unit 165 displays a camera search screen prompting the input of the search condition of the camera 2 on the display unit 130. When the search condition is input by the input unit 120 according to the guidance on the camera search screen, the camera management unit 165 extracts the information of the camera 2 that satisfies the search condition in the camera information 52 (FIG. 3), and the extracted camera. The information of 2 is displayed on the display unit 130. For example, when the search condition is a specific zip code, the camera management unit 165 extracts the information of the camera 2 including the same shooting location zip code as the specific zip code in the camera information 52, and displays the information on the display unit 130. indicate. As a result, the information of the camera 2 installed in the area of the specific zip code can be easily grasped.

[Camera 2]
FIG. 5 is a diagram showing an example of the configuration of the camera 2. The camera 2 shown in FIG. 5 has an imaging unit 220, a communication unit 210, an interface unit 230, a storage unit 240, and a processing unit 250.

The communication unit 210 is a device for communicating with another device (information processing device 1 or the like) via the network 9, and communicates in accordance with a predetermined communication standard such as Ethernet (registered trademark) or wireless LAN. Includes devices that perform (such as network interface cards).

The image pickup unit 220 is a device that captures an image, and includes an image pickup element such as a CMOS image sensor, an optical system that guides the light of a subject to the image pickup element, an image processing device that processes an image captured by the image pickup element, and the like.

The interface unit 230 is a device for inputting and outputting various data in the processing unit 250, and includes, for example, a general-purpose interface device such as USB, a reader / writer device for a recording medium (memory card, etc.), and the like.

The storage unit 240 stores a program executed by the processor of the processing unit 250, data temporarily stored in the process of the processing unit 250, data used for the processing of the processing unit 250, and the processing unit 250. The data obtained as a result of the processing of is stored. The storage unit 240 includes, for example, a main storage device (ROM, RAM, etc.) and an auxiliary storage device (flash memory, hard disk, optical disk, etc.). The storage unit 240 may be composed of one storage device, or may be composed of a plurality of storage devices of one or more types.

The processing unit 250 is a device that comprehensively controls the overall operation of the camera 2. For example, one or more processors (CPU,) that execute processing according to the instruction code of one or more programs stored in the storage unit 240. MPU etc.) is included. The processing unit 250 operates as one or more computers by executing one or more programs of the storage unit 240 by one or more processors. The processing unit 250 may include one or more dedicated hardware (ASIC, FPGA, etc.) configured to realize a specific function. The processing unit 250 may execute all the processing on one or more computers, or may execute at least a part of the processing on dedicated hardware.

The processing unit 250 captures an image (still image or moving image) in the imaging unit 220, and stores the captured data 31 in the storage unit 240. Further, the processing unit 250 extracts the feature information 32 related to the feature of the specific object included in the image from the image captured by the imaging unit 220 and stores it in the storage unit 240. When the object is a person and the features of the face of the person are extracted as the feature information 32, for example, the processing unit 250 may use the feature points such as the eyes, nose, mouth, face, and contour of the person in the image captured by the imaging unit 220. Feature information 32 is generated based on. The feature information 32 may be, for example, a multidimensional vector. In this case, the discrimination unit 162 calculates the degree of matching of the facial features indicated by these feature information based on the correlation coefficient between the vector indicated by the acquired feature information and the vector indicated by the registered feature information. By comparing the calculated degree of matching with a predetermined threshold value, it may be determined whether or not the person in the image and the registered person match.

The processing unit 250 communicates with the information processing device 1 by the communication unit 210, and appropriately transmits the feature information 32 stored in the storage unit 240 to the information processing device 1. The processing unit 250 may transmit information regarding the shooting date and time of the image in which the feature information 32 is generated to the information processing device 1 together with the feature information 32.

Here, the operation of the information processing apparatus 1 according to the first embodiment having the above-described configuration will be described.
FIG. 6 is a flowchart for explaining an example of processing in the information processing apparatus 1 according to the first embodiment. Each time the information processing device 1 acquires the feature information from the camera 2, the information processing device 1 executes the process shown in the flowchart of FIG. In the example of FIG. 6, the object determined by the information processing device 1 is a person. In the following, the person registered in the registration information 51 will be referred to as a "registrant", and the person in the image captured by the camera 2 will be referred to as a "photographed person".

When the acquisition unit 161 acquires the feature information (acquired feature information) of the photographed person from the camera 2 (ST100), the determination unit 162 includes the feature information (registered feature information) of the registrant included in the registration information 51 and the photographed person. It is collated with the acquired feature information of the person (ST105). As a result of collating the acquired feature information with the registered feature information, if the registrant corresponding to the photographed person is not found (No of ST110), the processing unit 160 ends the discrimination process.

When only one registrant corresponding to the photographed person is found by the collation in step ST105 (Yes of ST110 and No of ST115), the discriminating unit 162 corresponds to this one registrant as the photographed person. It is confirmed as the determination result of the registrant (ST135). The information processing device 1 uses this determination result for arbitrary processing (eg, a process of notifying a predetermined destination of a photograph taken by a predetermined registrant by a predetermined notification means such as an e-mail).

When the determination result of the registrant corresponding to the photographed person is confirmed (ST135), the history updating unit 164 adds the information regarding the photographing history of the registrant to the history information 53 (FIG. 4) (ST140). That is, the history update unit 164 includes information (individual ID) included in the registration information 51 for the determined registrant and information (camera ID, shooting) included in the camera information 52 for the camera 2 that captured the determined registrant. Based on the location postal code, etc.), a group of information 530 shown in FIG. 4 is generated, and this is added to the history information 53.

When two or more registrant candidates corresponding to the photographed person are found by the collation in step ST105 (Yes in ST110 and Yes in ST115), the determination unit 162 inputs the postal code of the address of the photographed person. Input at 120 (ST120). For example, when the user who operates the information processing device 1 is in charge of the counter business and directly faces the photographed person who visits the window, the user verbally gives the postal code of the photographed person's address from the photographed person. You may listen to it and input it to the information processing device 1 by the input unit 120.

When the postal code is input in the input unit 120 (ST120), the discriminating unit 162 collates the registered postal code (FIG. 2) included in the registration information 51 for two or more candidates with the postal code input in the input unit 120. (ST125). As a result of collating the zip code entered in the input unit 120 with the registered zip code, the registrant whose registration information 51 includes the same registered zip code as the entered zip code among two or more candidates (that is, If there is only one registrant (registrant living in the area with the same registered zip code as the entered zip code) (Yes in ST130), the discriminating unit 162 corresponds to that one registrant as the photographed person. It is confirmed as the determination result of the registrant (ST135). The history update unit 164 adds information regarding the shooting history of the confirmed registrant to the history information 53 (FIG. 4) (ST140). On the other hand, as a result of collating the postal code in step ST125, if there is no registrant living in the area of the same registered postal code as the postal code input in the input unit 120, or if there are two or more (ST130). No), the processing unit 160 ends the discrimination process.

As described above, according to the present embodiment, the collation result for the feature information (the collation result between the acquired feature information and the registered feature information) and the collation result for the zip code (the place related to the object in the image). Based on the collation result of the postal code and the registered postal code of the place related to the registered object), the registered object corresponding to the object in the image is determined. For example, when two or more registered object candidates corresponding to the objects in the image are determined in the collation result between the acquired feature information and the registered feature information, the two or more candidates are included in the registration information 51. The registered zip code and the zip code entered in the input unit 120 (the zip code of the place related to the object in the image) are collated, and based on the result of this collation, from the two or more candidates, The registered object corresponding to the object in the image is determined. Therefore, even if the accuracy of discriminating the object based on the collation result of the feature information is lowered due to the shooting conditions of the camera 2 or the similarity of the appearance of the object, the collation result of the postal code is used for the discrimination process. By doing so, it is possible to effectively suppress a decrease in discrimination accuracy.

Further, according to the present embodiment, when the registered object corresponding to the object in the image is determined by the determination unit 162, the camera information 52 is obtained for the camera 2 that has captured the image including the determined object. The shooting location postal code included in is added to the history information 53 as a postal code related to the shooting history of the determined object. Therefore, the geographical and temporal movement patterns of each object registered in the registration information 51 (when the object is a person, the behavior pattern of the person) can be effectively grasped.

Further, according to the present embodiment, since the postal code of the place where each camera 2 takes a picture is included in the camera information 52 (FIG. 3), it becomes easy to manage the camera 2 for each district represented by the postal code. For example, the camera 2 can be easily set for each district. In particular, zip codes are used almost all over the world, so by using zip codes to manage cameras 2 in each district, even if cameras 2 are installed in various countries around the world, in each country. The camera 2 can be easily managed for each district.

Further, according to the present embodiment, the feature information of the object generated by each camera 2 is provided to the information processing device 1, and the image of the object captured by the camera 2 stays inside the camera 2, so that the information processing is performed. It is possible to effectively prevent the image of the object from being leaked to others other than the user of the device 1.

<Second embodiment>
Next, the information processing device 1 according to the second embodiment will be described. The configuration of the information processing device 1 according to the second embodiment is substantially the same as that of the information processing device 1 according to the first embodiment, but the operation of a part of the processing unit 160 is related to the first embodiment. It is different from the information processing device 1. That is, in the information processing apparatus 1 according to the second embodiment, when the registered object corresponding to the object in the image cannot be determined, the object in the image is automatically registered as a new object. To do. For example, by automatically registering an unspecified person photographed by the camera 2 as an object, it is possible to utilize the discrimination result of the object (unspecified person) photographed by each camera 2 for marketing or the like. It becomes. Hereinafter, the differences from the information processing apparatus 1 according to the first embodiment (particularly, the differences in the operations of the discrimination unit 162 and the registration unit 163) will be mainly described.

FIG. 7 is a diagram showing an example of registration information 51A stored in the storage unit 150 in the information processing device 1 according to the second embodiment. In the example of FIG. 7, the object registered in the registration information 51A is a person whose name or the like is unknown (unspecified person). Reference numeral 510A in FIG. 7 represents a group of information registered for one person. The group of information 510A shown in the example of FIG. 7 includes a personal ID for identifying an individual person, a date of registration (registration date), an estimated gender, an estimated age, a zip code of the place where the image was taken, and a photographed place. Includes personal feature information. The registration information 51A includes a group of information 510A as shown in FIG. 7 for each of one or more registered persons.

The "estimated gender" and "estimated age" included in the group of information 510A (FIG. 7) are estimated by the camera 2, respectively. That is, the processing unit 250 of the camera 2 estimates the gender and age of the person based on the image of the person included in the image captured by the image pickup unit 220 (for example, based on the facial features in the image). The estimation results of gender and age are provided from the camera 2 to the information processing device 1 together with the characteristic information of the person. The acquisition unit 161 of the information processing device 1 acquires the estimation results of gender and age together with the feature information.

When the registered object corresponding to the object in the image cannot be discriminated by the discriminating unit 162, the registration unit 163 newly registers the acquired feature information acquired by the acquisition unit 161 as the feature of the object. It is added to the registration information 51A as information, and the shooting location postal code included in the camera information 52 is added to the registration information 51A for the camera 2 that has taken an image including the indistinguishable object. For example, when the registered object (person) corresponding to the object (person) in the image cannot be discriminated by the discriminating unit 162, the registration unit 163 obtains "feature information" and "feature information" in the acquisition unit 161. A group of information 510A (FIG. 7) including "estimated gender" and "estimated age" is generated and added to the registration information 51A.

Similar to the first embodiment described above, the determination unit 162 has a collation result for the feature information (a collation result between the acquired feature information and the registered feature information) and a collation result for the zip code (for the object in the image). The discrimination process is performed based on the collation result of the postal code of the related place and the registered postal code of the place related to the registered object). However, when the determination unit 162 in the second embodiment inputs the zip code of the place related to the object in the image, the zip code may be input from the input unit 120 as in the first embodiment. Instead, the shooting location zip code (postal code of the place where the shooting was performed by the camera 2) included in the camera information 52 is input for the camera 2 that generated the feature information. As a result, the user does not have to operate the input unit 120 to input the zip code, and the zip code of the place related to the photographed object can be automatically input from the camera information 52.

For example, when the discrimination unit 162 discriminates two or more registered object candidates corresponding to the objects in the image in the collation result for the feature information, the mail included in the registration information 51A for the two or more candidates. The number (registered zip code) is collated with the zip code (shooting location zip code) included in the camera information 52 for the camera 2 that generated the feature information. The discrimination unit 162 discriminates the registered object corresponding to the object in the image from two or more candidates based on the result of collating the registered zip code with the shooting location zip code. For example, the determination unit 162 is an object in which the registration information 51A contains the same registered zip code as the shooting location zip code included in the camera information 52 for the camera 2 that generated the feature information from two or more candidates. Is determined as a registered object corresponding to the object in the image. As a result, even if the object cannot be completely determined by the collation of the feature information, the object can be accurately discriminated by using the collation result of the registered zip code and the shooting location zip code.

Further, in the collation result between the registered zip code and the shooting location zip code, the discriminating unit 162 uses the registered zip code included in the registration information 51A for two or more candidates as the camera information 52 for the camera 2 that generated the feature information. If it does not match the included shooting location zip code, the zip code included in the history information 53 (FIG. 4) (hereinafter, may be referred to as "history zip code") and feature information are generated for two or more candidates. The camera 2 is further collated with the shooting location postal code included in the camera information 52. For example, the discriminating unit 162 is a target in which the history information 53 includes the same history zip code as the shooting location zip code included in the camera information 52 for the camera 2 that generated the feature information from among two or more candidates. The object is discriminated as a registered object corresponding to the object in the image.

When the object can be photographed by a plurality of cameras 2, the object registered in the registration information 51A may be photographed by the cameras 2 in a place different from the area of the registered zip code included in the registration information 51A. In this case, since the registered zip code included in the registration information 51A and the shooting location zip code included in the camera information 52 are different, the object cannot be correctly determined from the collation result of these zip codes. However, since the history information 53 includes the history postal code of the place where the object was photographed in the past, the history postal code included in the history information 53 may match the shooting place postal code included in the camera information 52. There is sex. Therefore, even if the object cannot be determined based on the collation result between the registered zip code included in the registration information 51A and the shooting location zip code included in the camera information 52, the history zip code and the camera information 52 included in the history information 53 By further using the collation result with the postal code of the shooting place included in the above for the discriminating process, it is possible to accurately discriminate an object (for example, an unspecified person) that can be photographed by a plurality of cameras 2.

Here, the operation of the information processing apparatus 1 according to the second embodiment having the above-described configuration will be described.
8 and 9 are flowcharts for explaining an example of processing in the information processing apparatus 1 according to the second embodiment. Each time the information processing device 1 acquires feature information from the camera 2, the information processing device 1 executes the processes shown in these flowcharts. Similar to FIG. 6, the object determined by the information processing device 1 is a person, the person registered in the registration information 51A is the "registrant", and the person in the image taken by the camera 2 is the "photographed person". Write.

When the acquisition unit 161 acquires the feature information (acquired feature information) of the photographed person from the camera 2 (ST200), the determination unit 162 includes the feature information (registered feature information) of the registrant included in the registration information 51A and the photographed person. It is collated with the acquired feature information of the person (ST205).

As a result of collating the acquired feature information with the registered feature information, if the registrant corresponding to the photographed person in the image in which the feature information is generated cannot be discriminated by the discriminating unit 162 (No of ST210), the registration unit 163 Adds the acquired feature information acquired by the acquisition unit 161 to the registration information 51A as the feature information of the newly registered registrant. Further, the registration unit 163 adds the postal code included in the camera information 52 for the camera 2 that photographed the photographed person who could not be discriminated to the registration information 51A of the newly registered registrant (ST215). For example, when the registrant corresponding to the photographed person in the image cannot be discriminated by the discriminating unit 162, the registration unit 163 determines the "feature information", "estimated gender", and "estimated age" acquired by the acquisition unit 161. A group of information 510A (FIG. 7) including the shooting location postal code included in the camera information 52 is generated for the camera 2 that has taken a picture of the photographed person who could not be discriminated, and this is added to the registration information 51A.

When only one registrant corresponding to the photographed person is found by the collation in step ST205 (Yes of ST210 and No of ST220), the discriminating unit 162 corresponds to this one registrant as the photographed person. It is confirmed as the determination result of the registrant (ST255). The information processing device 1 uses this determination result for arbitrary processing.

When the determination result of the registrant corresponding to the photographed person is confirmed (ST255), the history updating unit 164 adds the information regarding the photographing history of the registrant to the history information 53 (FIG. 4) (ST260). That is, the history update unit 164 includes information (personal ID) included in the registration information 51A for the determined registrant and information (camera) included in the camera information 52 for the camera 2 that captured the image including the determined registrant. Based on the ID, the postal code of the shooting location, etc.), a group of information 530 shown in FIG. 4 is generated, and this is added to the history information 53.

When two or more registrant candidates corresponding to the photographed person are found by the collation in step ST205 (Yes in ST210 and Yes in ST220), the discriminating unit 162 is included in the registration information 51A for two or more candidates. The registered zip code is collated with the shooting location zip code included in the camera information 52 for the camera 2 that generated the feature information (ST225). For example, the determination unit 162 searches for a registrant whose registration information 51A contains the same registered zip code as the shooting location zip code from two or more candidates.

As a result of collating the postal code in step ST225, when there is only one registrant whose registration information 51A contains the same registered postal code as the shooting location postal code among two or more candidates (Yes, in ST230). And, Yes) of ST235, the determination unit 162 determines the one registrant as the determination result of the registrant corresponding to the photographed person (ST255). The history update unit 164 adds information regarding the shooting history of the confirmed registrant to the history information 53 (FIG. 4) (ST260).

On the other hand, as a result of collating the postal code in step ST225, if there are two or more registrants whose registration information 51A contains the same registered postal code as the shooting location postal code, or if there are two or more candidates. When the registrant does not exist (No of ST230 or No of ST235), the discriminating unit 162 uses the history postal code included in the history information 53 (FIG. 4) for two or more candidates and the camera 2 that generates the feature information. Is collated with the shooting location postal code included in the camera information 52 (ST240). For example, the determination unit 162 searches for a registrant whose history information 53 includes a history zip code that is the same as the shooting location zip code from among two or more candidates.

As a result of collation in step ST240, when only one registrant whose history information 53 contains the same history zip code as the shooting location zip code is found among two or more candidates (Yes of ST245 and Yes). , ST250 Yes), the determination unit 162 determines the one registrant as the determination result of the registrant corresponding to the photographed person (ST255). The history update unit 164 adds information regarding the shooting history of the confirmed registrant to the history information 53 (FIG. 4) (ST260). On the other hand, as a result of the collation in step ST240, when there are two or more registrants whose history information 53 contains the same history zip code as the shooting location zip code, or the registration thereof. When there is no person (No of ST245 or No of ST250), the processing unit 160 ends the discrimination process without determining the registrant corresponding to the photographed person.

As described above, according to the present embodiment, when the registered object corresponding to the object in the image cannot be discriminated by the discriminating unit 162, the acquired feature information acquired by the acquiring unit 161 is obtained. It is added to the registration information 51A as the feature information of the newly registered object. Further, in this case, the postal code included in the camera information 52 for the camera 2 that has captured the image including the indistinguishable object is added to the registration information 51A. As a result, the feature information of the object included in the image taken by the camera 2 and the postal code of the place where the object was taken can be automatically registered in the registration information 51A. For example, as information on an unspecified person photographed by the camera 2, feature information indicating the facial features of the person and the postal code of the place where the person was photographed are automatically registered in the registration information 51A. It can be used for marketing.

Further, according to the present embodiment, when two or more registered object candidates corresponding to the objects in the image are determined in the collation result for the feature information, the two or more candidates are included in the registration information 51A. The registered zip code is collated with the shooting location zip code included in the camera information 52 for the camera 2 that generated the feature information, and based on this collation result, the registered zip code corresponding to the object in the image is registered. The target object is determined. In this way, by using the collation result between the registered zip code and the shooting location zip code for the discrimination process, it is possible to effectively suppress a decrease in the discrimination accuracy of the object.

Further, according to the present embodiment, as a result of collating the registered zip code with the imaging location zip code, if neither of the registered zip codes of the two or more candidates matches the imaging location zip code, the two or more candidates The historical postal code included in the history information 53 is further collated with the shooting location postal code included in the camera information 52 for the camera 2 that generated the feature information, and based on this collation result, it corresponds to the object in the image. The registered object to be registered is determined. Therefore, it is possible to accurately discriminate an object that cannot be discriminated by the collation result of the registered zip code and the imaging location zip code based on the collation result of the history zip code and the photographing location zip code.

The present invention is not limited to the above-described embodiment, but includes various variations.

For example, in the above-described embodiment, the facial features of a person included in the image captured by the camera 2 are extracted as feature information, but the features of any body part not limited to the face are extracted as feature information. Alternatively, the characteristics of the entire body may be extracted as characteristic information.

Further, in the above-described embodiment, an example of discriminating a person by a discriminating process is given, but the object of the discriminating process may be a creature other than a human being or an object (non-living body). For example, an arbitrary moving object such as a vehicle may be an object of discrimination processing.

Further, in the above-described embodiment, an example of generating feature information of an object in the camera 2 is given, but the present invention is not limited to this example. In another embodiment of the present invention, in a device other than the camera 2 (for example, an information processing device 1 or an arbitrary server device connected to the network 9), feature information of an object is obtained based on an image taken by the camera 2. It may be generated.

Below, the description of the scope of claims attached first to the application of the original application is added. The claims listed below are the scope of the claims originally attached to the application of the original application.
<Claim 1>
Feature information generated based on an image taken by one or more cameras capable of photographing a specific object, and the feature information regarding the feature of the object extracted from the image including the object is acquired. Acquisition department and
A storage unit that stores registration information including the feature information regarding one or more registered features of the object, and a storage unit.
The target in the image in which the feature information is generated by collating the feature information acquired by the acquisition unit with the feature information included in the registration information and based on the collation result of the feature information. It has a discriminating unit for discriminating the registered object corresponding to the object, and has a discriminating unit.
The registration information includes a registered zip code, which is the zip code of the place associated with the registered object.
The discriminating unit inputs the zip code of the place related to the object in the image, collates the registered zip code included in the registration information with the input zip code, and collates the feature information. Based on the result and the collation result for the zip code, the registered object corresponding to the object in the image is determined.
Information processing device.
<Claim 2>
It has an input unit for inputting information according to the user's operation.
When the discriminating unit discriminates two or more candidates for the registered object corresponding to the object in the image in the collation result for the feature information, the postal code of the place related to the object in the image. The number is input in the input unit, the registered zip code included in the registration information is collated with the entered zip code for the two or more candidates, and the two or more candidates are collated based on the result of the collation. The registered object corresponding to the object in the image is determined from the above.
The information processing device according to claim 1.
<Claim 3>
The storage unit
Camera information including the shooting location zip code, which is the zip code of the place where each of the plurality of cameras shoots,
The history information and the history information about the shooting history of the registered object are memorized.
When the registered object corresponding to the object in the image is discriminated by the discriminating unit, the shooting location included in the camera information for the camera that has taken the image including the discriminated object. It has a history update unit that adds a zip code to the history information as a zip code related to the shooting history of the determined object.
The information processing device according to claim 1 or 2.
<Claim 4>
The storage unit stores camera information including a shooting location zip code, which is a zip code of a place where each of the plurality of cameras shoots.
When the registered object corresponding to the object in the image cannot be discriminated by the discriminating unit, the feature information acquired by the acquisition unit is newly registered as the feature information of the object. The shooting location postal code included in the camera information for the camera that captured the image including the indistinguishable object was related to the newly registered object. It has a registration unit that is added to the registration information as the registration zip code of the place.
The discriminating unit inputs the shooting location zip code included in the camera information for the camera that captured the image for which the feature information is generated as the zip code of the location related to the object in the image.
The information processing device according to claim 1.
<Claim 5>
When the discriminating unit discriminates two or more candidates for the registered object corresponding to the object in the image in the collation result for the feature information, the two or more candidates are included in the registration information. The registered zip code is collated with the shooting location zip code included in the camera information for the camera that captured the image for which the feature information was generated, and based on the result of the collation, the above two or more. The registered object corresponding to the object in the image is determined from the candidates of.
The information processing device according to claim 4.
<Claim 6>
The storage unit stores history information regarding the shooting history of the registered object, and stores the history information.
When the registered object corresponding to the object in the image is discriminated by the discriminating unit, the shooting location included in the camera information for the camera that has taken the image including the discriminated object. It has a history update unit that adds a zip code to the history information as a zip code related to the shooting history of the determined object.
The information processing device according to claim 5.
<Claim 7>
In the collation result of the registered zip code and the shooting location zip code, the discriminating unit uses the registered zip code included in the registration information for the two or more candidates to display the image from which the feature information is generated. If the photographed camera does not match the shooting location zip code included in the camera information, the historical zip code, which is the zip code included in the history information, and the feature information are generated for the two or more candidates. The camera that captured the image is collated with the shooting location zip code included in the camera information, and based on the collation result between the history zip code and the shooting location zip code, the two or more candidates From the inside, the registered object corresponding to the object in the image is determined.
The information processing device according to claim 6.
<Claim 8>
Characteristic information generated based on an image taken by one or more cameras capable of photographing a specific object, and the feature information regarding the feature of the object extracted from the image including the object is computerized. It is an information processing method that is processed by
The storage unit of the computer stores registration information including the feature information regarding one or more registered features of the object.
The acquisition process for acquiring the feature information and
The target in the image in which the feature information is generated by collating the feature information acquired in the acquisition step with the feature information included in the registration information and based on the collation result of the feature information. It has a discrimination step of discriminating the registered object corresponding to the object.
The registration information includes a registered zip code, which is the zip code of the place associated with the registered object.
In the determination step, the postal code of the place related to the object in the image is input, the registered postal code included in the registration information is collated with the input postal code, and the feature information is collated. Including determining the registered object corresponding to the object in the image based on the result and the collation result for the zip code.
Information processing method.
<Claim 9>
A program for causing a computer to execute the information processing method according to claim 8.
<Claim 10>
A camera system including the information processing device according to any one of claims 1 to 7 and a plurality of the cameras.

1 ... Information processing device, 110 ... Communication unit, 120 ... Input unit, 130 ... Display unit, 140 ... Interface unit, 150 ... Storage unit, 160 ... Processing unit, 161 ... Acquisition unit, 162 ... Discrimination unit, 163 ... Registration unit 164 ... History update unit, 165 ... Camera management unit, 51, 51A ... Registration information, 52 ... Camera information, 53 ... History information, 2 ... Camera, 210 ... Communication unit, 220 ... Imaging unit, 230 ... Interface unit, 240 ... storage unit, 250 ... processing unit, 9 ... network

Claims (1)

Feature information generated based on an image taken by one or more cameras capable of photographing a specific object, and the feature information regarding the feature of the object extracted from the image including the object is acquired. Acquisition department and
A storage unit that stores registration information including the feature information regarding one or more registered features of the object, and a storage unit.
The target in the image in which the feature information is generated by collating the feature information acquired by the acquisition unit with the feature information included in the registration information and based on the collation result of the feature information. It has a discriminating unit for discriminating the registered object corresponding to the object, and has a discriminating unit.
The storage unit stores camera information including the postal code of the place where the plurality of cameras take pictures.
Information processing device.

Images