JP2007102342A - Automatic counter - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately count the number of visitors where there is an unspecified majority of visitors, at an event site or the like. <P>SOLUTION: A plurality of images showing scenes at different time points are obtained by repeatedly photographing an area where there are visitors from the same direction. The frontal facial information of visitors facing front is extracted from each image, and objects of counting are discriminated, based on the extracted frontal facial information and counted (S104 and S105). Before the discrimination of the counting objects, the information which has not been detected in the prior processing is discriminated from the extracted frontal facial information (S102). Then, discrimination of the counting objects is executed only for the frontal facial information which is detected for the first time (S103). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an automatic counting device used for a number survey in a place where a large number of people gather.

There is known a technique for identifying attributes such as individual or sex and age from face information of a subject included in an image and analyzing a tendency for each person or each attribute. For example, in Patent Documents 1 and 2, customers in a store are photographed by cameras installed in various places in the store, and individual customers are identified based on facial features extracted from each captured image, and their footsteps are tracked. And a method and system for identifying a customer's gender and age from a facial feature amount and examining a customer's tendency to stay at a predetermined place in a store for a long time.
JP 2000-200377 A JP 2004-348618 A

  In general, when identifying an individual or an attribute using face information included in an image, front face information is often used. This is because using the front face increases the amount of information and thus increases the identification accuracy. For this reason, in a system for identifying a face, a camera is usually installed at a location where all persons pass without fail according to the direction in which the person passes, and an image obtained by photographing the person from the front is acquired. For example, the systems of Patent Documents 1 and 2 also photograph the front face of a customer entering the store by installing a camera facing outside near the entrance.

  However, for example, in a special venue provided on a part of the floor of a building, there are no clear entrances and exits, so visitors enter from all directions. In addition, there are cases where the visitors who are entering are facing their own direction. In such an environment, even if the camera is installed at a specific location, the front face of the visitors who are not facing the camera cannot be photographed.

  The present invention is an automatic counting device capable of acquiring front face information of a large number of visitors even in the above-described environment and accurately measuring the number of visitors using the acquired information. The purpose is to provide.

  The present invention provides an automatic counting device that automatically counts objects present at a predetermined location. This apparatus includes an extraction unit that sequentially performs a process of extracting front face information of a target included in an image with respect to a plurality of images obtained by repeatedly capturing scenes at different times at predetermined locations from the same direction. “Front face information” is a set of a plurality of feature amounts indicating features of a face facing the front. The feature amount extracted from the target is preferably a feature amount necessary for identifying the target individual.

  The apparatus further compares the front face information extracted from the one image by the extracting means with the front face information extracted from the image processed before the one image by the extracting means. First identification means for identifying an object detected for the first time in an image is provided. Moreover, the 2nd identification means which identifies the counting object based on the front face information of the object identified by the 1st identification means, and the counting means which counts the number of the counting objects identified by the 2nd identification means are provided.

  With the above configuration, the front face information of the majority of visitors can be acquired even in an environment where the orientation of the visitors is not constant, and the number of visitors can be accurately measured using the acquired front face information. it can.

  The second identification unit is a unit that can identify the attribute and / or state of the counting target included in each image based on the front face information, and the counting unit sets the number of the counting target to the attribute and / or It is preferable to use a means that can count for each state. Thereby, it becomes possible to check the number of objects by attributes such as sex and age, or by states such as a smile and an angry face.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. The system exemplified below is a facility management system for a multi-purpose hall (hereinafter referred to as a hall). Specifically, this is a system in which the function of the apparatus of the present invention is incorporated in a server computer that controls cameras and lighting installed in the hall. In the following description, the server computer constituting the system corresponds to the apparatus of the present invention.

[System configuration]
FIG. 1 shows a schematic configuration of the system. As shown in the figure, this system includes a plurality of photographing units 1 ₁ to 1 _n , a server computer 2, a storage 3, a setting terminal 4 and a DVD recorder 5. The photographing units 1 ₁ to 1 _n are installed at predetermined positions on the wall or ceiling of the hall. The server computer 2, the storage 3, the setting terminal 4 and the DVD recorder 5 are installed in a management room adjacent to the hall.

The photographing units 1 ₁ to 1 _n and the server computer 2 are connected by a wireless network, and the storage 3, the setting terminal 4 and the DVD recorder 5 are directly connected to the server computer 2. However, the connection between apparatuses is not limited to the illustrated form. For example, the photographing units 1 ₁ to 1 _n and the server computer 2 may be connected via a wired network, and the storage 3 may be a network storage connected to the server computer 2 via the network.

Figure 2 shows an internal configuration of the imaging unit 1 _1. Configuration of the imaging unit 1 ₂ to 1 _n is the same as the imaging unit _{1 1.} Photographing unit _{1 1} includes an imaging unit 11, encoding unit 12, feature extraction unit 13, a communication control unit 14 and the setting unit 15.

  The imaging unit 11 is a video camera having a rotatable structure and a clock function. The rotatable structure is, for example, a structure such as a dome-type surveillance camera or a structure in which a camera is attached to a turntable. The rotation angle of the video camera, the shooting magnification, the switching between moving image shooting / still image shooting, and the shooting interval during still image shooting can be changed by setting. Setting information C <b> 1 related to shooting is supplied from the setting unit 15. The image information acquired by shooting is sent to the encoding unit 12 together with information of the shooting date / time or the shooting start date / time acquired by the clock function.

  The encoding unit 12 encodes the image (moving image M or still image I) sent from the imaging unit 11 and records it in the memory in the imaging unit. The image encoding method is set for each photographing unit. In the present embodiment, the moving image is encoded in the MPEG4 format, and the still image is encoded by an encoding method selected by setting from JPEG or TIFF. Setting information C2 related to the encoding method is supplied from the setting unit 15. The encoding unit 12 assigns various types of management information when recording an image in a memory. In the present embodiment, as shown in FIG. 3, the shooting unit identifier UID, the setting information C1 and C2, the shooting date and time DT are added to the image. Hereinafter, an image to which such management information is assigned is referred to as image information 6.

  The feature extraction unit 13 sequentially reads out the image information 6 recorded in the memory and detects a human face from the image. Any known technique may be used as a technique for detecting a face from an image. However, in the present embodiment, since a large number of faces need to be detected from one image, for example, a skin color area of a predetermined size is used. It is preferable to employ a detection method that is as simple as possible, such as a method for determining a face.

  Next, the feature extraction unit 13 assigns a temporary visitor identifier GID to the face detected from each image. The temporary visitor identifier GID is, for example, an identifier obtained by adding a number indicating the position where the face of the visitor is detected after the shooting unit identifier UID. The number indicating the position where the face is detected may be a number that can roughly specify the position of each face. For example, the image is divided into a grid and the position of each grid is represented by a row number and a column number, and the face is detected. A method using the matrix number of the lattice closest to the position can be considered. Since the temporary visitor identifier GID is assigned for each image, different identifiers are assigned to the same person for each image at this time.

  Subsequently, the feature extraction unit 13 extracts a plurality of feature amounts F1 to FN representing facial features for each detected person. Various techniques are known for identifying a face based on feature amount extraction, and any of these techniques can be employed in this embodiment. However, while the trend analysis of visitors requires high-speed processing of many faces, high accuracy is not required, so it is preferable to select a method with high processing efficiency even if the accuracy is somewhat low.

  The type and number of feature amounts extracted by the feature extraction unit 13 are set for each photographing unit according to the purpose of analysis. Also, the function of the feature extraction unit 13 can be turned off by setting. That is, it is possible to set so as not to perform feature extraction. The setting information C3 regarding the feature extraction is supplied from the setting unit 15.

  The feature extraction unit 13 records the extracted feature amounts F1 to FN in a memory with various management information. In the present embodiment, as shown in FIG. 5, the extracted feature amounts F1 to FN include the shooting unit identifier UID, the setting information C1 and C2 and the shooting date and time information DT included in the image information 6, and The setting information C3 supplied to the feature extraction unit and the temporary visitor identifier GID described above are given as management information. Hereinafter, the feature amounts F1 to FN to which the management information is attached are referred to as visitor information 7.

The communication control unit 14 controls communication between the photographing units 1 ₁ to 1 _n and the server computer 2. Specifically, three types of communication are controlled. First, the communication control unit 14 receives the setting information C <b> 1 to C <b> 3 sent from the server computer 2 to the photographing unit and supplies it to the setting unit 15. Secondly, the communication control unit 14 transmits the image information 6 generated by the encoding unit 12 to the server computer 2. Thirdly, the communication control unit 14 transmits the visitor information 7 generated by the feature extraction unit 13 to the server computer 2.

  The setting unit 15 supplies the setting information supplied from the communication control unit 14 to the imaging unit 11, the encoding unit 12, and the feature extraction unit 13, respectively. As described above, the setting information C1 that the setting unit 15 supplies to the imaging unit 11 is information that specifies the camera rotation angle, shooting magnification, moving image / still image switching, and shooting interval, and is the setting that is supplied to the encoding unit 12. The information C2 is information that specifies an encoding method, and the setting information C3 that is supplied to the feature extraction unit 13 is information that specifies a feature amount to be extracted.

  Next, the configuration of the server computer 2 will be described with reference to FIG. The server computer 2 includes a communication control unit 21, an input / output control unit 22, a setting unit 23, a recording unit 24, a feature extraction unit 25, an analysis unit 26, and a saved version image generation unit 27. The server computer 2 is a hall facility management computer in which a program for realizing automatic event analysis is incorporated.

The communication control unit 21 controls communication between the server computer 2 and the photographing units 1 ₁ to 1 _n . The input / output control unit 22 controls reading / writing of the storage 3, input / output by the setting terminal 4, and output to the DVD recorder 5.

The setting unit 23, the recording unit 24, the feature extraction unit 25, the analysis unit 26, and the saved version image generation unit 27 include a plurality of element programs that cause the server computer 2 to execute each process of each unit described later.
This is realized by being incorporated in the server computer 2. In the present embodiment, the server computer 2 includes a plurality of processors, and the processing of each element program is processed in parallel by the plurality of processors.

The setting unit 23 receives a setting operation related to the photographing units 1 ₁ to 1 _n from the setting terminal 4. When the setting unit 23 detects that an operation for calling the setting screen is performed on the setting terminal 4, the setting unit 23 outputs the setting screen to the monitor of the setting terminal 4 via the input / output control unit 22. In the setting screen, the shooting unit selection screen appears first, and when the unit is selected, the camera rotation angle, shooting magnification, moving image / still image switching, shooting interval (setting information C1), A selection menu and an input frame for designating an encoding method (setting information C2), a feature amount to be extracted and the like (setting information C3) are displayed. When the operator of the system selects or inputs a desired set value, the selected or input setting information is supplied to the setting unit 23 via the input / output control unit 22. The setting unit 23 stores the supplied setting information in a memory in the server computer 2 and transmits the setting information to the imaging unit 1 via the communication control unit 21.

The recording unit 24 receives the image information 6 and the visitor information 7 transferred from the photographing units 1 ₁ to 1 _n via the communication control unit 21, and outputs them to the storage 3 via the input / output control unit 22.

  The feature extraction unit 25 receives a setting operation related to the feature extraction processing on the server computer 2 side from the setting terminal 4, and further executes the feature extraction processing based on the setting. When the feature extraction unit 25 detects that a setting screen call operation has been performed on the setting terminal 4, the feature extraction unit 25 outputs the setting screen to the monitor of the setting terminal 4 via the input / output control unit 22. On the setting screen, the type and number of feature quantities extracted by the feature extraction unit 25 can be set. It is also possible to set so that feature extraction is not performed. The setting information C3 input on the setting screen is supplied to the feature extraction unit 25 via the input / output control unit 22.

  When the operator designates a predetermined feature amount on the setting screen and performs the extraction process, the feature extraction unit 25 sequentially reads the image information recorded in the storage 3 and detects a person's face from the image. A temporary visitor identifier GID is assigned to the face detected from each image. Subsequently, the feature extraction unit 25 extracts a plurality of feature amounts F1 to FN representing facial features for each detected person, adds various management information to generate the visitor information 7 shown in FIG. Record in 3.

The operator of the system selectively turns on / off the function of the feature extraction unit 13 of the photographing units 1 ₁ to 1 _n and the function of the feature extraction unit 25 of the server computer 2, so Feature extraction can be performed. For example, when the number of feature quantities to be extracted is small and extraction in a short time is possible, the feature extraction unit 13 is turned on, and the feature extraction process is performed in parallel with shooting on the shooting unit 1 side. When the number of feature amounts to be extracted is large and requires time, an image taken with the function of the feature extraction unit 13 turned off is recorded in the storage 3 as image information 6, and the time is later extracted by the feature extraction unit 25. The extraction process can be performed over time.

  The analysis unit 26 receives a setting operation related to the analysis process from the setting terminal 4 and executes the analysis process based on the setting. When the analysis unit 26 detects that the setting screen call operation has been performed on the setting terminal 4, the analysis unit 26 outputs the setting screen to the monitor of the setting terminal 4 via the input / output control unit 22. On the setting screen, a selection menu for specifying an analysis item is displayed. The setting information C4 input on the setting screen is supplied to the analysis unit 26 via the input / output control unit 22.

  The analysis unit 26 reads the visitor information 7 recorded in the storage 3 by the recording unit 24 or the feature extraction unit 25, and performs analysis processing. Thereafter, the analysis unit 26 outputs the analysis result A to the storage 3 via the input / output control unit 22. If there is a display request from the setting terminal 4, the analysis result A is also output to the monitor of the setting terminal 4 via the input / output control unit 22.

  The saved version image generation unit 27 generates a digest by extracting a part of images from an image obtained by capturing an event pattern by adding an index to the important points of the image obtained by capturing the event pattern. Provide the function to do. The saved version image generation unit 27 receives a setting operation related to the saved version image generation process from the setting terminal 4 and generates a saved version image based on the setting. On the setting screen, a menu for selecting whether to generate an indexed image and whether to generate a digest is displayed. The setting information C5 input on the setting screen is supplied to the saved version image generation unit 27 via the input / output control unit 22.

  The saved version image generation unit 27 reads the image information 6 recorded in the storage 3 by the recording unit 24 and the analysis result A recorded in the storage 3 by the analysis unit 26 from the storage 3. Then, based on the analysis result A, an index is assigned or an image constituting a digest is selected. The indexed image or the generated digest is transferred to the DVD recorder 5 via the input / output control unit 22 and recorded on the DVD by the DVD recorder 5.

[System usage example]
Next, a method of using the system having the above configuration will be described with an example.

The operator of the system, to call up the display for setting the photographing unit 1 ₁ to 1 _n first selects some of the imaging unit used for photographing from a plurality of imaging units 1 ₁ to 1 _n. Then, the above-described setting information C1, C2, and C3 are set for the selected photographing unit. In the present embodiment, the setting information once set can be registered and recalled and used later.

  For example, in concerts, sports games, lectures, etc., first, in order to shoot performances on the stage, several ceiling-mounted shooting units behind the spectator seats are operated, and the camera changes the direction of the stage. Set the rotation angle to face. About these imaging | photography units, it sets so that a moving image imaging | photography or one still image imaging | photography may be performed every several seconds, and also a desired encoding system is selected. Furthermore, in order to photograph the faces of all the spectators, the photographing units installed on the left and right of the spectator seat are operated, and the rotation angles of the cameras are set so that each camera can catch the spectator's face obliquely from the front. In order to capture the entire audience without omission, it is preferable to set the rotation angle of the camera so that the imaging range of the camera that captures the audience from the front left and the imaging range of the camera that captures the audience from the front right slightly overlap. Then, a setting is made so that one still image is taken every several seconds, and a desired encoding method is selected.

  On the other hand, for events where visitors can freely move around the hall, such as business shows, exhibition / sale events, and children's character shows, the direction of the camera of each shooting unit so that each part of the hall is shot from at least two directions. Set. In this case as well, the rotation angle may be set so that the shooting range of each shooting unit slightly overlaps the shooting range of the adjacent shooting unit so that the entire hole is shot without omission. Then, desired settings are made for the moving image / still image and the encoding method.

  The setting related to feature extraction may be performed on the photographing unit or on the server computer. The type and number of feature quantities to be extracted depend on the information to be collected, that is, the purpose of analysis. If it is necessary to specify each visitor in performing the analysis, when setting the setting information C3, the type and number of feature quantities that can specify the individual are selected. On the other hand, if it is only necessary to be able to discriminate attributes such as sex, the type and number of feature quantities that can discriminate those attributes are selected. When the purpose of analysis is unknown, it is preferable to take a moving image and store the captured image in the storage 3 as it is. Alternatively, even if feature extraction is performed, it is preferable to extract a sufficient number of feature quantities that can identify an individual.

  When the setting is completed and the start of shooting is instructed by the system operator, each shooting unit starts shooting under the conditions set for each. The photographing unit whose feature extraction function is set to ON also performs feature extraction from the image. Information acquired or generated by each photographing unit is transferred to the server computer 2 and stored in the storage 3. These processes are repeated until the operator instructs the system to end photographing.

  An example of how the system is used after information is stored in the storage 3 will be described separately for each function of the system.

[Count of visitors]
Hereinafter, the counting process of the number of visitors executed by the analysis unit 26 will be described with reference to FIGS. 6 and 7. In the present embodiment, the analysis unit 26 performs different processes for an event in which the visitor is facing the stage direction and an event in which the visitor freely travels in the hall. Which process is to be executed is selected in advance by an operator. However, the analysis unit 26 determines the type of the event based on the setting information of the photographing unit set by the operator (for example, based on the set direction of each camera), and automatically performs an appropriate process from the two types of processes. You may make it choose.

(1) First visitor count function In an event where the visitors are facing the same direction, such as the stage direction, the scene of the visitors can be seen by shooting the scene in the hall from the stage side at a certain time. Images can be acquired. However, when shooting with a single camera, visitors in the back seat may be hidden behind the visitors in front, and it is difficult to acquire an image showing the faces of all members. Or, if the hall area is large and the number of visitors is large, even if it is possible to obtain an image showing the faces of all the members, each individual visitor is only small, so the facial features are extracted from that image Difficult to do.

Therefore, the scenes in the hall are divided and photographed by a plurality of photographing units to obtain a plurality of images, so that each visitor's face is included in at least one of the images. Of the shooting units 1 ₁ to 1 _N , which shooting unit is to be operated is determined according to the position of the visitor seat and the number of visitors, but in the following explanation, the entire visitor seat is visited. An example will be described in which a photographing unit for photographing from the front right of the passenger seat and a photographing unit for photographing the entire visitor seat from the left front of the attendee seat are described.

  First, when using this function, the system operator needs to make a setting for extracting feature quantities necessary for specifying an individual when setting the feature extraction unit setting information C3. That is, the processing described below is based on the premise that the feature amounts F1 to FN included in the visitor information stored in the storage 3 are feature amounts that can identify an individual.

  First, the analysis unit 26 reads visitor information generated from images captured at the same time by the two selected photographing units (referred to as photographing units 1 a and 1 b) from the storage 3. That is, the visitor information indicating the two photographing units with the photographing unit identifier UID selected and the photographing date and time information DT indicating the predetermined date and time is discriminated and read. The visitor information read here may be information generated on the photographing unit side or information generated on the server computer side.

  Subsequently, as illustrated in FIG. 6, the analysis unit 26 executes the target identification process 8 and the personal identification process 9 using the read visitor information. The object identification process 8 is a process for identifying a visitor's face from an image. In the present embodiment, the object identification process 8 is performed on the captured image of the imaging unit 1a and the captured image of the imaging unit 1b. Further, in the present embodiment, by setting the attributes or states of visitors who want to be identified, it is possible to identify only the visitors who have the attributes or states. Here, the “attribute” is a characteristic unique to the visitor such as gender and age, and the “state” is a characteristic that changes during the visit like a facial expression. If no attribute or state is set, all faces included in the image are detected.

  In the object identification process 8, the feature amounts F1 to FN included in the visitor information are collated with the feature amount indicating the attribute or state of the target to be counted, and the person from which the visitor information is extracted is counted based on the collation result. Identify whether it should be a target. For example, if the number of males in the visitor is to be checked, the feature amounts F1 to FN included in the visitor information are collated with the feature amount indicating the male feature to determine whether the person indicated by the visitor information is male. Identify women. Alternatively, if the number of smiling visitors is to be checked, the feature amounts F1 to FN included in the visitor information are compared with the feature amounts indicating the smile feature, and whether or not the person indicated by the visitor information is a smile To identify.

  The personal identification process 9 is a process for detecting a target included in a plurality of images. In the present embodiment, as shown in FIG. 6 (a), the feature amount included in the target visitor information detected from the photographed image of the photographing unit 1a by the object identification process 8 and the photographed image of the photographing unit 1b are detected. The visitors having a plurality of pieces of visitor information are detected by sequentially collating the feature amounts included in the target visitor information. In the present embodiment, when the matching results match, the temporary visitor identifier GID included in the plurality of visitor information is replaced with a common new visitor identifier GID. The common visitor identifier GID assigned here can be used when other analysis processing is performed thereafter.

  Further, the analysis unit 26 executes the counting process 10 using two types of counters. The first counter is a counter that counts the number of visitor information determined to be counted in the object identification process 8. The second counter is a counter that counts the number of visitor information determined to be a duplicate count in the personal identification process 9. For all the visitor information read from the storage 3, when the object identification process 8 and the personal identification process 9 are completed, the analysis unit 26 performs an operation of subtracting the value of the second counter from the value of the first counter. Do.

  As another method, as shown in FIG. 6 (b), when the individual identification process 9 is executed first and a plurality of pieces of visitor information exist for one visitor, one of them is included. A method of leaving only the visitor information and excluding other visitor information from the processing target and executing the object identification process 8 only for the remaining visitor information is also conceivable. However, since the personal identification process 9 has a larger calculation amount at the time of the feature amount comparison than the object identification process 8, the efficiency of the identification process is better when counting is performed according to the procedure of FIG.

  The case where the visitor information obtained by the two photographing units 1a and 1b is used has been described above as an example. However, even in the case where the visitor information obtained by the three or more photographing units is used, the personal identification processing 9 Except for the fact that the number of verifications increases, this is the same as when there are two photographing units. In practice, if there are dozens of spectator seats, it will be necessary to operate four or more photographing units. For example, all the photographing units arranged at predetermined intervals on the left and right sides of the spectator seat are operated so that photographing from the diagonally front left and right is performed every five to ten rows of the spectator seats. For example, if the first row of the camera is photographed in the first row to the seventh row of the spectator seat, and the next row camera is photographed in the third row to the ninth row, it is possible to prevent omissions. In this case, when counting the number of visitors, it is possible to prevent duplication counting by checking the overlap of the visitor information between the image obtained by the front row photographing unit and the image obtained by the rear row photographing unit. .

  The number of visitors determined by the above processing is stored as an analysis result together with the shooting unit identifier UID of the shooting unit used to acquire the image, the date and time DT when the image was shot, and the attribute or status information to be counted. 3 is saved. Further, it is displayed on the screen of the setting terminal 4 in response to a request from the operator.

  The analysis part 26 can also perform another analysis process using the value of the number of visitors obtained by the said process. For example, in addition to the generation of graphs representing the ratio of the number of visitors (gender ratio, etc.) and the change in the number of visitors depending on the time of day, various analyzes based on the number of visitors can be performed. The analysis results obtained by these processes are also stored in the storage 3.

  The analysis results accumulated in the storage 3 are of course useful as statistical information, but the system according to the present embodiment can generate further added value by using the analysis results. . The function for that will be described later.

(2) Second visitor count function In the event that visitors can freely travel through the hall, the direction of the faces of the visitors in the hall changes from moment to moment, so visitors come from one shot image It can be said that it is almost impossible to identify all count objects. Even if images taken from a plurality of directions are used as in the first visitor number counting function, it is very difficult to determine which photographing unit is to be operated considering that the orientation of the face changes by 360 degrees. . Also, if there are visitors who could shoot the front face and visitors who could only shoot the side face, the feature values obtained from the face differ between the front and side faces (some feature quantities can only be obtained from the front face). Therefore, verification is difficult and may lead to a decrease in identification accuracy.

  Therefore, in order to be able to count the number of visitors using only the visitor information obtained from the front face, repeat the same scene in the hall so that as many front faces as possible can be obtained. Take a picture. It is preferable to select a photographing unit that is close to a place where there is an object or a notice that attracts visitors' attention. Hereinafter, the second visitor number counting function will be described by exemplifying a case where repeated shooting is performed using one shooting unit.

  First, when using this function, the system operator selects still image shooting at a predetermined interval when setting information C1 is set. Further, when setting information C3 is set, settings are made so that only the front face is detected and the feature amount is extracted from the front face image.

  FIG. 7 is a flowchart showing the second visitor count process executed by the analysis unit 26. The analysis unit 26 reads the visitor information including the shooting unit identifier UID of the selected shooting unit from the storage 3 in order from the earliest time indicated by the shooting time information DT (S101). Similar to the first visitor count function, the feature amount information read here may be information generated on the photographing unit side or information generated on the server computer side.

  Every time one piece of visitor information is read, the analysis unit 26 identifies whether or not the visitor indicated by the visitor information is a visitor that has been detected in the previous process (first face identification process) (S102). In the initial face identification process, the feature amount included in the newly read visitor information is collated with the feature amount included in all the already read visitor information. If there is no already-read visitor information that matches the matching result, it is determined that the visitor is the first detected visitor (S103). If it is not the first detection, the next visitor information is read. In the case of a first-detected visitor, an object identification process is executed (S104).

  The object identification process S104 is a process for identifying a visitor (person) from the image. In this embodiment, by setting an attribute or state of a visitor to be identified, it is possible to identify only a visitor having that attribute or state. If no attributes or status are set, all visitors included in the image are identified and detected. In the object identification process S104, the feature amounts F1 to FN included in the visitor information of the first detected visitor are collated with the feature amount indicating the attribute or state of the target to be counted, and based on the collation result, the visitor information It is identified whether or not the extraction source person is to be counted.

  If the collation results match, a predetermined counter is counted up in the counting process (S105), and the visitor information is stored in the memory as already-read visitor information. Thereafter, the processing of steps S101 to S105 is repeated until there is no visitor information indicating the time of the time zone in which the number of visitors is to be checked.

  According to the above processing, if the front face can be photographed even once, the number of visitors can be counted, so the number of visitors can be accurately obtained even in an environment where the direction of the visitors changes every moment.

  The number of visitors determined by the above processing is stored as an analysis result together with the shooting unit identifier UID of the shooting unit used to acquire the image, the date and time DT when the image was shot, and the attribute or status information to be counted. 3 is saved. Further, it is displayed on the screen of the setting terminal 4 in response to a request from the operator. Further, similarly to the first visitor count function, the obtained number of visitors is used for automatic generation of other statistical information.

[Generate saved image]
Next, the function of generating a saved version image using the analysis result, that is, the function of the saved version image generating unit 27 of the server computer 2 will be described. There are four types of stored version images that can be generated by the system of the present embodiment. The first preserved version image is an indexed image (hereinafter referred to as “indexed scene image”) of key points of a moving image (or still image slide) that captures the scene of the event from the start to the end. . The second preserved version image is a part of a moving image (or a still image slide) obtained by photographing the scene of the event from the start to the end and re-edited (hereinafter referred to as “digest scene image”). is there. The third preserved version image is an image obtained by indexing important points of the video screened during the event (hereinafter referred to as “indexed screened image”). The fourth stored version image is an image obtained by extracting a part of the video screened during the event and editing it again (hereinafter referred to as “digest screened image”).

  FIG. 8 is a diagram showing the concept of generating indexed scene scene images and digest scene scene images. In the system according to the present embodiment, as described above, in an event such as a concert or a lecture, an image obtained by photographing the stage side is encoded to generate image information 6 such as a moving image and a slide. The analysis result A can be obtained by performing the analysis process 31 by the face identification process on the image obtained by photographing the person's seat. At this time, as described with reference to FIG. 3, the image information 6 includes shooting date / time information DT. Further, in the analysis processing for counting the number of visitors at a predetermined time, the analysis result A also includes information DT on the shooting date / time.

  The stored version image generation unit 27 uses the shooting date / time information DT to associate the image information 6 obtained by shooting on the stage side with the analysis result A for each shooting time. As a result, as shown in the figure, a process 32 for estimating the visitor's evaluation on the stage for each time or for each time zone is executed. Further, the stage-side shot image is automatically edited 33 based on the estimated evaluation, and the edited moving image, slide, etc. are recorded on the DVD.

  FIG. 9 shows an example of processing of the saved version image generation unit 27. For example, it is assumed that the number of smiling women and smiling men is counted every 30 seconds by the first number counting function, and the result is stored in the storage 3. The stored version image generation unit 27 reads the information of the result from the storage 3, and determines the time zone when the number of smiling women exceeds a predetermined number (30 in the example in the figure). That is, it is estimated that the time zone in which the number of smiling women exceeds the predetermined number is a time zone in which the female evaluation is high, and the time zone in which the female evaluation does not exceed the predetermined number is the time zone in which the female evaluation is low. Subsequently, as shown in the figure, a digest is generated by extracting only the images in the time zone with high evaluation. Furthermore, an index is added at a time when the evaluation is particularly high and a time when the evaluation is low.

  If this function is used, for example, a female DVD that collects and records only images of the time when the female reaction was good, a male DVD that collects and records only the images of the time when the male response was good, and the reaction of the visitors in particular You can index good and bad times. That is, it is possible to select an image based on the reaction of the visitor, not the editor. Furthermore, although it is not always possible to get honest responses from visitors in questionnaires etc., this function can be used to obtain responses directly from the visitor's face information. I can grasp it.

  In the above, the function to generate a saved version image from an image of the event scene (mainly the stage) has been explained, but recording the start time of the video and audio shown during the event as well Thus, the screening time and the analysis result can be associated with each time. Therefore, also in this case, it is possible to automatically perform image editing according to the reaction of the visitors by the method described above.

  In the above example, the saved version image is generated using the first number counting function, but the second number counting function may be used. With the second person counting function, it is not possible to observe the reaction of visitors at a predetermined time, but it is possible to count the number of visitors by dividing the time zone. Therefore, it is possible to decide whether or not to select an image for that time period according to the reaction of the visitors in each time period (for example, morning and afternoon). Can be used for generation.

  In addition, when the analysis unit 26 performs analysis processing other than the number of people counting function, if the analysis result is a result that leads to the evaluation of the event, a method of generating a stored version image using the analysis result is also conceivable. For example, based on the increase / decrease (differential value, etc.) of the number of smiling visitors, a process of indexing the times when the number of smiling visitors suddenly increases or decreases rapidly can be considered.

  In the above examples, the images once stored in the storage 3 or other storage media are edited. However, the time when the number of people by the analysis unit 26 is counted in real time during shooting and the reaction of the visitors is good. A method is also conceivable in which only the image of the band is recorded in the storage 3 and the automatic editing process is executed while shooting. For example, the image information transferred from the photographing unit is not immediately recorded in the storage 3 but is left on the memory of the server computer 2. Then, for example, when image information for 5 minutes is accumulated in the memory, feature extraction processing and analysis processing are executed on the image information to estimate the visitor's evaluation. If the evaluation is good, the image information for 10 minutes is recorded in the storage 3. On the other hand, if the evaluation is bad, the image information on the memory is discarded without being recorded in the storage 3. According to this method, when the stage performance etc. is finished, the saved version image such as digest video is also completed, so the DVD with the digest video recorded as a souvenir for visitors who finished watching the stage The service that was provided can be realized.

[System effects]
As described above, the system of the present embodiment can freely set the shooting direction and number of times, so that it is possible to select an optimal shooting method for collecting information according to the type of event. Can perform analysis processing, such as counting the number of visitors, using images obtained by the shooting method. Therefore, it is particularly suitable for application to a multipurpose hall where all kinds of events may be held.

  According to the 1st visitor count function, you can check the exact number of people using images from multiple cameras, so you can visit a wide range that cannot be taken with just one camera. Even if there is a person, or even if there is a visitor who can not shoot with a single camera hidden behind a human figure, the counting can be performed accurately.

  According to the second visitor number counting function, it is possible to check the exact number of people by using images repeatedly taken from the same direction, so that an unspecified number of visitors face in any direction. Even if it is, it can count accurately.

  In addition, according to the saved version image generation function, an index is assigned based on how the visitors responded to the image to be saved, rather than what is reflected in the image to be saved. Or a digest image can be selected, so that an edited image suitable for the visitor, not the editor, can be generated.

The figure which shows schematic structure of the system in one Embodiment of this invention. Diagram showing the internal configuration of the shooting unit The figure which shows one recording form of image information Diagram showing one form of visitor information recording Diagram showing the configuration of the server computer The figure for demonstrating a 1st person count function (counting function) The figure for demonstrating the 2nd people count function (count function) Conceptual diagram showing preservation version image generation function The figure which shows an example of the edit by the preservation version image generation function

Explanation of symbols

C1-C5 Setting information, F feature value, I still image, M movie, A analysis result 6 image information, 7 visitor information

Claims (2)

An automatic counting device for automatically counting objects existing in a predetermined place,
Extraction means for sequentially performing a process of extracting front face information of a target included in an image for a plurality of images obtained by repeatedly capturing scenes at different times from the same direction;
The front face information extracted from one image by the extracting means is first detected in the one image by collating with the front face information extracted from the image processed before the one image by the extracting means. A first identification means for identifying
Second identifying means for identifying the counting object based on the front face information of the object identified by the first identifying means;
A counting device comprising: counting means for counting the number of counting objects identified by the second identifying means. The second identification means is means capable of identifying an attribute and / or state of a counting target included in each image based on the front face information,
2. The automatic counting apparatus according to claim 1, wherein the counting unit is a unit capable of counting the number of the counting objects for each attribute and / or state.

Images