JP2015230616A - Image processing method and image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing method capable of extracting accurate attribute information of a person and tracking the person from a display object to a specific point.SOLUTION: An image processor includes an image acquisition part 101 for acquiring a video or an image obtained by photographing a monitoring area, a person attribute extraction part 104 for extracting attribute information of a person existing in the monitoring area, a tracking part 102 for performing identification processing of the person by using the attribute information to generate tracking information of each person by tracking processing, a visual recognition determination part 103 for determining whether the person visually recognizes a display object to generate visual recognition determination information, a visit determination part 105 for determining whether the person visits a specific point to generate visit determination information, a tracking information extraction part 106 for extracting related tracking information on the basis of the visual recognition determination information or the visit determination information, and a tracking information conversion part 107 for converting the tracking information into a flow line of the person or a stream of persons.

Description

  The present invention relates to an image processing method and an image processing apparatus, and in particular, to measure the effect of a posted object posted on a person in a place where an unspecified number of persons pass, such as a store, a station, an airport, a plaza, and a stadium. This invention relates to a technique suitable for use in

  Conventionally, in a system for measuring the effect of a posted object, a posted object is used by using the camera installed on the posted object and the camera installed at the destination of the posted object, and the number of people whose faces taken by both cameras match. Measure the effect of. Here, the posting object is, for example, an advertisement or a signboard, and the guidance destination of the posting object is, for example, a store or a counter.

  In Patent Document 1, a face of a person photographed by a camera on the advertisement side is detected, and a feature amount of the face is extracted. In addition, feature amounts are extracted in the same manner for a person photographed with a camera installed in a store. And when this feature quantity corresponds with what was extracted by the advertisement side, the technique which calculates an advertisement effect as the person who browsed the advertisement visited the store is disclosed.

JP 2008-102176 A

  However, in the technique for extracting and holding the facial feature amount, it is necessary to hold the same level of information as that of the face image, which may cause a problem regarding the privacy of the person and the use of personal data. In addition, when extracting facial features, the person's face must be shown on the camera, and the camera installation position must be determined so that the person's face is in front of the person when viewing the posted object and reaching the guidance point. There is. Furthermore, there is a problem that it is necessary to make the position where the up and down inclination of the photographed face is small, and the restriction on the camera installation position is large.

  Furthermore, in the method of identifying the same person between two distant cameras and measuring the effect of the posted object, there is a problem that the behavior from the posted object to the guidance point cannot be grasped.

On the other hand, it is also conceivable to perform person identification using information on human attributes, such as hair color, clothing color, body shape, and accessories. However, if the position of the camera for extracting information is too far from the target person or the person is moving, there is a problem that a clear image of the person cannot be obtained and accurate person attributes cannot be extracted. there were.
In view of the above-described problems, an object of the present invention is to extract accurate attribute information of a person and to track a person from a posted object to a specific point.

  An image processing method according to the present invention includes an image acquisition step of acquiring a video or image obtained by photographing a monitoring area, an attribute extraction step of extracting attribute information of a person existing in the monitoring area, and the person using the attribute information A tracking process for performing identification processing and generating tracking information for each person through the tracking process, a visual recognition determination process for determining whether or not the person has viewed the posted object and generating visual determination information, and the person specifying Based on the visit determination process that determines whether or not a point has been visited and generates visit determination information, and the visual determination information generated in the visual determination process or the visit determination information generated in the visit determination process An extraction process for extracting tracking information; and a conversion process for converting the tracking information extracted in the extraction process into a flow line of a person or a flow of a person.

  According to the present invention, accurate attribute information of a person can be extracted, and person tracking from a posted object to a specific point can be performed.

It is a block diagram which shows the function structural example of the image processing apparatus which concerns on embodiment of this invention. It is a flowchart explaining an example of the effect measurement process which the image processing apparatus which concerns on embodiment of this invention performs. It is a figure explaining a tracking process. It is a figure explaining a person identification process. It is a figure explaining visual recognition judgment processing. It is a figure explaining visit judgment processing. It is a figure explaining tracking information, visual recognition judgment information, and visit judgment information. It is a figure explaining the flow line drawing to a frame image. It is a figure explaining the flow line to MAP information, and drawing of a person flow.

Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.
<First Embodiment>
FIG. 1 is a block diagram illustrating a configuration example of a posted object effect measurement system according to the first embodiment.
In FIG. 1, 101 is an image acquisition unit, 102 is a tracking unit, 103 is a visual recognition determination unit, 104 is a person attribute extraction unit, 105 is a visit determination unit, 106 is a tracking information extraction unit (hereinafter referred to as an extraction unit), and 107 is tracking. An information conversion unit (hereinafter referred to as conversion unit) 108 is a display unit.

  The image acquired from the image acquisition unit 101 is obtained by photographing at least one point. In addition, images (frame images) of each frame constituting a moving image in which one or more persons frame in or out of the image or move in the image over a plurality of frames are sequentially acquired. Then, the image acquisition unit 101 sequentially transmits the images of each frame acquired from the monitoring area to the tracking unit 102.

  The moving image (video) sent to the tracking unit 102 may be acquired from an imaging device capable of capturing at least one moving image, or may be acquired from a device that holds such a moving image in advance. Good. In the present embodiment, the acquisition destination is not limited to a specific acquisition destination.

  The tracking unit 102 tracks a plurality of persons shown in the image of the frame received from the image acquisition unit 101 by matching using an image feature amount, and an identifier (person) that uniquely identifies the position, size, and person. ID). Here, a case where a frame image is acquired from an imaging apparatus capable of capturing one moving image will be described as an example. The tracking unit 102 detects the moving object (or object) in the image without determining whether a person is included in the image, assigns a moving object ID, determines the characteristics of the moving object, and determines the determination. The moving object may be tracked based on the feature.

First, tracking is started by detecting or designating at least one tracking target from the frame image of interest. That is, for the detected tracking target, for example, a person detected as a person using a learning discriminator, an image feature amount is extracted and a person ID is issued.
By storing the image feature amount in association with the person ID, the image feature of the designated person can be specified even if a plurality of persons are tracked. To start tracking, the user may input a rectangle to the frame image, and the rectangle may be set as the tracking target, or the tracking target may be detected from sensor information for grasping the position of the object.

  Next, search and matching are performed on the person tracked in the past frame based on the image feature amount extracted at the start of tracking, and the position and size with the highest similarity (matching score) are set as the tracking result. At this time, the same person ID as the person ID added in the past frame is associated with the tracking result to indicate that the tracking result is the same person as the tracking target of the past frame. If the tracking target is not found in the frame image of interest, the tracking of the target is terminated.

  The image feature quantity used for matching may be a color histogram within a rectangle including a person, a feature quantity obtained from a feature descriptor, or shape information obtained by extracting a target edge. That is, the image feature amount to be used is not limited to a specific feature amount.

By applying this process to sequentially obtained frame images, it is possible to identify the position and size of the person appearing in the frame image, and to distinguish each person based on the person ID.
Further, the association between the person ID and the image feature amount, the tracking position, and the tracking size is for specifying the image feature amount, the tracking position, and the tracking size as specific person information, and corresponds to the content of the person ID. The attaching method is not limited to specific contents and a specific method.

The tracking process will be described in detail with reference to FIG.
In FIG. 3, reference numeral 300 denotes a frame image having a plurality of tracking objects 301, 302 a tracking frame that can be obtained from the tracking position and size of the tracking object 301, and 303 a person ID of the tracking object 301.

Consider a case where the frame images 300 are acquired sequentially. At this time, each frame image is distinguished by setting t = 1, 2, 3,... For each frame image according to the time series.
Two tracking objects 301 exist in the frame image 300 at t = 1. Further, it is assumed that the tracking of the tracking target 301 on the left side in the frame image 300 has been started before t = 1.

First, tracking is started for the right target in the frame image 300 that has not been tracked in the past frame, and a new person ID 303 is issued. Next, the person on the left side in the frame image 300 is tracked by the search using the information of the past frame.
At t = 2 and t = 3, since an existing person (ID = 1, ID = 2) is tracked in the past frame, only the tracking position and tracking size by search are specified.

  At t = 4, since a new person has entered the frame, tracking is started. Similar to t = 2, at t = 5, only the tracking position and the tracking size by the search are specified. At t = 6, since the person whose person ID 303 is 2 has gone out of the frame, the person cannot be found by the search, and the tracking of the person whose person ID 303 is 2 ends.

  Further, when an image is acquired from an imaging device capable of capturing one or more moving images, tracking can be performed by the same processing by acquiring the image by distinguishing the imaging device. Furthermore, a tracking unit may be provided for each imaging apparatus and each process may be performed. The tracking process of the tracking unit 102 is for identifying the position of a person in the image and distinguishing each person, and the tracking method of the tracking unit 102 is not limited to a specific method.

  Furthermore, when the tracking unit 102 starts tracking, the tracking unit 102 identifies a person by using an image feature amount of the person tracked in the past or person attribute information described later. That is, the similarity is calculated by comparing with a new image feature amount to be tracked or attribute information of a person described later, and if the similarity is more than a certain level, it is determined that the person is the same person as the person tracked in the past To do.

  If it is possible to identify a person tracked in the past, the person ID of the person is set as the ID of the person tracked in the past. Even if a person who has once been tracked is identified by person identification, it can be treated as the same person when tracking is performed again. Further, identification of a person may be performed not only when tracking is started but also during tracking.

The person identification process performed by the tracking unit 102 will be described in more detail with reference to FIG.
In the frame image 300 at t = 1, two tracking objects 301 exist and are tracked. At t = 2, the person whose person ID 303 is 2 is out of the frame and the tracking is finished. Also, a new person enters the frame from the right side of the image, and tracking is started.

  At this time, a person identification process is performed to determine whether the person has been tracked in the past. Here, it is determined that the person has not been tracked in the past, and a new person ID is issued. Therefore, at t = 3, the person is tracked as a person having a person ID of 3. At t = 4, the person enters the frame from the left of the image, and tracking is started.

  Similarly, a person identification process is performed to determine whether the person has been tracked in the past. Here, since it is determined that the person ID 303 tracked in the past is a person 2, a new person ID is not issued, and the person ID 303 is tracked using 2 again. Here, an example in which a person is identified by a frame image obtained from the same imaging device has been described, but it is also possible to identify a person between frame images obtained from different imaging devices by the same processing.

  When the tracking unit 102 performs tracking processing of a person in a certain frame image, the tracking unit 102 generates tracking information including information related to tracking, such as a frame number, a person ID, a tracking position, and a tracking size. In addition, the tracking information may include a matching score, a movement amount, a parameter value handled during the tracking process, and the like. Further, the tracking position and the tracking size may be values in image coordinates, or values in world coordinates converted from information of sensors or a plurality of cameras.

  The visual recognition determination unit 103 acquires all tracking positions obtained from the frame image of interest from the tracking information generated by the tracking unit 102 for a certain frame image, and determines whether or not each person has visually recognized the posted object. . Various methods using the position of the posted object and the information of the person of interest can be applied to the visual recognition determination.

  For example, the visual recognition determination unit 103 estimates the line-of-sight direction from the image information for the person of interest, obtains the visual range of the person from the person position and the line-of-sight direction, and visually recognizes the posted object when the posted object exists within the range May be determined. The viewing range may be a range of a predetermined right and left angle from the line-of-sight direction, or may be a range of a predetermined right and left angle in the moving direction of the person without estimating the line-of-sight direction. The visual field range may be estimated based on the moving direction of the moving object without determining whether the person is a person.

  Alternatively, a viewable area may be set based on the position of the posting object, and when the tracking position of the person of interest exists in the viewable area, it may be determined that the posting object has been viewed. In either method, in order to increase the accuracy of the determination, it may be determined that the posted object is visually recognized when the determination condition is satisfied for a certain frame or more. The visually recognizable area may be set in the memory in the visual recognition determination unit 103 by the user.

  That is, the visual recognition determination method is not limited to a specific method. Moreover, the position setting of the posting object used in the visual recognition determination unit 103 may be a position on the image set in advance by a user input. Alternatively, the posted object position may be detected using image recognition, and is not limited to a specific method.

The visual recognition determination process performed by the visual recognition determination unit 103 will be described in more detail with reference to FIG.
FIG. 5A illustrates a case where visual recognition is performed using the line-of-sight direction 502 of the tracking target 301. Consider a frame image 300 in which a tracking target 301 and a posting object 500 are shown. At this time, the gaze direction 502 is obtained by gaze direction estimation.

  A visual range 503 is calculated based on the visual recognition distance and angle set in advance with respect to the visual line direction 502. If the posted object position 501 exists within the viewing range 503, it is determined that the tracking target 301 has viewed the posted object 500. In this example, the posted object position 501 and the viewing range 503 are within the image. However, the viewing range 503 may extend outside the image, and thus the posted object position 501 may also be set outside the image. Absent.

  Furthermore, the shape of the visible range 503 may be a rectangle or an ellipse, and is not limited to a sector shape. Further, the calculation of the visual recognition range 503 may be based on the moving direction of the person or the face direction instead of the line-of-sight direction.

  FIG. 5B shows a case where the visual recognition is determined using the tracking frame 302 and the visible region. A viewable area 504 of the posted object is set for the image, and the tracking position 505 is acquired based on the tracking frame 302. When the tracking position 505 exists in the viewable area 504, it is determined that the tracking target 301 has viewed the posted object 500. In this example, the tracking position 505 is shown as the foot of the tracking target 301, but it may be the waist position or the head position of the tracking target 301. Accordingly, the range of the visible region 504 is not limited to a specific range.

  Then, when the visual recognition determination unit 103 determines that the noticed person has visually recognized the posting object, the visual recognition determination unit 103 generates visual recognition information that associates the identifier of the visual posting object (posting object ID) with the visual recognition person ID. To do. Note that the visual recognition determination information is for specifying which person has viewed which posted object, and the visual determination information format is not limited to a specific format.

  The person attribute extraction unit 104 extracts the attribute information of the person determined to have viewed the posted object in the visual recognition determination unit 103 from the frame image determined to have viewed the posted object using a learning discriminator along the attribute. . Here, the attribute may indicate information on the person's age, gender, hair color, clothes color, skin color, body type, accessories such as glasses and hats, possessed objects such as a bag, and past frames. Gait may be extracted from this information. It is not limited to specific information. In addition, the extraction of the attribute information is not limited to the frame image determined to have visually recognized the posted object, and may be performed by designating a period of several frames before and after.

  The attribute information of the person is used for person identification by the tracking unit 102. For each attribute, a general person identification technique is used, and a person is identified by calculating a similarity based on differences in age, gender, color information, and the like. Instead of the attribute information of the person, characteristic information of the moving object (or object) may be extracted and used for tracking the moving object.

  The visit determination unit 105 acquires all person positions obtained in the frame of interest from the tracking information generated by the tracking unit 102 for a certain frame image, and determines whether or not each person has visited a specific point. Various methods using position information of a specific point and information of a person of interest can be applied to the visit determination.

  For example, when a visitor area related to a specific point is input in advance and the tracking position of the person of interest exists, it may be determined that a visit has been made. At this time, in order to improve the visit determination accuracy, it may be determined that a visit has been made to a specific point when a visit determination is made for a certain frame or more. In addition, a line segment is drawn as a passing line at the entrance of a specific point, and the passage is determined by crossing the passing line and the person's flow line. Also good. Further, it may be determined that all of the tracked persons have been visited in the frame image taken inside the specific point.

  That is, the visit determination method is not limited to a specific method. In addition, the specific spot area and the detection line used by the visit determination unit 105 may be a predetermined area set by a user input or a specific spot detected using image recognition. It is not limited to a specific method.

The visit determination process performed in the visit determination part 105 is demonstrated in detail using FIG.
FIG. 6A shows a case where a visit is determined using the tracking position 505 and the visit area 601. When the tracking position 505 related to the tracking target 301 exists in the preset visit area 601, it is determined that the tracking target 301 has visited a specific point.

  FIG. 6B shows a case where a visit is determined using the flow line 602 and the passage line 603. First, a flow line 602 is obtained by connecting tracking positions 505 obtained from tracking results of past frames in time series order. Then, it is determined by calculation whether the passage line 603 set in advance with respect to the entrance 600 and the flow line 602 intersect. As a result of this determination, when it is determined that the passing line 603 and the flow line 602 intersect, it is determined that the tracking target 301 has visited a specific point.

  When the visit determination unit 105 determines that the focused person has visited a specific point, the visit determination unit 105 generates visit determination information that associates the identifier (point ID) of the visited point with the person ID of the visited person. The visit determination information is for specifying which person has visited which point, and the format of the visit determination information is not limited to a specific format.

  The extraction unit 106 extracts tracking information related to the person ID by specifying the person ID based on the visual recognition determination information and the visit determination information. That is, the user's input or analysis program specifies the posted object ID or the spot ID, thereby specifying the person ID that has been visually determined or visited. Then, all the tracking information related to the person ID is extracted. Here, when there are a plurality of person IDs determined to be visually recognized or visited, the tracking information relating to all the determined person IDs is extracted for each person ID.

The tracking information extraction process performed by the extraction unit 106 will be described in more detail with reference to FIG.
FIG. 7 shows information generated by the processing so far, in which 701 is tracking information, 702 is visual recognition determination information, and 703 is visit determination information. For example, when extracting tracking information of a person who visually recognizes a posting object whose posting object ID is A, information whose posting object ID is “A” is first searched from the visual recognition determination information.

Next, the corresponding person ID “1” is acquired from the found information. Here, when there are a plurality of found information, a plurality of person IDs are acquired from all the information.
Next, all the tracking information regarding the person ID “1” obtained from the tracking information 701 is extracted. With regard to the visit determination information, the tracking information can be extracted by the same process.

  Further, the tracking information may be extracted after setting a condition of logical product or logical sum for the person ID obtained from the visual recognition determination information and the person ID obtained from the visit determination. In FIG. 7, the information format is shown in the database format. However, the information holding format is not limited to a specific format, and items included in each information are not limited to specific items. .

  The conversion unit 107 converts the tracking information extracted by the extraction unit 106 into a person's flow line or a person's flow. For example, when converting tracking information of a specific person into a flow line, the tracking position of the person can be converted into a flow line of the person by connecting the tracking positions of the person in the order of the processed frames. At this time, smoothing processing may be applied to the flow line using a spline function or the like.

  In addition, when converting to the flow of a plurality of persons at a specific point, by calculating the movement vector from the difference in tracking position between two frames for all persons, and by taking statistics of the movement vector within the specific point, It can be converted into human flow. The conversion method is an example, and the method of converting the tracking information into the flow of the person or the flow of the person is not limited to a specific method.

  Further, the number of person IDs obtained by the extraction unit 106 is used to digitize the effect of the posted object. For example, the total number of person IDs obtained from the viewing determination information is the number of viewers and the total number of person IDs obtained from the visit determination information is the number of visitors, and the person ID obtained from both the viewing determination information and the visit determination information is viewed. Number of people. Then, the ratio of the number of visually-visited visitors to the number of visually-visited persons is calculated to determine the proportion of people who have viewed the posted object, and the ratio of the number of visually-visited visitors to the number of visitors is used. Find the percentage of objects you saw.

  The display unit 108 displays an image or numerical value drawn by processing information obtained by either the extraction unit 106 or the conversion unit 107, or both, to the user. That is, the movement trajectory obtained by the conversion unit 107 is drawn on images relating to all shooting positions.

  At this time, MAP information is acquired from an input unit (not shown), or MAP information is created from frame images obtained by photographing a plurality of points. Then, the flow line or the flow of a plurality of persons may be drawn on the MAP by converting the flow line or the flow coordinates to the MAP coordinates. Further, the tracking position may be drawn by converting the coordinates of the tracking position into MAP coordinates. The conversion from the image coordinates to the MAP coordinates can be performed by calibrating the imaging device or the frame image and the MAP for acquiring the image.

The drawing process performed by the display unit 108 will be described in more detail with reference to FIGS.
FIG. 8 is a diagram illustrating a case where the flow line 602 obtained by the conversion unit 107 is drawn on the frame image 300. For simplification, it is assumed that the obtained frame image 300 relates to four points, and the flow line 602 to be drawn is a case where the extracted tracking information is one person. Of course, the point of the obtained frame image is one or more points, and the tracing to be drawn may be a plurality of persons.

FIG. 9A shows an example in which a flow line is drawn on the calibrated MAP information 900.
The MAP information 900 includes information as a sketch of each tracking range, and information on a posting object ID 901 and a spot ID 902. Further, the posting object ID 901 and the spot ID 902 may be held separately from the MAP information 900 and automatically input by associating with the MAP information 900 later, or may be manually input by the user.

  First, as in the case where the flow line 602 obtained by the conversion unit 107 is drawn on the frame image 300, the flow line 602 at the image coordinates is obtained. At this time, the calibration information with the MAP information 900 is converted into the MAP coordinate flow line 904 in the MAP coordinates. This MAP coordinate flow line 904 is drawn in the MAP information 900. This conversion process may be performed by the conversion unit 107.

  Further, the range that can be captured as a frame image does not necessarily cover the MAP information 900, and when there are a plurality of frame image shooting points, the MAP coordinate tracking position 903 between the shooting points cannot be obtained. The MAP coordinate flow line 904 is interrupted. Therefore, the interrupted movement trajectory may be predicted from the time-series order of the MAP coordinate tracking position 903, and the interpolation flow line 905 may be drawn. In FIG. 9A, the MAP coordinate flow line 904 is drawn with a solid line and the interpolation flow line 905 is drawn with a broken line, but each may be expressed with shades of color, or may be expressed with different colors. The camera may be installed so that the MAP coordinate flow line 904 is not interrupted.

FIG. 9B shows an example in which a person flow 906 is drawn on the calibrated MAP information 900.
As in the case of FIG. 9A, the interpolation flow 907 may be drawn for points where tracking information cannot be obtained. Further, the direction of flow of the person is indicated by the direction of the arrow, and the magnitude of the flow is indicated by the size of the arrow. In addition, the direction and size may be represented not by arrows but by color shading, and is not limited to a specific representation.

9A and 9B, two-dimensional MAP information has been described as an example, but the MAP information may be three-dimensional. That is, the format of the MAP information is not limited to a specific format.
Then, the display unit 108 displays numerical information related to the image or MAP on which the information is drawn and the posted object. Further, the display is not necessarily performed, and the generated flow line or person flow may be recorded by a recording unit (not shown).

  An example of the posted object effect measurement process by the posted object effect measurement system 100 according to the present embodiment described above will be described with reference to the flowchart of FIG. The processing in each step shown in FIG. 2 is as described above, and will be briefly described below.

In S201, the image acquisition unit 101 acquires an image for one frame.
In S202, a control unit (not shown) of the present system determines whether or not an end condition of the present process is satisfied. As a result of this determination, if the end condition is satisfied, that is, if the image acquisition unit 101 cannot acquire an image, the process ends. On the other hand, if the end condition is not satisfied, the process proceeds to S203.

In S <b> 203, the tracking unit 102 performs image processing on the frame image acquired from the image acquisition unit 101 and executes tracking processing.
In S204, the tracking unit 102 determines whether one or more tracking targets have been found or tracked in the tracking process in S203. As a result of this determination, when one or more tracking targets are found or tracked, the process proceeds to S205, and otherwise, the process proceeds to S209.

  In S205, the visual recognition determination unit 103 acquires all tracking positions obtained in the frame of interest from the tracking information generated by the tracking unit 102, and executes a determination process of whether or not each person has visually recognized the posted object. .

  In S206, the visual recognition determination unit 103 determines whether or not one or more tracking targets are determined to have visually recognized the posted object in the determination processing in S205. If it is determined that one or more tracking targets have visually recognized the posted object, the process proceeds to S207. Otherwise, the process proceeds to S208.

In S <b> 207, the person attribute extraction unit 104 extracts the attribute information of the person determined to have viewed the posted object in the visual recognition determination unit 103 from the frame image determined to have viewed the posted object.
In S <b> 208, the visit determination unit 105 acquires all person positions obtained in the frame of interest from the tracking information generated by the tracking unit 102, and determines whether each person has visited a specific point.

In S209, the extraction unit 106 identifies the person ID based on the visual recognition determination information and the visit determination information, thereby extracting the tracking information related to the person ID.
In S210, the conversion unit 107 converts the tracking information extracted by the extraction unit 106 into a person's movement trajectory or a flow of a plurality of persons.

In step S211, the display unit 108 draws information on the image obtained by the extraction unit 106 and the conversion unit 107 on the image or displays the information in the form of a user.
In the present embodiment, the process from the acquisition of the image in S201 to the display of the result in S211 is described as a batch process. However, the process from the acquisition of the image in S201 to the visit determination in S208 may be performed in advance, and the information extraction in S209 and the conversion of the tracking information in S210 may be performed together later.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

100 posted object effect measurement system 101 image acquisition unit 102 tracking unit 103 visual recognition determination unit 104 person attribute extraction unit 105 visit determination unit 106 extraction unit 107 conversion unit 108 display unit

Claims (17)

An image acquisition process for acquiring a video or image of the surveillance area;
An attribute extraction step of extracting attribute information of a person existing in the monitoring area;
A tracking step of performing identification processing of a person using the attribute information and generating tracking information for each person by tracking processing;
A visual recognition determination step of determining whether the person has viewed the posted object and generating visual determination information;
A visit determination step of determining whether the person has visited a specific point and generating visit determination information;
An extraction step for extracting relevant tracking information based on the visual recognition determination information generated in the visual recognition determination step or the visit determination information generated in the visit determination step;
An image processing method comprising: a conversion step of converting the tracking information extracted in the extraction step into a flow line or flow of a person. An image acquisition process for acquiring a video or image of the surveillance area;
A tracking step of generating tracking information of a moving object existing in the monitoring area;
A visual recognition determination step of estimating a visual recognition range of the moving object, determining whether or not a posting object exists in the visual recognition range, and generating visual recognition determination information of the moving object;
Determining whether or not the moving object has visited a specific point, and generating visit determination information for the moving object; and
A conversion step of converting the tracking information of the moving object specified based on the visual recognition determination information generated in the visual recognition determination step and the visit determination information generated in the visit determination step into a moving line of the moving object or a flow of the moving object. An image processing method. An image acquisition process for acquiring a video or image of the surveillance area;
A tracking step of generating tracking information of a moving object existing in the monitoring area;
A visual determination step of determining whether or not the moving object is present in a viewable region set based on a position of the posting object, and generating visual determination information of the moving object;
Determining whether or not the moving object has visited a specific point, and generating visit determination information for the moving object; and
A conversion step of converting the tracking information of the moving object specified based on the visual recognition determination information generated in the visual recognition determination step and the visit determination information generated in the visit determination step into a moving line of the moving object or a flow of the moving object. An image processing method.   The image processing method according to claim 1, further comprising: a display step of drawing the information obtained in the extraction step or the conversion step on the image or converting the information to a user.   The attribute extraction step extracts attribute information of a person determined to have visually recognized the posted object in the visual recognition determination step from a frame image determined to have visually recognized the posted object. Image processing method.   The visit determination step is characterized in that the visit determination information is generated by acquiring all person positions obtained in a frame of interest from tracking information generated in the tracking step for a frame image. 2. The image processing method according to 1.   The image processing method according to claim 1, wherein the extracting step extracts tracking information relating to the person ID by specifying a person ID based on the visual recognition determination information and the visit determination information. Image acquisition means for acquiring a video or image of the surveillance area;
Attribute extracting means for extracting attribute information of a person existing in the monitoring area;
A tracking unit that performs identification processing of a person using the attribute information and generates tracking information for each person by tracking processing;
Visibility determination means for determining whether or not the person has viewed the posted object and generating visual determination information;
Visit determination means for determining whether or not the person has visited a specific point and generating visit determination information;
Extraction means for extracting relevant tracking information based on the visual recognition determination information generated by the visual recognition determination means or the visit determination information generated by the visit determination means;
An image processing apparatus comprising: conversion means for converting the tracking information extracted by the extraction means into a person's flow line or a person's flow. Image acquisition means for acquiring a video or image of the surveillance area;
Tracking means for generating tracking information of a moving object existing in the monitoring area;
Visual recognition determining means for estimating a visual recognition range of the moving object, determining whether or not a posted object exists in the visual recognition range, and generating visual determination information of the moving object;
Determining whether or not the moving object has visited a specific point, and visit determining means for generating visit determination information of the moving object;
Conversion means for converting the tracking information of the moving object specified based on the visual recognition determination information generated by the visual recognition determination means and the visit determination information generated by the visit determination means into a moving line of the moving object or a flow of the moving object. An image processing apparatus. Image acquisition means for acquiring a video or image of the surveillance area;
Tracking means for generating tracking information of a moving object existing in the monitoring area;
Visual determination means for determining whether or not the moving object is present in a viewable region set based on the position of the posting object, and generating visual determination information of the moving object;
Determining whether or not the moving object has visited a specific point, and visit determining means for generating visit determination information of the moving object;
Conversion means for converting the tracking information of the moving object specified based on the visual recognition determination information generated by the visual recognition determination means and the visit determination information generated by the visit determination means into a moving line of the moving object or a flow of the moving object. An image processing apparatus.   9. The image processing apparatus according to claim 8, further comprising display means for drawing information on the image obtained by the extracting means or the converting means on an image or displaying the information in a user form.   9. The attribute extraction unit extracts attribute information of a person determined to have viewed the posted object by the visual determination unit from a frame image determined to have viewed the posted object. Image processing apparatus.   The said visit determination means acquires all the person positions obtained by the attention frame from the tracking information generated by the tracking means for a certain frame image, and generates the visit determination information. The image processing apparatus according to 8.   The image processing apparatus according to claim 8, wherein the extraction unit extracts tracking information related to the person ID by specifying a person ID based on the visual recognition determination information and the visit determination information. An image acquisition procedure for acquiring a video or image of the surveillance area;
An attribute extraction procedure for extracting attribute information of a person existing in the monitoring area;
A tracking procedure for performing person identification processing using the attribute information and generating tracking information for each person through tracking processing;
A visual recognition determination procedure for determining whether the person has viewed the posted object and generating visual determination information;
A visit determination procedure for determining whether or not the person has visited a specific point and generating visit determination information;
An extraction procedure for extracting relevant tracking information based on the visual recognition determination information generated in the visual recognition determination procedure or the visit determination information generated in the visit determination procedure;
A program for causing a computer to execute a conversion procedure for converting tracking information extracted in the extraction procedure into a flow line or flow of a person. An image acquisition procedure for acquiring a video or image of the surveillance area;
A tracking procedure for generating tracking information of a moving object existing in the monitoring area;
A visual recognition determination procedure for estimating the visual recognition range of the moving object, determining whether or not a posting object exists in the visual recognition range, and generating visual recognition determination information of the moving object;
Determining whether or not the moving object has visited a specific point, and a visit determination procedure for generating visit determination information of the moving object;
A conversion procedure for converting tracking information of a moving body specified based on the viewing determination information generated in the visual determination procedure and the visit determination information generated in the visit determination procedure into a moving line of the moving body or a flow of the moving body. A program characterized by being executed. An image acquisition procedure for acquiring a video or image of the surveillance area;
A tracking procedure for generating tracking information of a moving object existing in the monitoring area;
A visual determination procedure for determining whether or not the moving object is present in a viewable region set based on the position of the posting object, and generating visual determination information for the moving object;
Determining whether or not the moving object has visited a specific point, and a visit determination procedure for generating visit determination information of the moving object;
A conversion procedure for converting tracking information of a moving body specified based on the viewing determination information generated in the visual determination procedure and the visit determination information generated in the visit determination procedure into a moving line of the moving body or a flow of the moving body. A program characterized by being executed.