JP2012234285A - Image analysis device, image analysis method, image analysis program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device, a method, and the like for improving the comprehension of an analysis result by more accurately analyzing a motion of an analysis target.SOLUTION: An image analysis device SS extracts vector information existing in a rectangular region 20, and based on directions included in the extracted vector information existing in a predetermined range, calculates one or plural representative directions of the vector information existing in the predetermined range.

Description

  The present invention relates to a technical field such as an apparatus and method for analyzing an object displayed in an image.

  Conventionally, a person using a predetermined facility (transportation, commercial facility, public institution, etc.) is captured as a moving image, and the movement (flow) of the person indicated in the moving image is analyzed by applying a tracking process. It is done to grasp human behavior without using human means.

  As an example of such a tracking process, there is a tracking process in which an analysis target indicated in a reference frame among frames constituting a moving image is specified by labeling, and a tracking process in which the specified analysis target tracks a movement indicated by another frame is included. It is known (see, for example, Patent Document 1).

  Then, by displaying the result of this tracking process as a flow line or vector indicating the moving direction for each analysis target, it is possible to visually grasp the human behavior and to grasp the analysis result (ease of grasping and recognition). ) Has been disclosed.

  For example, Patent Document 2 discloses a technique for displaying monitoring information in a designated area as a flow line.

  Patent Document 3 discloses a technique for obtaining the trajectory of each passerby in a space and displaying the trajectories of all passers-by who pass within a predetermined time in a predetermined space.

  Patent Document 4 discloses a technique for displaying an arrow (vector) indicating a human flow on an image.

Japanese Examined Patent Publication No. 6-28449 JP 2003-101994 A JP 2005-346617 A JP 2007-243342 A

  However, in the techniques disclosed in Patent Documents 1 to 4, vectors or flow lines corresponding to all the analysis targets shown in the moving image are displayed. Therefore, since the number of flow lines or vectors displayed with the increase in the number of analysis objects also increases, when many analysis objects are displayed, the direction and size of individual vectors corresponding to each analysis object, Or it becomes difficult to identify the trajectory or quantity of the flow line, and the graspability for the analysis result is remarkably lowered.

  Furthermore, in the technique for displaying the flow line disclosed in Patent Document 3, it is difficult to grasp the direction and quantity in which the analysis target moves, and thus the movement indicated by the analysis target cannot be grasped in detail (analysis). It is lacking to represent the dynamic element of interest).

  In addition, in the technique for displaying a vector disclosed in Patent Document 4, the direction of the vector indicating the direction of motion of the analysis target is extremely limited, and thus the operation indicated by the analysis target cannot be grasped in detail ( The direction of the vector is uniquely determined).

  Therefore, the present invention has been made in view of the above problems, and an example of the object thereof is to provide an apparatus, a method, and the like that analyze the motion of the analysis target more accurately and improve the graspability of the analysis result. It is to be.

  In order to solve the above-mentioned problem, the invention described in claim 1 is an image analysis apparatus for analyzing a motion of an analysis target imaged as a moving image, and stores a moving image imaged in an imaging region for a certain period. Storage means, detection means for detecting the analysis object indicated in the moving image, and the detected analysis object calculates a positional change indicated in the imaging region, and the analysis object is distinguished from other analysis objects Position change calculating means for storing in association with analysis target identification information for performing the detection, and based on the stored position change, the detected analysis target indicates a position change per unit time indicated in the imaging region. A vector information calculating means for calculating each piece of analysis object identification information as vector information including a position and a direction in the imaging area, and a position of the imaging area based on the position included in the vector information. One or more vector information extracting means for extracting vector information existing in the range, and one or more vector information existing in the predetermined range based on the direction included in the extracted vector information existing in the predetermined range; It is characterized by comprising movement state calculating means for calculating a representative direction, and display control means for displaying the calculated representative direction to the outside.

  Accordingly, the movement of the analysis target can be analyzed more accurately, and the graspability for the analysis result can be improved.

  The invention according to claim 2 is the image analysis apparatus according to claim 1, wherein the vector information calculation unit calculates a movement start point of the analysis target as the vector information, and is included in the vector information The vector information existing within a predetermined range of the imaging region is extracted based on the movement start point to be extracted.

  Therefore, it is possible to accurately analyze the movement of the analysis target that enters the predetermined range of the imaging region, and to improve the graspability of the analysis result.

  A third aspect of the present invention is the image analysis apparatus according to the first or second aspect, wherein the vector information calculation means calculates a movement end point of the analysis target as the vector information, and the vector information Vector information existing within a predetermined range of the imaging region is extracted based on a movement end point included in the image pickup area.

  Therefore, it is possible to accurately analyze the movement of the analysis target that exits from the predetermined range of the imaging region, and to improve the graspability of the analysis result.

  A fourth aspect of the present invention is the image analysis apparatus according to any one of the first to third aspects, wherein the movement state calculating means is included in the extracted vector information within the predetermined range. The direction axis indicating the predetermined direction centered on the origin approximates the direction indicated by any one of the direction axes in the first coordinate system provided at a predetermined angle apart from the origin. The moving state calculation means calculates the direction included in the vector information and the direction indicated by the direction axis determined to be approximate as the representative direction of the vector information existing within the predetermined range. Features.

  Accordingly, since the representative direction of the vector information existing within the predetermined range is determined according to a predetermined direction, the processing load on the apparatus (CPU or the like) is reduced and the graspability for the analysis result is improved. be able to.

  A fifth aspect of the present invention is the image analysis apparatus according to any one of the first to fourth aspects, wherein the movement state calculating means is included in the extracted vector information within the predetermined range. The direction axis indicating the origin from the predetermined direction is approximate to the direction indicated by any of the direction axes in the second coordinate system provided at a predetermined angle around the origin. The moving state calculating means calculates the direction included in the vector information and the direction indicated by the direction axis determined to be approximate as the representative direction of the vector information existing within the predetermined range. .

  Accordingly, since the representative direction of the vector information existing within the predetermined range is determined according to a predetermined direction, the processing load on the apparatus (CPU or the like) is reduced and the graspability for the analysis result is improved. be able to.

  A sixth aspect of the present invention is the image analysis apparatus according to any one of the first to fifth aspects, wherein the vector information extracting means extracts at least two or more vector information, and the display control means. Is characterized in that the display mode of the representative direction to be displayed to the outside is changed in accordance with the quantity of vector information belonging to the calculated representative direction.

  Therefore, since the display mode of the representative direction to be displayed changes according to the calculated quantity of vector information belonging to the representative direction, detailed information on the analysis result can be easily grasped visually. can do.

  A seventh aspect of the present invention is the image analysis apparatus according to any one of the first to sixth aspects, wherein the vector information extracting means extracts at least two or more vector information, and the display control means. Is characterized in that the display mode of the representative direction to be displayed to the outside is changed according to the magnitude of the position change included in the vector information belonging to the calculated representative direction.

  Therefore, the display mode of the representative direction to be displayed to the outside changes according to the magnitude of the position change included in the calculated vector information belonging to the representative direction. It can be easily grasped visually.

  The invention according to claim 8 is the image analysis apparatus according to any one of claims 1 to 7, wherein the vector information extraction means extracts at least two or more vector information, and the display control means. Is characterized in that, when the quantity of vector information belonging to the representative direction calculated by the movement state calculating means is greater than or equal to a predetermined threshold value, the calculated representative direction is displayed to the outside.

  Therefore, when the calculated amount of vector information belonging to the representative direction is greater than or equal to a predetermined threshold, the calculated representative direction is displayed to the outside, so that an analysis result according to the user's needs is presented. can do.

  The invention according to claim 9 is the image analysis apparatus according to any one of claims 1 to 8, wherein the vector information extraction means extracts at least two or more vector information, and the display control means. Is characterized by extracting a representative direction having the largest quantity of vector information belonging to the representative direction calculated by the movement state calculating means and displaying the representative direction to the outside.

  Therefore, a representative direction with the largest quantity of vector information belonging to the calculated representative direction is extracted, and the representative direction is displayed to the outside, so that an analysis result according to the user's needs is presented. be able to.

  A tenth aspect of the present invention is the image analysis device according to any one of the first to ninth aspects, wherein the vector information extracting means is at least two or more based on a position included in the vector information. Vector information existing within a predetermined range of the imaging area of different sizes is extracted for each predetermined range of the imaging area, and the movement state calculation means is included in the extracted vector information existing within the predetermined range The representative direction of the vector information existing within the predetermined range is calculated for each predetermined range of the imaging region based on the determined direction.

  Therefore, the vector information existing in the predetermined range of the imaging area of at least two or more different sizes is extracted for each predetermined range of the imaging area, and the direction included in the extracted vector information existing in the predetermined range Based on the above, since the representative direction of the vector information existing within the predetermined range is calculated for each predetermined range of the imaging region, an analysis result according to the user's need can be presented.

  The invention according to an eleventh aspect is the image analysis apparatus according to any one of the first to twelfth aspects, wherein the vector information extracting unit divides the imaging region into regions of a predetermined size. Vector information existing in each divided region is calculated for each divided region, and the movement state calculation means determines the one or more representative directions based on the vector information calculated for each divided region. The calculation is performed for each divided area, and the display control unit displays one or more representative directions corresponding to the calculated divided areas in each divided area.

  The invention according to claim 12 is the image analysis apparatus according to claim 11, wherein a predetermined number of the divided areas are defined as a middle area, and the representative area is calculated for each middle area. The movement state calculation means selects one or more representative directions for each small area belonging to the middle area according to the quantity set for each middle area. The display control means calculates and displays one or more representative directions in the small area for each small area to which the calculated middle area belongs.

  A thirteenth aspect of the present invention is the image analysis device according to any one of the first to twelfth aspects, further comprising input means capable of arbitrarily setting a predetermined range of the imaging region. And

  Therefore, an analysis result according to the user's need can be presented.

  The invention according to claim 14 is an image analysis method in an image analysis apparatus that analyzes a motion of an analysis target captured in a moving image, and a storage step of storing a moving image captured in a capturing area for a certain period; A detection step of detecting the analysis target indicated in the moving image, and vector information including a position and a direction in the imaging region, indicating a change in position of the detected analysis target per unit time indicated in the imaging region. Calculated based on the tracking step of storing the analysis target in association with the analysis target identification information for identifying the analysis target from the other analysis target, and within the predetermined range of the imaging region based on the position included in the vector information Based on a vector information extraction step for extracting existing vector information and a direction included in the extracted vector information within the predetermined range, the vector information is present within the predetermined range. A moving state calculation step of calculating one or more representative direction of the vector information, the representative direction the calculated, and having a display control step of displaying to the outside.

  According to a fifteenth aspect of the present invention, there is provided a storage unit for storing a moving image obtained by capturing an image of an imaging region for a certain period, a computer included in an image analysis apparatus that analyzes a motion of an analysis target captured in a moving image, and the moving image Detecting means for detecting the analysis target shown in FIG. 1, calculating a position change per unit time indicated by the detected analysis target in the imaging region as vector information including a position and a direction in the imaging region, Position change calculating means for storing the analysis target in association with analysis target identification information for identifying the analysis target from other analysis target, a vector existing within a predetermined range of the imaging region based on the position included in the vector information A vector information extracting means for extracting information, based on a direction included in the extracted vector information within the predetermined range, a vector existing within the predetermined range; Moving state calculation means for calculating one or more representative direction of torque information, the representative direction the calculated, characterized in that to function as display control means, for displaying to the outside.

  The invention according to claim 16 is characterized in that the image processing program according to claim 15 is recorded so as to be readable by a computer.

  As described above, according to the present invention, the image analysis apparatus calculates the representative direction of the vector information existing within the predetermined range, so that the motion of the analysis target is analyzed more accurately, and the graspability for the analysis result is improved. Can be improved.

It is a block diagram which shows the structure of image analysis apparatus SS concerning this embodiment. It is a flowchart which shows the detail of operation | movement of image analysis apparatus SS until the analysis object detected from the moving image memorize | stores the position change shown in an imaging region. 4 is a diagram illustrating an example of a frame stored in an input image storage unit 5. FIG. It is a conceptual diagram which shows a person's movement within a predetermined period. It is a figure which shows an example of the content memorize | stored in position coordinate DB. It is a flowchart which shows the detail of operation | movement of image analysis apparatus SS until it displays the typical direction of the vector information which exists in the predetermined range of an imaging region based on the calculated vector information. It is a conceptual diagram which shows the example of calculation of the position change per unit time of the person H1. It is a conceptual diagram which shows the calculation result of the quantity vector information corresponding to the quantity of the position change per unit time of the person H1. It is a conceptual diagram which shows the calculation result at the time of calculating vector information with respect to all the people shown by a moving image. 3 is a conceptual diagram illustrating an extraction example in which vector information existing in a rectangular area 20 is extracted. FIG. It is a figure which shows a 1st coordinate system. It is a conceptual diagram which shows the method of determining an approximation. 7 is a diagram illustrating a determination result of determining whether a direction included in moved vector information corresponds to a range approximated by any direction axis in the first coordinate system T. FIG. 6 is a diagram showing a display example in which representative directions of vector information existing in a rectangular area 20 are displayed. FIG. It is a conceptual diagram which shows the display result at the time of changing the display mode of a typical direction according to the quantity of the vector information which belongs to the direction axis which shows a typical direction. It is a conceptual diagram which shows the display result at the time of changing the display mode of a typical direction according to the position change contained in the vector information which belongs to the direction axis which shows a typical direction. Concept showing the display result when the display mode of the representative direction is changed according to the quantity of vector information belonging to the direction axis indicating the representative direction and the magnitude of the position change included in the vector information FIG. A display example is shown in a case where a representative direction is calculated based on vector information existing in each divided area, and one or more representative directions corresponding to each calculated divided area are displayed in each divided area. FIG. It is a conceptual diagram which shows the example of a setting of a middle area | region. It is a figure which shows the example of a display when the movement state calculation part 11 displays the calculation result at the time of calculating a typical direction with reference to a middle area | region. It is a figure which shows the example of a display when setting the predetermined range of the imaging area of two or more different sizes, and calculating a typical direction based on the direction contained in the vector information which exists in the said area | region. It is a figure which shows the example at the time of setting an attribute label in the predetermined range of two or more imaging areas. It is a figure which shows a 2nd coordinate system. It is a conceptual diagram which shows the case where the vector information per unit time which an analysis object shows is calculated with an approximate curve.

  First, the best embodiment of the present application will be described with reference to the accompanying drawings. The embodiment described below is an embodiment in a case where the present application is applied to an image analysis apparatus that analyzes a motion of an analysis target captured as a moving image.

[1. Overview of image analysis device configuration and functions]
First, the configuration and functional overview of the image analysis apparatus SS according to the present embodiment will be described with reference to FIG.

  FIG. 1 is a block diagram showing the configuration of the image analysis apparatus SS according to the present embodiment.

  The image analysis apparatus SS includes an image input unit 1 that receives input of a moving image as input data, a keyboard, a touch panel, a jog dial, or other input mechanism, and an input unit 2 that outputs information input from a user to the control unit 3; A control unit 3 that generates an image or a moving image as output data from input data, a display unit 4 that includes a display and the like and displays output data.

  More specifically, the image input unit 1 receives an input of an electric signal from a video camera (digital video camera) (not shown) that captures a moving image in the imaging region and outputs a predetermined electric signal.

  In the present embodiment, the video camera is installed in a predetermined facility (transportation facility, commercial facility, public institution, etc.), etc., and for a certain period (for example, every hour, or continuously for 24 hours) The inside of the imaging area is imaged.

  The video camera is a well-known technique and will not be described in detail. For example, the light taken in from the optical lens is adjusted by a diaphragm mechanism or the like, and then a CCD (Charge Couple Semiconductor) image sensor or CMOS (Complementary Metal). -Oxide Semiconductor) An optical image (image) is formed on the image sensor. This image is converted into an electrical signal by the image sensor and output to the image input unit 1 (camera I / F).

  Then, the image input unit 1 receives an electrical signal output from the video camera, and pixel information (such as an electrical signal corresponding to each pixel) indicating pixels constituting the image and color information (R) corresponding to the pixels. , G, and B are output as still images (frames to be described later) to the control unit 3 (input image storage unit 5 to be described later) in association with the imaging time. .

  In other words, the image input unit 1 converts the input moving image into image information including a still image of a predetermined frame (for example, 20 frames per second (that is, a frame rate is 20 fps (frame per second))), For example, the image is output to the control unit 3 in association with the imaging time. That is, these frames constitute a moving image.

  As an example of the image input unit 1, for example, an interface corresponding to a well-known camera link (Camera Link) which is a standard specification for connecting an industrial digital camera and an image input board is applied.

  As will be described in detail later, the control unit 3 calculates, as vector information, a position change per unit time indicated in the imaging region by the analysis target indicated in the moving image, and within the predetermined range of the imaging region based on the vector information. Output data for displaying a representative direction of the vector information existing in is generated.

  Such an operation (processing) of the control unit 3 may be performed by a hardware circuit mounted in the control unit 3, or a microcomputer (CPU or the like) mounted in the control unit 3. It may be performed by executing a predetermined program (software).

  More specifically, the control unit 3 includes an input image storage unit 5 (an example of the storage unit of the present application), a moving object detection unit 6 (an example of the detection unit of the present application), and a position change calculation unit 7 (of the position change calculation unit of the present application). An example), a position coordinate storage unit 8 (an example of the position change calculation unit of the present application), a vector information calculation unit 9 (an example of the vector information calculation unit of the present application), and a vector information extraction unit 10 (an example of the vector information extraction unit of the present application). , A movement state calculation unit 11 (an example of the movement state calculation unit of the present application), a display control unit 12 (an example of the display control unit of the present application), an output frame memory 13, and the like.

  The input image storage unit 5 is a storage area for storing a moving image captured in the imaging area for a certain period. For example, a volatile or nonvolatile memory or a hard disk is applied.

  Specifically, the input image storage unit 5 in the present embodiment includes one or a plurality of frames (still images converted from the above-described moving image to a predetermined frame), imaging times in each frame, and coordinate values described later in detail. Are stored in association with each other.

  The input image storage 5 stores a plurality of frames corresponding to moving images captured for a certain period.

  The moving body detection unit 6 detects an analysis target indicated in the moving image.

  The captured moving image shows a person who uses the facility, a moving means (for example, a car, a loading platform, etc.) passing through the facility, and the like. The moving body detection unit 6 in the present embodiment detects this person, moving means, and the like as an analysis target.

  Since the method for detecting the analysis target is a known technique, a detailed description thereof is omitted. However, the moving object detection unit 6 uses the reference frame (the same as the reference frame stored in advance) as the reference frame among the frames constituting the moving image. Subtraction between pixels with a still image in the imaging region) and so-called background removal are performed, and the image is converted to an image having only pixels that have changed. Then, when image processing such as noise removal and binarization is performed, an analysis target of the reference frame is extracted. Then, a labeling process is performed to identify the extracted analysis object from other analysis objects, and a process of assigning a moving object ID (an example of analysis object identification information of the present application) to each analysis object is performed.

  This moving object ID is identification information for identifying the analysis target from other analysis targets.

  The position change calculation unit 7 calculates a position change indicated by the detected analysis target in the imaging region.

  For example, a known tracking process can be applied to the calculation of the position change.

  Since the tracking process is a well-known technique, a detailed description thereof will be omitted. First, the position change calculation unit 7 determines the position in the imaging region indicated by the reference frame as the analysis target specified by the reference frame. Identify.

  Next, the position change calculation unit 7 determines whether the same object as the analysis target is specified in a different frame (a frame captured after the reference frame).

  As a method for specifying the same object, for example, a shape parameter including the size of the analysis target specified in the reference frame and a ratio of the main axes is obtained, and the shape parameter is most approximated in other frames. A method of considering an object having a shape parameter as the same analysis target can be applied.

  When it is determined that they are the same object, the position change calculation unit 7 specifies the position in the imaging region indicated by the different frame as the analysis target in the different frame.

  Then, the position change calculation unit 7 detects the position in the imaging region indicated by the frame in which the identified analysis target is a reference and the position in the imaging region indicated by a frame in which the specified analysis target is different as the detected analysis target It is calculated as a position change indicated in the imaging area.

  This change in position can be calculated for each analysis target for all consecutive frames of a moving image captured for a certain period. In other words, the position change calculation unit 7 can calculate, as a position change, the position in the imaging region indicated by the analysis target indicated by the moving image in each of the continuous frames.

  By calculating the position change, the analysis object can grasp the position change (movement amount, movement direction) indicated in the imaging region.

  The position coordinate storage unit 8 is a storage area for storing the calculated position change in association with the moving object ID for each analysis target. For example, a volatile or nonvolatile memory or a hard disk is applied.

  Based on the stored position change, the vector information calculation unit 9 uses the detected analysis target position change per unit time indicated in the imaging area as vector information including the position and direction in the imaging area. Calculated for each moving object ID.

  Since the calculation of the vector information is a known technique, detailed description thereof is omitted, but is calculated as follows.

  The frame described above is obtained by dividing a moving image at a predetermined interval (predetermined period) (for example, 20 frames per second as described above). Then, as described above, the position change calculation unit 7 calculates the position change indicated by the analysis target in the imaging region for all the continuous frames and stores the position change in the position coordinate storage unit 8.

  Then, the vector information calculation unit 9 first uses the position change stored in the position coordinate storage unit 8 and indicated by the analysis target within the imaging region to calculate the position change per unit time indicated by the analysis target within the imaging region. calculate.

  Specifically, for example, when the frame is a moving image divided at 20 frames per second (hereinafter simply referred to as “frame rate is 20 fps”), the vector information calculation unit 9 performs the position change. The analysis target is calculated by calculating the position in the imaging region indicated by the reference frame specified by the calculation unit 7 and the position in the imaging region indicated by the 21st frame from the reference frame. Calculates a position change per unit time indicated in the imaging region.

  As described above, since a frame is obtained by dividing a moving image at a predetermined interval, a position change per predetermined interval of the analysis target can be calculated by calculating a position change indicated by the analysis target between arbitrary frames. Can do it.

  Then, the vector information calculation unit 9 ends the movement of the position in the imaging region indicated by the frame corresponding to the 20th frame from the reference frame, as the analysis target position in the imaging region indicated by the reference frame. As a point, vector information per unit time indicated by the analysis target is calculated.

  This vector information is obtained by calculating the position change per unit time indicated by the detected analysis target in the imaging region as information including the position and direction in the imaging region. And vector information. Further, the vector information may be calculated in the tracking process described above.

  In the above-described example, the unit time is assumed to be 1 second. However, the present invention is not limited to this. For example, the unit time can be set to an arbitrary time.

  For example, consider the case where the unit time is 5 seconds. In this case, when the frame rate is 20 fps, the vector information calculation unit 9 specifies the position in the imaging region indicated by the frame based on the analysis target specified by the position change calculation unit 7, the reference, By calculating the position in the imaging region indicated between the frames corresponding to the 100th frame from the frame, the change in position per unit time indicated by the analysis target in the imaging region is calculated.

  The vector information extraction unit 10 extracts, for each moving body ID, vector information existing within a predetermined range of the imaging area based on the position included in the vector information.

  Based on the direction included in the vector information existing within the predetermined range extracted by the vector information extraction unit 10, the movement state calculation unit 11 includes one or more representative directions of the vector information existing within the predetermined range. Is calculated.

  The display control unit 12 displays the representative direction calculated by the movement state calculation unit 11 to the outside.

  The output frame memory 13 performs control for writing output data.

[2. Operation of the image analysis apparatus SS until the analysis target detected from the moving image stores the position change indicated in the imaging region]
Next, details of the operation of the image analysis apparatus SS until the analysis target detected from the moving image stores the position change indicated in the imaging region (mainly the operation of the moving object detection unit 6 and the position change calculation unit 7). This will be described with reference to FIGS.

  FIG. 2 is a flowchart showing details of the operation of the image analysis apparatus SS until the analysis target detected from the moving image stores the position change shown in the imaging region.

  As shown in FIG. 2, when a frame associated with the imaging time output from the image input unit 1 is input (step S1), the frame and the imaging time in each frame are associated with each other to store the input image. Stored in the unit 5.

  Furthermore, the coordinate values described above are stored in association with this frame.

  This coordinate value is a value for discriminating a predetermined position in the imaging area indicated by the frame from other positions, for example, a plane having the horizontal direction of the frame as the X axis and the vertical direction of the frame as the Y axis. It is represented by X and Y coordinates in the coordinate system.

  Here, an example of a frame stored in the input image storage unit 5 will be described with reference to FIG.

  FIG. 3 is a diagram illustrating an example of a frame stored in the input image storage unit 5.

  A frame F shown in FIG. 3 is one of the frames converted from a moving image captured in a predetermined period in a predetermined imaging area of the shopping mall S where the stores S1 to S4 are installed.

  In this frame F, people who use the shopping mall S are shown, and the coordinate values unified in all the frames are set.

  With this coordinate value, the position of an analysis target (a person, a store, etc., which will be represented as “person” hereinafter) shown in the frame can be represented by an X coordinate value and a Y coordinate value. Therefore, each person or store indicated in the frame can specify the position in the imaging region indicated in the frame.

  The input image storage unit 5 stores a plurality of frames corresponding to moving images captured for a certain period.

  Next, the moving object detection unit 6 detects each person as an analysis target and assigns a moving object ID to each detected person (step S2). For example, a moving object ID is assigned to each person shown in the frame F of FIG.

  Next, the position change calculation unit 7 performs the tracking process (moving body tracking) described above, and calculates the detected position change of the person. (Step S3).

  Here, a calculation example of the position change will be described with reference to FIG.

  FIG. 4 is a conceptual diagram showing movement of a person within a predetermined period.

As shown in FIG. 4, for example, when a person H1 (an example of an analysis target) moves from V _1s (x1, y1) to V _10e (x11, y11) as a position (up to an arrow indicated by a broken line in the figure) Move).

In this case, the position change calculation unit 7 indicates that the person H1 is located at V _10e (x11, y11) with reference to the frame in which the person H1 is located at V _1s (x1, y1). The position indicated by the person H1 for each frame is calculated as a coordinate value up to the next frame. This calculated coordinate value is the position change that the person H1 shows in the imaging region.

  The position change calculation unit 7 can also calculate position changes for all persons (all moving body IDs) shown in the moving image.

  Next, the position coordinate storage unit 8 stores the position change calculated for the person H1 as a position coordinate DB in association with the moving object ID indicating the person H1 (step S4).

  FIG. 5 is a diagram illustrating an example of contents stored in the position coordinate DB.

  The position coordinate DB illustrated in FIG. 5 includes a moving object ID (for example, a person H1), a position coordinate as a position indicated by the person H1 indicated by the moving object ID in the imaging region, and a time at the position (imaging time when the image is captured as a frame). ) Are stored in association with each other.

  Thus, by storing the position indicated by the moving object ID in the imaging area in association with the position indicated by the person H1 and the time at the position, the person H1 can store the change in position indicated in the imaging area.

  In the position coordinate DB, position changes calculated for all persons shown in the moving image can be stored.

  If there is an end instruction (step S5: YES), the process ends. If there is no end instruction (step S5: NO), the process proceeds to step S1.

[3. Operation of image analysis apparatus SS until one or more representative directions of vector information existing within a predetermined range of the imaging region are displayed based on the calculated vector information]
Next, details of the operation of the image analysis apparatus SS until displaying one or more representative directions of vector information existing within a predetermined range of the imaging region based on the calculated vector information will be described with reference to FIGS. Will be described.

  FIG. 6 is a flowchart showing details of the operation of the image analysis apparatus SS until the representative direction of the vector information existing within the predetermined range of the imaging region is displayed based on the calculated vector information.

  As shown in FIG. 6, first, the vector information calculation unit 9 causes the person H1 to move from the captured moving image within the imaging area based on, for example, an instruction of the input unit 2 by the user or based on a preset instruction. The position change for a predetermined period indicated by is acquired (step S11).

  Specifically, the vector information calculation unit 9 refers to the position coordinate DB, and the moving object ID acquires the position coordinates indicated in the predetermined period. For example, when 1 hour is set as the predetermined period, the vector information calculation unit 9 refers to the time stored in association with the moving object ID and the position coordinates stored in association with the time. Position coordinates for one hour are acquired as position changes for a predetermined period.

  The position change for the predetermined period may be acquired for one moving object ID or may be acquired for all moving object IDs.

  Note that the vector information calculation unit 9 is not limited to the position change acquired in the above-described period, and may acquire all position changes shown in the moving image, for example.

  Next, the vector information calculation unit 9 calculates a position change per unit time based on the acquired position change for a predetermined period (step S12).

  Here, the calculation of the position change per unit time will be described with reference to FIGS.

  FIG. 7 is a conceptual diagram illustrating a calculation example of the position change per unit time of the person H1.

In the above-described FIG. 4, an example in which the position change per unit time when the person H1 moves from V _1s (x1, y1) to V _10e (x11, y11) during a predetermined period will be described. To do.

  Here, the position change per unit time means that the position indicated by the person H1 in the reference frame is the movement start point, and the position indicated by the person H1 is moved in a frame after a unit time (for example, 1 second) from the frame. The position change when it is set as the end point is shown.

Specifically for this calculation method, human H1 _is, in order to determine the position change per unit time in the case of moving from V 1s (x1, y1) to _{V 10e (x11, y11),} V 1s (x1, y1 ) As a reference, the position of the person H1 in a frame every unit time (for example, every 1 second) may be specified.

More specifically, the position indicated by the person H1 in the reference frame is V _1s (x1, y1), and the position indicated by the person H1 in a frame after a unit time from the frame is V _1e (x2, y2). If there is, the position change per unit time is calculated as the position change from V _1s (x1, y1) as the movement start point to V _1e (x2, y2) as the movement end point.

Next, the vector information calculation unit 9 specifies the position of the person H1 in the frame after the unit time based on V _1e (x2, y2), and calculates the position change per unit time. The vector information calculation unit 9, human H1 _is, the position change per unit time until V 10e (x11, y11), successively calculated.

FIG. 7 shows the calculation result of the position change per unit time when the person H1 calculated in this way moves from V _1s (x1, y1) to V _10e (x11, y11).

As shown in FIG. 7, as a change in position per unit time when a person H1 moves from V _1s (x1, y1) to V _10e (x11, y11), first, V _1s (x1, From y1), the first position change per unit time is calculated with the position of the person H1 indicated in the frame after the unit time as V _1e (x2, y2) with the movement end point.

Next, the next position change per unit time with the movement start point as V _2s (x2, y2) and the movement end point as V _2e (x3, y3), and the movement start point as V _3s (x3, y3). ), The next position change per unit time with the movement end point being V _3e (x4, y4), the movement start point is V _10s (x10, y10), and the movement end point is V _10e (x11, y11). The position change per unit time is sequentially calculated up to the last position change per unit time.

In the above-described example, the movement end point of the position change per unit time and the movement start point of the position change per unit time calculated after the position change (for example, V _1e and V _2s ) are Although the same point is shown, in FIG. 7, it represents as a point which mutually contacts for convenience of explanation.

  In the example shown in FIG. 7, for convenience of explanation, the calculated position change per unit time is displayed superimposed on a predetermined frame.

  The position change calculation unit 7 can also calculate position changes for all persons (all moving body IDs) shown in the moving image.

  Next, the vector information calculation unit 9 calculates the calculated position change per unit time as vector information including the position and direction in the imaging region (step S13).

For example, when the vector information focuses on a change in position per unit time where the movement start point is V _1s (x1, y1) and the movement end point is V _1e (x2, y2),
Vector information _{V 1} was generally known to be calculated by (x2-x1, y2-y1 ). Here, the movement start point is a so-called vector start point, and the movement end point is a so-called vector end point.

The vector information V ₁ was, since an amount having a magnitude and direction, can be human H1 is to grasp the position change per unit time indicated in the imaging area.

  Then, the vector information calculation unit 9 calculates a quantity of vector information corresponding to the calculated position change quantity per unit time.

  Here, the calculation result of vector information is demonstrated using FIG.

  FIG. 8 is a conceptual diagram showing a calculation result of the quantity vector information corresponding to the quantity of position change per unit time of the person H1.

As shown in FIG. 8, vector information V ₁ having the movement start point as V _1s (x1, y1) and the movement end point as V _1e (x2, y2) is set as the first vector information, and the movement start point is set as V _2s ( Vector information V ₂ with x2, y2), movement end point V _2e (x3, y3), vector information with movement start point V _3s (x3, y3), and movement end point V _3e (x4, y4) V ₃ is sequentially calculated up to vector information V ₁₀ with the movement start point as V _10s (x10, y10) and the movement end point as V _10e (x11, y11).

  In the example described above, the movement end point of certain vector information and the movement start point of the vector information calculated next to the vector information indicate the same point. However, in FIG. It is expressed as points that touch each other.

  In the example shown in FIG. 8, the calculated vector information is displayed superimposed on a predetermined frame.

  As described above, the vector information is information including a movement start point and a movement end point (position). Therefore, by arranging the movement start point and the movement end point at the positions corresponding to the coordinate values of the frame, the calculated vector information can be displayed superimposed on the corresponding position of the predetermined frame.

  By displaying in this way, it is possible to visually grasp the behavior of a person in the imaging region and improve the graspability of the analysis result.

Further, it is known that the size of the vector information V ₁ is generally calculated by ((x2−x1) ² + (y2−y1) ² ) ^1/2 .

And the magnitude of the vector information V ₁ shown in this frame, the size of the real (real world) in the imaging area, by calculating a correlation, the magnitude of the vector information V ₁ shown in the frame, the imaging area It can be grasped how much size is shown in.

Ie, X represented by the frame, the value of the Y coordinate value, by converting the real world it is sized meters (m) or the like, the magnitude of the vector information V ₁ shown in the frame, in the imaging area It is possible to grasp how large the image is (that is, it is possible to grasp an actual distance in the imaging region).

  Then, the vector information calculation unit 9 for all persons (all moving body IDs) indicated in the moving image for the predetermined period set in step S11 or for a certain period during which the input moving image is captured. Vector information can also be calculated.

  FIG. 9 is a conceptual diagram showing a calculation result when vector information is calculated for all persons shown in the moving image.

  In FIG. 9, vector information is calculated for all people shown in the moving image and is displayed superimposed on a predetermined frame.

  As shown in FIG. 9, when the vector information is calculated for all persons shown in the image and displayed superimposed on a predetermined frame, the vector information displayed as the number of objects to be analyzed increases. Since the number also increases, it becomes difficult to identify the direction and size of individual vector information corresponding to each analysis object, the quantity of vector information, and the like, and the graspability for the analysis result is significantly lowered.

  Therefore, in the present embodiment, vector information existing within a predetermined range of the imaging area is extracted, and a vector existing within the predetermined range is extracted based on a direction included in the extracted vector information within the predetermined range. One or more representative directions of information are calculated and displayed.

  Specifically, upon receiving an instruction for designating a predetermined range in the imaging area based on an instruction of the input unit 2 by the user or based on a preset instruction, the vector information extraction unit 10 Vector information existing within a predetermined range is extracted (step S14).

  More specifically, for example, the rectangular area 20 in FIG. 9 is designated as being within a predetermined range in the imaging area (for example, a coordinate value indicating a fixed point of the rectangular area 20 is input or displayed on the display screen. It is assumed that a rectangular area is drawn with a pointer or the like). Then, the vector information extraction part 10 extracts the vector information which exists in the rectangular area 20 with reference to a coordinate value.

  FIG. 10 is a conceptual diagram illustrating an extraction example in which vector information existing in the rectangular area 20 is extracted.

As shown in FIG. 10, vector information V _{1 to} V _n is extracted as vector information existing in the rectangular area 20.

  In actuality, there are cases where a large number of vector information items are extracted such that they cannot be identified, but in FIG. 10, for the convenience of explanation, the quantity of vector information items is limited. Further, the length of the vector information varies depending on the movement distance per unit time of the person to be analyzed, but in FIG. 10, for the convenience of explanation, the length of the vector information is shown as substantially the same.

Further, as in the above-described example, all vector information includes a movement start point and a movement end point. However, for convenience of explanation in FIG. 10, representative vector information (vector information V ₁ and vector information V _{n is included).} ) Only, and the other illustrations are omitted.

  Various methods can be applied to this vector information extraction method. For example, when a movement start point of vector information exists in the rectangular area 20, the vector information extraction unit 10 extracts the vector information. You may do it.

Specifically, when describing the vector information V ₁ as an example, if the movement start point V _1S vector information V ₁ is included in the rectangular region 20, the vector information extraction unit 10 extracts the vector information.

  As a result, even when only the movement start point of the vector information exists in the rectangular area 20 (that is, vector information indicating a motion that leaves (passes) the rectangular area 20 from now on), the vector information extracting unit 10 Therefore, it is possible to accurately analyze the motion of the analysis object that leaves the rectangular area 20.

  Further, when the movement end point of the vector information exists in the rectangular area 20, the vector information extraction unit 10 may extract the vector information.

Specifically, when the movement end point V _1e of the vector information V ₁ is included in the rectangular area 20, the vector information extraction unit 10 extracts the vector information.

  Thereby, even when only the movement end point of the vector information exists in the rectangular area 20 (that is, vector information indicating a movement entering the rectangular area 20), the vector information extracting unit 10 extracts the vector information. In addition, it is possible to accurately analyze the movement of the analysis target entering from the rectangular region 20.

  Next, the movement state calculation unit 11 calculates a representative direction of the vector information based on the direction included in the vector information existing in the rectangular area 20 (step S15).

  Again, the vector information indicates various directions according to the movement (motion) of the analysis target as described above. If such vector information corresponding to the motion of the analysis target is displayed, it becomes difficult to identify individual vector information corresponding to each analysis target.

  Therefore, the movement state calculation unit 11 calculates a representative direction of the vector information existing in the rectangular area 20.

  Details of the process in step S15 will be described below.

  In the present embodiment, the movement state calculation unit 11 calculates this representative direction using the first coordinate system (small area model).

  Here, the first coordinate system will be described with reference to FIG.

  FIG. 11 is a diagram illustrating the first coordinate system.

  As shown in FIG. 11, the first coordinate system T has a direction axis a to h indicating the direction away from the origin 0 around the origin 0 (an example of a direction axis indicating a predetermined direction around the origin of the present application). ) Are provided at an angle of 45 ° (an example of a predetermined angle of the present application) and 8 (a plurality of examples of the present application) with respect to the origin 0. That is, the first coordinate system T is a coordinate axis obtained by dividing the planar coordinate system into eight.

  Then, the movement state calculation unit 11 determines whether the direction included in the vector information existing in the rectangular region 20 approximates the direction indicated by any one of the direction axes a to h in the first coordinate system T.

  Various methods can be applied as a method for determining this approximation, and an example thereof will be described with reference to FIG.

  FIG. 12 is a conceptual diagram illustrating a method for determining approximation.

  When determining the approximation, the movement state calculation unit 11 refers to data predetermined for a range that approximates the direction indicated by one direction axis.

  As the approximate range, for example, when one direction axis is virtually rotated within a predetermined angle range around the origin 0, the virtually rotated direction axis is the predetermined angle. All directions shown in the range are determined as approximate ranges.

  Specifically, as shown in FIG. 12, for a range approximating the direction indicated by the direction axis a as one direction axis, the direction axis a is virtually 22.5 in the one direction with the origin 0 as the center. When the direction axis a is rotated 22.5 ° in the opposite direction (that is, the direction axis a is rotated within the range of 45 ° around the origin 0), the virtually rotated direction axis a is one direction. It is determined that all the directions shown when rotating 22.5 ° to 22.5 ° and 22.5 ° in the opposite direction are in a range approximating the direction indicated by the direction axis a. That is, the range approximated to the direction indicated by the direction axis a corresponds to the range indicated by the hatched portion in FIG.

  As described above, the movement state calculation unit 11 determines one or a plurality of representative directions by using the first coordinate system T, thereby determining a range to be approximated based on the movement start point of the vector information. Therefore, it is possible to accurately analyze the movement of the analysis object that leaves the predetermined range of the imaging region while reducing the processing load on the apparatus (CPU or the like).

  Hereinafter, a range that approximates the direction indicated by the directional axis is simply referred to as an “approximate range”.

  In the example illustrated in FIG. 12, the range that approximates the direction indicated by the direction axis b to the direction axis h is also determined in the same manner as the direction axis a.

  The method for setting the approximate range is not limited to the above-described example, and can be arbitrarily set. Specifically, the approximate range may be set differently for one direction axis and the other direction axis.

  In the above-described example, the approximate range is determined using the direction axis a to the direction axis h obtained by dividing the planar coordinate system into eight. However, the present invention is not limited to this. You may make it use the coordinate axis divided | segmented by the direction axis (for example, 4 divisions, 6 divisions, etc.).

  Then, as the process of step S <b> 15, the movement state calculation unit 11 virtually moves the movement start point included in the vector information existing in the rectangular area 20 to the origin of the first coordinate system T. Then, the movement state calculation unit 11 determines whether the direction included in the moved vector information corresponds to an approximate range of any of the direction axes a to h in the first coordinate system T.

  Here, with reference to FIG. 13, the movement state calculation unit 11 determines whether the direction included in the moved vector information corresponds to a range approximated by any of the direction axes in the first coordinate system T. The determination result will be described.

  FIG. 13 is a diagram illustrating a determination result obtained by determining whether the direction included in the moved vector information corresponds to a range approximated by any of the direction axes in the first coordinate system T.

As illustrated in FIG. 13, the movement start point V _1S of the vector information V ₁ is virtually moved to the origin 0, and the direction included in the vector information V ₁ is included in a range approximated by the direction axis f. .

Accordingly, the moving state calculation unit 11, the vector information V ₁ was, it is determined that approximates the direction indicated by the axis f.

  Then, the movement state calculation unit 11 sequentially makes the same determination for the vector information existing in the rectangular area 20.

  And as the last process of step S15, the movement state calculation part 11 calculates the typical direction of the vector information which exists in the rectangular area 20. FIG.

  Various methods can be applied to representative direction calculation methods.

  Specifically, for example, the movement state calculation unit 11 represents the direction indicated by the direction axis determined to approximate the most vector information among the directions indicated by the direction axis determined to approximate vector information. Direction.

  When the movement state calculation unit 11 calculates the representative direction of all vector information existing in the rectangular area 20 (step S16: YES), the display control unit 12 displays the vector existing in the rectangular area 20. A representative direction of information is displayed (step S17).

  Here, a display example in which representative directions of vector information existing in the rectangular area 20 are displayed will be described with reference to FIG.

  FIG. 14 is a diagram showing a display example in which representative directions of vector information existing in the rectangular area 20 are displayed.

  As shown in FIG. 14, a representative direction of vector information existing in the rectangular area 20 is displayed in the rectangular area 20. The direction displayed in the rectangular area 20 is the direction axis (direction axis b in FIG. 13) determined to be the most approximate among the direction axes of the first coordinate system T as the above-described representative direction calculation method. The direction indicated by) is displayed.

  Hereinafter, the direction axis determined in this way is referred to as a “direction axis indicating a typical direction” (direction axis b in FIG. 13). Further, the representative direction calculated in this way is simply referred to as “representative direction”.

  The displayed representative direction indicates the movement indicated by the most people among the movements of the persons shown in the moving image in the rectangular area 20.

  Here, the rectangular area 20 indicates a position near the entrance of the store S1. And the displayed representative direction has shown the direction which penetrate | invades into this shop S1.

  Therefore, it can be seen from this display result that there are many people who visit the store S1 and the store S1 is a popular store.

  When the movement state calculation unit 11 has not calculated the representative directions of all the vector information existing in the rectangular area 20 (step S16: NO), the movement state calculation unit 11 proceeds to the process of step S11.

  In addition, various forms can be applied to the display mode of the representative direction.

  In the example shown in FIG. 14, only the direction axis determined to approximate the most vector information is displayed as the representative direction, but it is determined to approximate the second or third most vector information. The direction axis may also be displayed together.

  That is, the quantity in the representative direction displayed in the rectangular area 20 can be arbitrarily set.

  Moreover, the display mode of the representative direction may be changed according to the quantity of vector information belonging to the direction axis indicating the representative direction.

  The quantity of vector information belonging to the direction axis indicating the representative direction refers to the quantity of vector information determined to be approximate to the direction axis determined to be the representative direction by the movement state calculation unit 11. .

  Here, a display result when the display mode of the representative direction is changed according to the quantity of vector information belonging to the direction axis indicating the representative direction will be described with reference to FIG.

  FIG. 15 is a conceptual diagram showing a display result when the display mode of the representative direction is changed according to the quantity of vector information belonging to the direction axis indicating the representative direction.

  As shown in FIG. 15, the display control unit 12 changes the thickness of the representative direction 30 in five stages of 1 to 5 according to the quantity of vector information belonging to the direction axis indicating the representative direction. Can also be displayed.

  The display mode of the representative direction may be changed according to the magnitude of the position change included in the vector information belonging to the direction axis indicating the representative direction.

  The magnitude of the position change included in the vector information belonging to the direction axis indicating the representative direction is a vector determined to be approximate to the direction axis determined to be the representative direction by the movement state calculation unit 11. This refers to the magnitude of the position change included in the information.

  The magnitude of the position change can be grasped based on the coordinate values indicated by the movement start point and the movement end point included in the vector information.

  Therefore, the display control unit 12 changes the display mode of the representative direction based on the magnitude of the position change included in the vector information determined to approximate the direction axis determined to be the representative direction. Let

  Here, the display result when the display mode of the representative direction is changed according to the magnitude of the position change included in the vector information belonging to the direction axis indicating the representative direction will be described with reference to FIG. To do.

  FIG. 16 is a conceptual diagram illustrating a display result when the display mode of the representative direction is changed according to the position change included in the vector information belonging to the direction axis indicating the representative direction.

  As illustrated in FIG. 16, the display control unit 12 determines the length of the representative direction 40 in five stages 1 to 5 according to the position change included in the vector information belonging to the direction axis indicating the representative direction. It is also possible to change the display.

  As the magnitude of this position change, a value obtained by integrating the magnitude of the position change of each vector information may be applied, or a value obtained by calculating the average of the magnitude of the position change of each vector information may be applied.

  Further, the display mode of the representative direction may be changed according to the quantity of vector information belonging to the direction axis indicating the representative direction and the magnitude of the position change included in the vector information.

  FIG. 17 shows a display when the display mode of the representative direction is changed according to the quantity of vector information belonging to the direction axis indicating the representative direction and the magnitude of the position change included in the vector information. It is a conceptual diagram which shows a result.

  As shown in FIG. 17, the display control unit 12 displays the representative direction 50 according to the quantity of vector information belonging to the direction axis indicating the representative direction and the magnitude of the position change included in the vector information. It is also possible to change the thickness and length of the image.

  In the example described above, the movement state calculation unit 11 calculates the representative direction based on the vector information existing in the rectangular area 20, but divides the frame (in the imaging area) into areas of a predetermined size. The one or more representative directions are calculated for each divided area based on vector information existing in the divided areas, and one or more representative directions corresponding to the calculated divided areas are You may make it display on each said division area.

  Here, a calculation result when the movement state calculation unit 11 calculates the representative direction based on the vector information existing in the divided area will be described with reference to FIG.

  FIG. 18 calculates one or more representative directions based on the vector information existing in the divided areas, and sets one or more representative directions corresponding to the calculated divided areas in the divided areas. It is a figure which shows the example of a display at the time of displaying.

  As shown in FIG. 18, the display image 60 shown in the display unit 4 includes typical directions 62 and 63 calculated based on vector information existing in a plurality of predetermined divided areas 61. It is displayed. The divided area 61 is obtained by dividing a frame into a plurality of divided areas having a predetermined size.

  In FIG. 18, the member numbers of the divided areas other than the divided area 61 are not shown for convenience of explanation. Further, the representative directions are indicated by arrows in FIG. 18, but the member numbers in the representative directions other than the representative directions 62 and 63 are not shown for convenience of explanation. .

  From this display example, for example, since the representative direction 62 indicates the direction of entering the store S1, many people come to the store S1, and it can be seen that the store S1 is a popular store. Moreover, since the typical direction 63 has shown the direction which passes in front of store S2, there are few people who come to store S2, and it turns out that store S1 is a store which is not so popular.

  The movement state calculation unit 11 can also calculate the above-described representative direction with reference to the middle region.

  The middle area is a group of a predetermined number of small areas as one area, where one divided area when the frame is divided into a plurality of divided areas in advance is a small area. In this middle area, the quantity in the representative direction calculated for each middle area is set in advance, and the movement state calculation unit 11 responds to the quantity in the typical direction set in each middle area. Then, one or more representative directions are calculated for each small area to which the middle area belongs, and one or more representative directions are calculated for each small area to which the calculated middle area belongs. To be displayed.

  For example, when the vector information described above is calculated for a person indicated by a moving image, the directions included in the calculated vector information may be dispersed. In addition, there are places where the vector information is dense and places where the vector information is not calculated at all within the range of the imaging region. That is, the calculated vector information varies.

  Then, when a representative direction is calculated based on such vector information, variation also occurs in this representative direction.

  Therefore, the middle area is set so that vector information is calculated uniformly over the entire moving image, and the quantity in the representative direction calculated for each middle area is set in advance.

  Here, an example of setting the middle region will be described with reference to FIG.

  FIG. 19 is a conceptual diagram illustrating an example of setting the middle region.

  As shown in FIG. 19, the display image 70 displayed on the display unit 4 includes a plurality of middle regions 71 whose frames are determined in advance, and a quantity 72 of one or more representative directions calculated for the middle region 71. 1 is set for each. The middle area 71 is a collection of a predetermined number of the small areas as one area as described above. For example, four middle areas 71 having the same size as the small area 61 shown in FIG. 18 are collected. It indicates the size (that is, the middle region 71 is four times as large as the small region 61). In the display image 70, 20 middle regions are set.

  In FIG. 19, the member numbers of the middle region other than the middle region 71 and the member numbers of quantities other than the representative direction quantity 72 calculated in the middle region 71 are not shown for convenience of explanation. doing.

  Various methods can be applied to the method of calculating one or more representative directions for each of the small regions belonging to the middle region.

  Specifically, for example, paying attention to a certain middle region, a direction axis determined to approximate the largest amount of vector information is extracted for each small region belonging to the middle region. Then, among the direction axes extracted for each small area, the direction axis determined to approximate the most vector information or the direction axis determined to approximate the third most vector information You may make it display according to the quantity of the typical direction calculated to an area | region (Namely, the quantity of the vector information judged to approximate to a direction axis is displayed in an order from the largest).

  In addition, when the vector information extracted in the small area belonging to the middle area is equal to or less than a certain quantity, the representative direction calculated in the middle area may not be displayed.

  In addition, when the vector information extracted in the small area belonging to the middle area is a certain quantity or more, a representative calculated for the middle area for each middle area according to the ratio of the quantity of the extracted vector information. A specific direction may be determined. Then, by applying such a calculation method, the movement state calculation unit 11 uses FIG. 20 for calculation results when the movement state calculation unit 11 calculates one or more representative directions for each small region belonging to the middle region. I will explain.

  FIG. 20 shows a display example when the movement state calculation unit 11 displays a calculation result in a case where one or a plurality of representative directions are calculated for each small region belonging to the middle region with reference to the middle region. FIG.

  As shown in FIG. 20, the display image 80 shown in the display unit 4 displays one or more representative directions of the quantity corresponding to the quantity 82 of the representative direction calculated in the middle area 81. Yes. In other words, in the display image 80, a representative direction is not displayed for a case where the number of representative directions calculated in the middle region is 0, and the number of representative directions calculated in the middle region is 1. For those with one representative direction, the number of representative directions calculated for the middle region is two. For those with two representative directions, the number of representative directions calculated for the middle region. Three representative directions are respectively displayed for those having three. Also, the actual display image can be set so that only the representative direction is displayed without displaying the middle region and the quantity of the representative direction calculated in the middle region. In addition, as a predetermined range of the above-described imaging region (the above-described rectangular region 20 or the like), a predetermined range of imaging regions having at least two different sizes is set, and based on a direction included in vector information existing in the region. Thus, a representative direction can be calculated.

  Here, FIG. 21 is used for a display example in the case where a predetermined range of imaging regions having two or more different sizes is set and a representative direction is calculated based on a direction included in vector information existing in the region. I will explain.

  FIG. 21 is a diagram showing a display example when a predetermined range of imaging areas having two or more different sizes is set and a representative direction is calculated based on a direction included in vector information existing in the area. is there.

  As shown in FIG. 21, in the display image 90 shown in the display unit 4, a passage 92 and a passage 93 are displayed in the left-right direction around a display shelf 91 on which products are displayed.

  A rectangular area 94 (an example of a predetermined range of an imaging area having a different size) is provided in front of the display shelf 91, and a rectangular area 95 (an example of a predetermined range of an imaging area having a different size) is provided at the entrance of the passage 92. A rectangular area 96 (an example of a predetermined range of imaging areas having different sizes) is set at each of the 93 entrances.

  Further, a representative direction 96 calculated based on the direction included in the vector information existing in the rectangular area 94 is displayed in the rectangular area 94, and the direction included in the vector information existing in the rectangular area 95 is displayed in the rectangular area 95. A representative direction 97 calculated based on the information is displayed. In the rectangular area 96, a representative direction 98 calculated based on the direction included in the vector information existing in the rectangular area 96 is displayed.

  The representative directions 96 to 98 displayed on the display image 90 include, for example, the number of people coming to the display shelf 91, the passage 92, or the passage 93, the size of the position change, and the number of people passing through. And the magnitude of the position change is shown.

  If the representative direction 96 indicates that the number of people coming to the display shelf 91 is large, it is understood that there are many people interested in the products displayed on the display shelf 91.

  In addition, if it is indicated from the representative direction 96 that the position change of the person who comes toward the display shelf 91 is large, it is understood that there are many people who want to arrive at the display shelf 91 as soon as possible.

  If the representative direction 96 indicates that the number of people passing through the display shelf 91 is large, it is understood that the product displayed on the display shelf 91 is an unpopular product.

  Further, if the representative direction 97 or the representative direction 98 indicates that the number of people heading toward the back of the passage 92 or the passage 93 is large, the person is interested in the back of the passage 92 or the passage 93. It is understood that there are facilities etc.

  It should be noted that the predetermined ranges of the two or more different sized imaging regions can be arbitrarily set based on the operation of the input unit 2 by the user.

  An attribute label can also be set in a predetermined range of the imaging area.

  This attribute label is information given for a predetermined range of the imaging region, for example, identification information for identifying the predetermined range of the imaging region from other ranges, information characterizing the predetermined range of the imaging region, etc. Applicable.

  Here, an example in which an attribute label is set in a predetermined range of the imaging region will be described with reference to FIG.

  FIG. 22 is a diagram illustrating an example when attribute labels are set in a predetermined range of two or more imaging regions.

  As shown in FIG. 22, an attribute label 105 is set in a rectangular area 100 (an example of a predetermined range of two or more imaging areas). Note that the predetermined range of the imaging area indicated by the rectangular area 100 indicates a place where a low-priced (low price) product is displayed in the store. Therefore, in the attribute label 105, information “low price” is set as information characterizing a predetermined range of the imaging region.

  Similarly, attribute labels 106 to 109 are set in the rectangular area 101 to the rectangular area 104.

  The attribute label 106 includes information of “medium price” indicating a place where a product of medium price (for example, an affordable price that is easy for a user to purchase) is displayed, and the attribute label 107 includes high price (for example, expensive). The information “high price” indicating the location where the product of the new product is displayed, and the attribute label 108 includes the information “new product” indicating the location where the newly released product is displayed. In 109, information of “trigger” indicating a store entrance or a measurement start point is set.

  The attribute labels 105 to 109 set in this way are displayed on a display image shown on the display unit 4, for example. A user who sees the attribute labels 105 to 109 can easily grasp the characteristics of a predetermined range of the imaging region.

  In the example described above, the movement state calculation unit 11 calculates the representative direction using the first coordinate (small area model), but uses the second coordinate system. A representative direction may be calculated.

  Here, the second coordinate system will be described with reference to FIG.

  FIG. 23 is a diagram illustrating a second coordinate system.

  As shown in FIG. 23, the second coordinate system U has a direction axis a to h indicating the direction indicating the origin 0 with respect to the origin 0 (an example of a direction axis indicating the origin from a predetermined direction of the present application) Eight (a plurality of examples of the present application) are provided 45 degrees apart from each other around the origin 0 (an example of a predetermined angle of the present application).

  Then, the movement state calculation unit 11 virtually moves the movement end point included in the vector information existing in a predetermined range (for example, the rectangular area 20) of the imaging area to the origin of the second coordinate system U. Then, the movement state calculation unit 11 determines whether the direction included in the moved vector information corresponds to an approximate range of any of the direction axes a to h in the second coordinate system U.

  In this way, the movement state calculation unit 11 uses the second coordinate system U to calculate a representative direction, thereby determining a range that approximates the movement end point of the vector information. While reducing the processing load, it is possible to accurately analyze the movement of the analysis target that enters the predetermined range of the imaging region.

  In the above-described example, the vector information per unit time indicated by the analysis target is calculated using a straight line connecting the movement start point and the movement end point. However, an approximate curve (regression line) that passes through the movement start point or the movement end point. You may make it calculate by.

  Here, the case where the vector information per unit time indicated by the analysis target is calculated using an approximate curve will be described with reference to FIG.

  FIG. 24 is a conceptual diagram illustrating a case where vector information per unit time indicated by the analysis target is calculated using an approximate curve.

  First, when calculating the vector information per unit time indicated by the analysis target, the unit time is assumed to be 5 seconds.

As shown in FIG. 24, a case where the person H1 moves as a position from V _1s (x1, y1) to V _5e (x6, y6) in 5 seconds (moves to the arrow indicated by the broken line in the figure) is considered. . At this time, the position indicated by the person H1 in the reference frame is V _1s (x1, y1), the position indicated by the person H1 in the frame a second after the frame is V _1e (x2, y2), and 1 It is _{assumed that the} positions indicated by the person H1 are specified as V _3s (x3, y3), V _4s (x4, y4), and V _5s (x5, y5) at second intervals, respectively. When the movement end point of the position change per certain time and the movement start point of the position change per unit time calculated after the position change (for example, V _1e and V _2s etc.) indicate the same point. Since this has already been described with reference to FIG. 7, the description thereof is omitted.

Here, the vector information when the unit time is 5 seconds passes through either the movement start point V _1s (x1, y1) or the movement end point V _5e (x6, y6), V _1s (x1, It can be obtained by calculating a regression line in V _5e (x6, y6) from y1).

  Since this regression line calculation method is a known technique, a detailed description thereof is omitted, but it can be calculated by a so-called least square method.

Specifically, n pieces of two-variable data (x ₁ , y ₁ ), (x ₂ , y ₂ ), ..., (x _i , y _i ), ..., (x _n , y _n ) The regression line is calculated by substituting Equation (2) and Equation (3) into Equation (1), where the regression line is Equation (1).

本実施例においては、Ｖ_１ｓ（ｘ１、ｙ１）からＶ_５ｅ（ｘ６、ｙ６）における値を式（２）及び式（３）へ代入し、ａ及びｂを算出し、式（１）の回帰直線を算出する。

In the present _embodiment, by substituting from V 1s (x1, y1) values in _V 5e (x6, y6) to formula (2) and (3), to calculate the a and b, the regression equation (1) Calculate a straight line.

  Then, the regression line calculated in this way may be applied as vector information per unit time indicated by the analysis target.

  In this way, by calculating the vector information per unit time with a regression line, the vector information is calculated in consideration of the position change between the movement start point and the movement end point of the vector information per unit time. The movement of the analysis object can be accurately grasped.

  In addition, embodiment described above does not limit the invention concerning a claim. And not all the combinations of the configurations described in the above embodiment are indispensable means for solving the problems of the invention.

  Also, programs corresponding to the flowcharts shown in FIGS. 2 and 6 are recorded on a recording medium such as a flexible disk or a hard disk, or acquired and stored via the network NT, and these are stored in a general-purpose personal computer or the like. It is also possible to cause the personal computer or the like to function in the same manner as the operation indicated by the CPU of the image analysis apparatus SS according to the above-described embodiment.

  As described above, in the present embodiment, the image analysis apparatus SS that analyzes the motion of the analysis target captured as a moving image includes the input image storage unit 5 that stores the moving image captured in the imaging region for a certain period of time. The moving object detection unit 6 that detects the analysis object indicated in the moving image, and the position change that the detected analysis object indicates in the imaging area and is associated with the moving object ID 8 Based on the position change calculation unit 7 to be stored in the memory and the stored position change, the detected analysis target indicates the position change per unit time indicated in the imaging area, and the position and direction in the imaging area. The vector information calculation unit 9 that calculates for each moving body ID as vector information including, and the vector information existing within a predetermined range of the imaging region is extracted based on the position included in the vector information Based on the vector information extraction unit 10 and the direction included in the extracted vector information existing in the predetermined range, a moving state for calculating one or more representative directions of the vector information existing in the predetermined range Since the calculation unit 11 and the display control unit 12 that displays the calculated representative directions to the outside are provided, the movement of the analysis target is analyzed more accurately, and the graspability for the analysis result is improved. Can be made.

  In the above embodiment, an example in which the present application is applied to the image analysis apparatus SS has been described. However, the present invention can be applied to, for example, a personal computer or an electronic device for home use. .

DESCRIPTION OF SYMBOLS 1 Image input part 2 Input part 3 Control part 4 Display part 5 Input image storage part 6 Moving object detection part 7 Position change calculation part 8 Position coordinate memory | storage part 9 Vector information calculation part 10 Vector information extraction part 11 Movement state calculation part 12 Display control 13 Output frame memory SS Image analysis device

Claims (16)

An image analysis device that analyzes a motion of an analysis target captured as a moving image,
Storage means for storing a moving image captured for a certain period in the imaging area;
Detecting means for detecting the analysis target shown in the moving image;
A position change calculating means for calculating a position change indicated by the detected analysis object in the imaging region and storing it in association with analysis object identification information for identifying the analysis object from other analysis objects;
Based on the stored change in position, the detected analysis object uses the change in position per unit time indicated in the imaging area as the vector information including the position and direction in the imaging area for each analysis object identification information. Vector information calculating means for calculating
Vector information extraction means for extracting vector information existing within a predetermined range of the imaging region based on a position included in the vector information;
A moving state calculating means for calculating one or more representative directions of the vector information existing in the predetermined range based on the direction included in the extracted vector information existing in the predetermined range;
Display control means for displaying the calculated representative direction to the outside;
An image analysis apparatus comprising: The image analysis apparatus according to claim 1,
The vector information calculation means calculates a movement start point of the analysis target as the vector information, and extracts vector information existing within a predetermined range of the imaging region based on the movement start point included in the vector information An image analysis apparatus characterized by: The image analysis apparatus according to claim 1 or 2,
The vector information calculation means calculates the movement end point of the analysis target as the vector information, and extracts vector information existing within a predetermined range of the imaging area based on the movement end point included in the vector information An image analysis apparatus characterized by: The image analysis device according to any one of claims 1 to 3,
The movement state calculation means includes
The direction included in the extracted vector information within the predetermined range is
Determining whether a direction axis indicating a predetermined direction around the origin approximates a direction indicated by any one of the direction axes in the first coordinate system provided at a predetermined angle apart from the origin;
An image analyzing apparatus, wherein a direction included in the vector information and a direction indicated by a direction axis determined to be approximated are calculated as representative directions of vector information existing within the predetermined range. The image analysis apparatus according to any one of claims 1 to 4,
The movement state calculation means includes
The direction included in the extracted vector information within the predetermined range is
Determining whether the direction axis indicating the origin from a predetermined direction approximates the direction indicated by any of the direction axes in the second coordinate system provided at a predetermined angle apart from the origin.
An image analyzing apparatus, wherein a direction included in the vector information and a direction indicated by a direction axis determined to be approximated are calculated as representative directions of vector information existing within the predetermined range. An image analysis apparatus according to any one of claims 1 to 5,
The vector information extracting means extracts at least two or more vector information,
The display control means changes the display mode of the representative direction to be displayed to the outside in accordance with the quantity of vector information belonging to the calculated representative direction. The image analysis apparatus according to any one of claims 1 to 6,
The vector information extracting means extracts at least two or more vector information,
The display control unit changes a display mode of a representative direction to be displayed to the outside according to a magnitude of a position change included in vector information belonging to the calculated representative direction. Image analysis device. The image analysis apparatus according to any one of claims 1 to 7,
The vector information extracting means extracts at least two or more vector information,
The display control means includes
An image analysis apparatus for displaying the calculated representative direction to the outside when the quantity of vector information belonging to the representative direction calculated by the movement state calculating means is equal to or greater than a predetermined threshold. . The image analysis device according to any one of claims 1 to 8,
The vector information extracting means extracts at least two or more vector information,
The display control means includes
An image analyzing apparatus characterized by extracting a representative direction having the largest quantity of vector information belonging to a representative direction calculated by the movement state calculating means and displaying the representative direction to the outside. The image analysis apparatus according to any one of claims 1 to 9,
The vector information extracting means includes
Based on the position included in the vector information, vector information existing within a predetermined range of the imaging area of at least two different sizes is extracted for each predetermined range of the imaging area,
The movement state calculation means determines a representative direction of the vector information existing within the predetermined range based on the direction included in the extracted vector information within the predetermined range within the predetermined range of the imaging region. An image analysis apparatus characterized by calculating each time. The image analysis apparatus according to any one of claims 1 to 10,
The vector information extraction unit calculates, for each divided region, vector information existing in a divided region that divides the imaging region into regions of a predetermined size,
The movement state calculation means calculates the one or more representative directions for each divided region based on the vector information calculated for each divided region,
The display control unit displays one or a plurality of representative directions corresponding to the calculated divided areas in the divided areas. The image analysis apparatus according to claim 11,
A predetermined number of the divided areas are set as one area, and a quantity in a representative direction calculated for each intermediate area is set in advance.
The movement state calculation means calculates one or a plurality of representative directions for each small area belonging to the middle area according to the quantity set for each middle area,
The display control unit displays one or more representative directions in the small area for each small area to which the calculated middle area belongs. The image analysis apparatus according to any one of claims 1 to 12,
An image analysis apparatus further comprising an input unit capable of arbitrarily setting a predetermined range of the imaging region. An image analysis method in an image analysis apparatus for analyzing a motion of an analysis target captured in a moving image,
A storage step of storing a moving image captured for a certain period in the imaging region;
A detection step of detecting the analysis target shown in the moving image;
A position change calculating step in which the detected analysis object calculates a position change indicated in the imaging region, and stores the analysis object in association with analysis object identification information for identifying the analysis object from other analysis objects;
Based on the stored change in position, the detected analysis object uses the change in position per unit time indicated in the imaging area as the vector information including the position and direction in the imaging area for each analysis object identification information. A vector information calculation step to calculate
A vector information extracting step of extracting vector information existing within a predetermined range of the imaging region based on a position included in the vector information;
A moving state calculating step of calculating one or more representative directions of the vector information existing in the predetermined range based on the direction included in the extracted vector information existing in the predetermined range;
A display control step of displaying the calculated representative direction to the outside;
An image analysis method characterized by comprising: A computer included in an image analysis apparatus that analyzes the motion of an analysis target captured in a moving image,
Storage means for storing a moving image captured for a predetermined period in the imaging region;
Detecting means for detecting the analysis target shown in the moving image;
A position change calculating means for calculating a position change indicated by the detected analysis object in the imaging region and storing the analysis object in association with analysis object identification information for identifying the analysis object from other analysis objects;
Based on the stored change in position, the detected analysis object uses the change in position per unit time indicated in the imaging area as the vector information including the position and direction in the imaging area for each analysis object identification information. Vector information calculating means for calculating
Vector information extraction means for extracting vector information existing within a predetermined range of the imaging region based on a position included in the vector information;
A moving state calculating means for calculating one or more representative directions of the vector information existing in the predetermined range based on the direction included in the extracted vector information existing in the predetermined range;
Display control means for displaying the calculated representative direction to the outside;
An image analysis program characterized by functioning as 16. A recording medium on which the image processing program according to claim 15 is recorded so as to be readable by a computer.

Images