JP2007280151A - Image analyzer, image analytical method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To analyze an image to monitor or to do likewise a proper portion in the picked-up image, while taking, in overall, knowledges in the present and in the past into consideration. <P>SOLUTION: This image analyzer/image analytical method includes: a frame image acquiring part 14 for acquiring a frame image sequentially; an image analytical part 16 for judging a marked position vector in the frame image acquired sequentially; and an analytical area judging part 18 for generating an analytical area for specifying an area in the frame image serving as an analytical object by the image analytical part 16, based on the marked position vector. The analytical area judging part 18 makes a coupling weight vector of a self-organization map for short period storage get near to a coupling weight vector of a self-organization map for long period storage, according to the lapse of time, while making at least one part of the coupling weight vector of the self-organization map for the short period storage get near to the marked position vector, based on the marked position vector. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image analysis apparatus, an image analysis method, and a program, and more particularly, to information processing for determining an analysis target portion in an image.

Various systems have been proposed in which a pedestrian in front of a vehicle is quickly and reliably detected and notified to a vehicle driver. For example, Patent Document 1 below discloses a pedestrian detection device that detects a pedestrian based on an image from a stereo camera and thereby performs various warnings.
JP 2005-228127 A

  Analyzing an image taken in front of the vehicle, detecting a target object such as a pedestrian from there, and determining the importance of the target object, it is inefficient to always analyze the entire image In other words, it is difficult to quickly detect the target object and determine its importance. Therefore, it is conceivable that only a partial region of the image is to be analyzed.

  In this regard, humans monitor pedestrians that are actually visible, and if they are judged to be less important (has passed the danger), they look at the places where other pedestrians may be based on past experience. Thus, it is extremely important to streamline this type of information processing by comprehensively taking into account current and past knowledge in front of the vehicle and performing image analysis for monitoring, etc. at appropriate locations. is there.

  The present invention has been made in view of the above problems, and its purpose is to comprehensively consider current and past knowledge, and to perform image analysis for monitoring or the like on an appropriate location in a captured image. An object of the present invention is to provide an image analysis apparatus, an image analysis method, and a program that can be performed.

  In order to solve the above problems, an image analysis apparatus according to the present invention includes an image acquisition unit that sequentially acquires images, a target position vector determination unit that determines target position vectors in the sequentially acquired images, A first combination weight vector storage means for storing a plurality of combination weight vectors respectively corresponding to the units arranged in the competitive layer of the self-organizing map, and a unit arranged in the competitive layer of the second self-organizing map Second coupling weight vector storage means for storing a plurality of corresponding coupling weight vectors, and storage in the first coupling weight vector storage means based on the attention position vector in the image determined by the attention position vector determination means. While at least part of the plurality of coupled weight vectors corresponding to the target position vector is brought close to the target position vector, First connection weight vector updating means for bringing the plurality of connection weight vectors stored in the first connection weight vector storage means closer to the plurality of connection weight vectors stored in the second connection weight vector storage means according to progress; First output vector generation means for generating an output vector of the first self-organizing map based on the plurality of connection weight vectors stored in the first connection weight vector storage means; and the first output vector generation means Based on the output vector of the first self-organizing map generated by the above, at least a part of the plurality of connection weight vectors stored in the second connection weight vector storage means according to the output vector is output. Second connection weight vector updating means for bringing the vector close to a vector, and the first connection weight vector of the image Performing a predetermined image analysis processing on the portions corresponding to the plurality of coupling weight vector stored in the 憶 means, characterized in that.

  The image analysis method according to the present invention is determined by an image acquisition step of sequentially acquiring images, an attention position vector determination step of determining an attention position vector in the sequentially acquired images, and the attention position vector determination step. A plurality of connection weight vectors respectively corresponding to units arranged in the competitive layer of the first self-organizing map, stored in the first connection weight vector storage means, based on the attention position vector in the image The plurality of connection weight vectors stored in the first connection weight vector storage unit as time elapses while at least a part corresponding to the target position vector is brought close to the target position vector. A plurality of connections, each corresponding to a unit located in a competitive layer of the second self-organizing map A first connection weight vector updating step for bringing the first connection weight vector closer to the first vector, and a first self-organization map output vector based on the plurality of connection weight vectors stored in the first connection weight vector storage means Based on the output vector generation step and the output vector of the first self-organizing map generated in the first output vector generation step, the plurality of connection weight vectors stored in the second connection weight vector storage means A second connection weight vector updating step for causing at least a part of the output vector to approach the output vector, and the plurality of connection weight vectors stored in the first connection weight vector storage means of the image A predetermined image analysis process is performed on the part corresponding to the above.

  Further, the program according to the present invention is arranged in an image acquisition unit that sequentially acquires images, a target position vector determination unit that determines target position vectors in the sequentially acquired images, and a competitive layer of the first self-organizing map. A first connection weight vector storage means for storing a plurality of connection weight vectors respectively corresponding to the selected units, and a plurality of connection weight vectors respectively corresponding to the units arranged in the competitive layer of the second self-organizing map. Based on the attention position vector in the image determined by the second connection weight vector storage means and the attention position vector determination means, the attention value among the plurality of connection weight vectors stored in the first connection weight vector storage means. The first combination weight as time passes while at least part of the position vector is close to the target position vector First connection weight vector updating means for bringing the plurality of connection weight vectors stored in the Kutor storage means closer to the plurality of connection weight vectors stored in the second connection weight vector storage means, and the first connection weight vector storage means 1st output vector generation means for generating an output vector of the first self-organizing map based on the plurality of connection weight vectors stored in the first output vector, and the first output vector generation means Based on the output vector of the self-organizing map, a second connection weight vector that causes at least a part of the plurality of connection weight vectors stored in the second connection weight vector storage means to be close to the output vector. A computer is made to function as an update means, and the computer is caused to have the first image out of the images. Goomomi vector storage means to perform a predetermined image analysis processing on the portions corresponding to the plurality of coupling weight vector stored in, characterized in that. This program may be stored in various computer-readable information storage media such as a CD-ROM and a DVD-ROM.

  According to the present invention, the connection weight vector of the second self-organizing map reflects the past attention position vector. On the other hand, the coupling weight vector of the first self-organizing map is based on the coupling weight vector of the second self-organizing map having such a character and the influence of the recent attention position vector is added thereto. Accordingly, when a target position vector that approximates a past target position vector is obtained from an image captured by a camera or another image, at least a part corresponding to the target position vector is quickly and very much included in the target position vector. Change to something close. If there is no input of the target position vector, the connection weight vector of the first self-organizing map approaches the connection weight vector of the second self-organizing map as time passes. Therefore, when the target region for image analysis processing in the image is determined based on the coupling weight vector of the first self-organizing map, if there is a target position, the region corresponding to that position becomes the target region for image analysis processing. If the position disappears, the area corresponding to the past attention position becomes the target area for the image analysis processing. For this reason, like a human being, it is possible to perform an image analysis process for monitoring or the like on an appropriate portion in an image in consideration of current and past knowledge.

  In one aspect of the present invention, third connection weight vector storage means for storing a plurality of connection weight vectors respectively corresponding to units arranged in the competitive layer of the third self-organizing map; and the first output vector generation means Based on the output vector of the first self-organizing map generated by the above, at least a part of the plurality of connection weight vectors stored in the third connection weight vector storage means according to the output vector is output. A third output unit for generating an output vector of the third self-organizing map based on a plurality of connection weight vectors stored in the third connection weight vector storage unit; Vector generating means, wherein the second combined weight vector updating means is generated by the third output vector generating means. Based on the output vector of said third self-organizing map, close to the output vector at least a portion corresponding to the output vector of the plurality of coupling weight vector stored in the second coupling weight vector storage means. In this way, the third self-organizing map is used like a buffer, and the connection weight vector of the second self-organizing map is updated based on the output vector of the first self-organizing map. Can be delayed, and fluctuations in the coupling weight vector of the second self-organizing map can be suppressed.

  In the aspect of the invention, the predetermined image analysis process is a process of determining an attention position vector in the image by the attention position vector determination unit. In this way, the process of determining the target position vector can be limited to an appropriate part of the image, and the process can be speeded up.

  In one aspect of the present invention, the attention position vector determination means determines the importance of the attention position vector together with the attention position vector in the image, and the first combination weight vector update means determines the first combination weight vector. At least a part of the plurality of combined weight vectors stored in the weight vector storage unit according to the target position vector is brought close to the target position vector according to the importance determined together with the target position. In this way, when the connection weight vector of the first self-organizing map is updated according to the target position vector, the higher the degree of importance of the target position vector, the larger the update, and the closer the target position vector is. .

  In the aspect of the invention, the first output vector generation unit may generate the connection weight vector corresponding to a surrounding unit for each of the plurality of connection weight vectors stored in the first connection weight vector storage unit. And the connection weight vector selected according to the density is generated as an output vector of the first self-organizing map. If the density of the connection weight vector corresponding to a certain unit is high, it can be evaluated that the position vector of interest that approximates the connection weight vector has been repeatedly input to the first self-organizing map. The attention position vector repeatedly input can be selectively used as the output of the first self-organizing map. As a result, the connection weight vector of the second self-organizing map can be made to correspond to the attention position vector repeatedly input.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of an image analysis apparatus according to an embodiment of the present invention. As shown in the figure, the image analysis apparatus 10 includes a moving image camera 12, a frame image acquisition unit 14, an image analysis unit 16, an analysis region determination unit 18, an importance calculation unit 20, and a warning unit 22. ing. The frame image acquisition unit 14, the image analysis unit 16, the analysis region determination unit 18, and the importance calculation unit 20 are realized by causing a known computer to execute an analysis program. It is installed in a computer by a computer-readable information storage medium such as a ROM or a DVD-ROM.

  The image analysis apparatus 10 is installed in, for example, an automobile and notifies the driver of the danger in driving the vehicle. The moving image camera 12 is installed so as to capture a moving image in front of the vehicle. As shown in FIG. 2, the moving image data in front of the vehicle captured by the moving image camera 12 includes an image (frame image) in front of the vehicle every predetermined time (for example, 1/60 seconds). The unit 14 cuts out a frame image at each time from the moving image data. Then, frame image data indicating the frame image is supplied to the image analysis unit 16. FIG. 3 shows a frame image cut out from the moving image data shown in FIG. As shown in the figure, a road (a lane in which a vehicle on which the image analysis apparatus 10 is mounted) is projected in the center of the frame image, and a pedestrian T crossing the road is projected in some cases.

  The image analysis unit 16 determines the presence / absence of the pedestrian T (see FIG. 3) and the position coordinates of the pedestrian T (coordinates for specifying the position in the frame image) from the frame image indicated by the frame image data, and determines the frame image. When the pedestrian appears, the attention position vector Xs indicating the position coordinate is supplied to the importance calculation unit 20 and the analysis region determination unit 18.

  When the image analysis unit 16 analyzes the frame image to determine the presence / absence of the pedestrian T and the position coordinates of the pedestrian T, the image analysis unit 16 does not represent the entire frame image but a partial region in the frame image indicated by the symbol R in FIG. As described above, a part of the frame image is the object of image analysis. The part to be analyzed is specified by the analysis area data supplied from the analysis area determination unit 18 and is a part where a pedestrian is highly likely to be displayed in the frame image.

  In the image analysis unit 16, presence / absence of the pedestrian T and the position of the pedestrian T are performed by performing various moving object detection processes on the image region specified by the analysis region data supplied from the analysis region determination unit 18. Coordinates can be determined. For example, a feature point in a frame image is specified by applying a Moravec Operator, and a region (difference region) in the frame image is specified by calculating a difference between frames of the frame image. . Then, the position of the moving object (pedestrian T) is determined from the feature points included in the difference area based on a predetermined reference. In addition, the presence or absence of the pedestrian T and the position coordinates of the pedestrian T in the frame image may be determined by pattern matching based on the appearance of the human.

Importance calculation section 20, data indicating position coordinates of the pedestrian T supplied from the image analyzing unit 16, i.e. on the basis of the target position vector X ^s, and calculates the importance d of the noted position vector X ^s. Here, the importance d is calculated by paying attention to whether the pedestrian T is on the left or right of the road, the moving speed of the pedestrian T, and the degree of danger approaching the pedestrian T depending on the position of the pedestrian T. . For this reason, as shown in FIG. 5, first, a road vanishing point D, which is a point where the road projected in the frame image disappears in the distance, is determined, and on the left side (region A) from the position of the road vanishing point D. It is determined whether the pedestrian T is projected or is projected on the right side (region B). Also, position coordinates (x (t), y (t)) where the pedestrian T is projected at the current time t are obtained with the upper left corner of the frame image as the origin, the lower direction in the y direction, and the right side in the x direction. . Further, it is determined whether the pedestrian T is moving in the right direction or the left direction based on the inter-frame difference of x (t). Further, the moving speed S (t) of the pedestrian T is obtained from the moving distance of the position coordinates (x, y) in the frame time interval.

  Thereafter, the importance d is calculated by the following equations (1) to (4). That is, if the pedestrian T is projected in the area A and the moving direction of the pedestrian T is left, the importance d is calculated by the equation (1). If the pedestrian T is projected in the area A and the movement direction of the pedestrian T is right, the importance d is calculated by the equation (2). Also, if the pedestrian T is projected in the region B and the moving direction of the pedestrian T is left, the importance d is calculated by Expression (3). Furthermore, if the pedestrian T is projected in the area B and the moving direction of the pedestrian T is right, the importance d is calculated by the equation (4). Here, k1 to k4 are constants and satisfy k4 <k1 <k3 <k2.

  The road vanishing point D may be detected by, for example, detecting a straight line by Hough transform or a clustering technique and detecting from the intersection of these straight lines.

  Returning to FIG. 1, the warning unit 22 includes a display device such as an LCD and a sound generation device such as a speaker. If the importance d calculated by the importance calculation unit 20 is equal to or greater than a predetermined value, the display device is displayed. The vehicle driver is warned by displaying a message to that effect or by outputting a warning sound by a sounding device.

  The importance d calculated by the importance calculation unit 20 is also supplied to the analysis region determination unit 18. The analysis region determination unit 18 generates analysis region data based on the importance d and the attention position vector Xs supplied from the image analysis unit 176.

  FIG. 6 shows the configuration of the analysis region determination unit 18 in detail. As shown in the figure, the analysis area determination unit 18 manages three self-organizing maps (SOMs), a short-term storage SOM management unit 18s, and a medium-term storage SOM management unit 18m. And SOM 181 for long-term storage.

The short-term memory SOM management unit 18s manages the short-term memory SOM (first self-organizing map) shown in FIG. 7, and updates and outputs the connection weight vector W ^s _{p, q} of the short-term memory SOM. Calculate the vector. The short-term memory SOM is input with the target position vector X ^s, and the connection weight vector W ^s _{p, q} for each of the units U ^s _{p, q} arranged in N rows and M columns in the competitive layer is input. It was updated by the target position vector X ^s and importance d.

That is, the short-term memory for SOM, the target position vector X ^s supplied from the image analyzing unit 16 is inputted. The target position vector X ^s is a two-dimensional vector as shown in the following equation (5).

Further, as shown in FIG. 6, the short-term memory for SOM management unit 18s, the coupling weight vector of long-term memory for SOM managed by long-term memory for SOM management unit 18l ^W _{l p, q} are fed back, The connection weight vector W ^s _{p, q} of the short-term memory SOM is updated according to the following equation (6).

Here, W ^sp _{, q} (t) is a combination weight vector of the short-term memory SOM at time t, and obj takes a value of 1 if the pedestrian T is projected in the frame image at time t, and is projected. If not, the value is 0. Β ^s is a given constant. Further, α ^s is calculated by the following equation (7) using the importance d.

Here, σ is a constant, p _c and q _c specify the winner unit in the competitive layer, and the winner unit is specified by the following equation (8).

The second term in the update formula (6) has a function of bringing the coupling weight vector W ^s _{p, q} closer to the target position vector X ^s that is the input vector. In this case, the degree of approach to the target position vector X ^s is proportional to the degree of importance d by the equation (7). The third term has a function of bringing the connection weight vector W ^s _{p, q} closer to the connection weight vector W ^l _{p, q} of the long-term storage SOM. According to the update formula (6), the connection weight vector W ^s _{p, q} of the short-term storage SOM is normally close to the connection weight vector W ^l _{p, q} of the long-term storage SOM, and the attention position vector X ^s There is input, becomes closer to the target position vector X ^s are affected.

In determining the output of the short-term memory SOM, the short-term memory SOM management unit 18s calculates the density conf ^s _{p, q} of each of the connection weight vectors W ^s _{p, q} by the following equation (9).

^Here, _{d s p, q} is the unit ^U _{s p,} the coupling weight vector ^W of _{q s p,} and _q, the surrounding unit _{^{U s p + 1, q,}} U s p-1, q, U s p, q + 1 , U ^s _{p, q−1} , and weight averages W ^s _{p + 1, q} , W ^s _{p−1, q} , W ^s _{p, q + 1} , W ^s _{p, q−1} , and average distance, ).

Then, from the short-term memory SOM, the connection weight vector W ^s _{p, q} is output only for the unit U ^s _{p, q} whose density conf ^s _{p, q} is equal to or greater than the predetermined threshold θ _s . From the short-term memory for SOM management unit 18s, the coupling weight vector ^W _{s p, q,} the coupling weight vector ^W _{s p,} density conf ^s _p of the _{q, q} is inputted to the medium-term storage for SOM manager 18m Yes.

Moreover, short-term memory for SOM management unit 18s includes coupling weight vector ^W _{s p} of short-term memory for _SOM, among _q, density conf ^s _p, those corresponding to the maximum value of _q, as well as selected as the fixation point, The data of the surrounding area having a predetermined area is fed back to the image analysis unit 16 as analysis area data. Thereby, the image analysis processing in the image analysis unit 16 is limited to a partial region of the frame image.

The medium-term memory SOM management unit 18m manages the medium-term memory SOM (third self-organizing map) shown in FIG. 8, and updates and outputs the connection weight vector W ^m _{p, q} of the medium-term memory SOM. Calculate the vector. The combination weight vector W ^s _{p, q} of the unit U ^s _{p, q} of the short-term storage SOM whose density conf ^s _{p, q} is equal to or greater than the threshold θ _s is input to the medium-term storage SOM. Then, the connection weight vector W ^m _{p, q} for each of the units U ^m _{p, q} arranged in N rows and M columns in the competitive layer is the input connection weight vector W ^s _{p, q} and the density conf ^s _p, Updated by _q .

That is, the medium-term memory SOM includes a combination weight vector W ^s _{p, q} of the short-term memory SOM having a density conf ^s _{p, q} that is greater than or equal to the threshold θ _s as shown in the following equation (11). is input as input vector ^{X m.}

Then, the connection weight vector W ^m _{p, q} of the medium term storage SOM is updated according to the following equation (12).

Here, W ^m _{p, q} (t) is a coupling weight vector of the medium term storage SOM at time t, and obj takes a value of 1 if the pedestrian M is projected in the frame image at time t, and is projected. If not, the value is 0. Β ^m is a given constant. W ^init _{p, q} is an initial arrangement of the connection weight vector W ^m _{p, q} of the SOM for medium-term storage. Furthermore, alpha ^m is density conf ^s _p inputted from the mid-term storing SOM management unit _18s, with _q, is calculated by the following equation (13).

Here, sigma is a constant, the p _c and q _c is intended to identify the winner unit in the competitive layer, the winner unit is identified in the same manner as in the above formula (8).

The second term in the update equation (12) has a function of bringing the coupling weight vector W ^m _{p, q} closer to the input vector X ^m . In this case, the degree of approach the input vector ^{X m} is density conf ^s _p by the equation _(13), is proportional to _q. The third term has a function of bringing the coupling weight vector W ^m _{p, q} closer to the initial arrangement W ^init _{p, q} . According to the update formula (12), the connection weight vector W ^sp _{, q} of the medium term storage SOM is equal to the input vector X ^m , that is, the short term storage SOM is dense while the pedestrian T is displayed in the frame image. When the degree conf ^s _{p, q} approaches a high value of the connection weight vector W ^s _{p, q} and the pedestrian T disappears from the frame image, the initial position W ^init _{p, q} is approached.

In determining the output of the medium term storage SOM, the medium term storage SOM management unit 18m calculates the density conf ^m _{p, q} of each of the connection weight vectors W ^m _{p, q} by the following equation (14).

^Here, _{d m p, q} are unit ^U _{m p,} coupling weight vector ^W of _{q m p,} and _q, the surrounding unit _{^{U m p + 1, q,}} U m p-1, q, U m p, q + 1 , U ^m _{p, q−1} , and W ^m _{p + 1, q} , W ^m _{p−1, q} , W ^m _{p, q + 1} , W ^m _{p, q−1,} and the average distance, ).

When the pedestrian T disappears from the frame image, the combined weight vector W ^m _{p, q} is output from the medium term storage SOM. Then, the medium-term storage for SOM management unit 18m, the connection weight vector ^W _{m p, q,} the coupling weight vector ^W _{m p,} density conf ^m _p according to _{q, q} is input into long-term memory for SOM management unit 18l Is done.

The long-term storage SOM management unit 181 manages the long-term storage SOM (second self-organizing map) shown in FIG. 9, and updates and outputs the connection weight vector W ^l _{p, q} of the long-term storage SOM. Calculate the vector. When the pedestrian M disappears from the frame image _, the combination weight vector W ^m _{p, q} of the unit U ^m _{p, q} of the medium-term storage SOM is input to the long-term memory SOM. Then, the connection weight vectors W ^l _{p, q} for each of the units U ^l _{p, q} arranged in N rows and M columns in the competitive layer are the input connection weight vectors W ^m _{p, q} and the density conf ^m _p, Updated by _q .

That is, to the long-term memory SOM, the connection weight vector W ^m _{p, q} of the medium-term memory SOM is input as the input vector X ¹ as shown in the following equation (15).

Then, the connection weight vector W ^l _{p, q} of the long-term storage SOM is updated according to the following equation (16).

Here, W ^l _{p, q} (t) is a connection weight vector of the long-term memory SOM at time t, and Δt is the storage time of the medium-term memory SOM (after the pedestrian M appears in the frame image, the pedestrian T Is the time until disappearance). Furthermore, α ^l is calculated by the following equation (17) using the density conf ^m _{p, q} input from the medium term storage SOM management unit 18m.

Here, sigma is a constant, the p _c and q _c is intended to identify the winner unit in the competitive layer, the winner unit is identified in the same manner as in the above formula (8).

The second term in the update equation (16) has a function of bringing the coupling weight vector W ^l _{p, q} closer to the input vector Xl. At this time, the degree of approach to the input vector X ^l is proportional to the density conf ^m _{p, q according} to the equation (17). According to the update equation (12), the connection weight vector W ^l _{p, q} of the long-term storage SOM approaches the connection weight vector W ^m _{p, q} of the medium-term storage SOM. Coupling weight vector ^W _{l p, q} thus updated is fed back to the short-term memory for SOM management unit 18s, the coupling weight vector ^W _{s p} of short-term memory for _SOM, used for updating of _q.

According to the present embodiment, the connection weight vector W ^l _{p, q} of the long-term storage SOM reflects the past attention position vector X ^s . On the other hand, the coupling weight vector ^W _{s p} of short-term memory for _{SOM, q} is a bond weight vector ^W _{l p} of long-term memory for SOM having such _character, the basis of _q, there the effects of recent interest position vector ^{X s} It will be added. Therefore, when the target position vector X ^s that approximates the past target position vector X ^s is obtained from the frame image captured by the video image camera 12, at least a portion corresponding to the noted position vector X ^s, quickly, it changes to those very close to the noted position vector X ^s. Further, if there is no input of the attention position vector X ^s (if the pedestrian M disappears from the frame image), the connection weight vector W ^s _{p, q} of the short-term storage SOM becomes the connection weight of the long-term storage SOM as time passes. It approaches the vector W ^l _{p, q} . Therefore, when the target region of the image analysis processing (presence / absence of pedestrian T and position coordinate calculation of pedestrian T) in the frame image is determined based on the combination weight vector W ^sp _{, q} of the short-term memory SOM, walking If the person T appears in the frame image, the area corresponding to the position becomes the target area for the image analysis process, and if the pedestrian T disappears from the frame image, the area corresponding to the past attention position becomes the target area for the image analysis process. It becomes. For this reason, like a human being, the present and past knowledge can be comprehensively considered, and the detection process of the pedestrian T can be performed on an appropriate location in the frame image.

FIG. 10 shows a state in which a moving image in front of the vehicle is processed using the image analysis apparatus 10 according to the present embodiment. The uppermost part of the figure shows frame images at times t = 0, t = 10, t = 20, t = 30, and t = 40. The rectangles represented in the upper frame images indicate the position of the pedestrian T. Also, below each frame image, a combination weight vector W ^s _{p, q of} t = 0, t = 20, t = 30, and SOM for short-term storage, SOM for medium-term storage, and SOM for long-term storage at t = 40, W ^m _{p, q} and W ^l _{p, q} are shown in order from the top. Coupling weight vector W ^{s p} short for _SOM, the figure _q is gazing point is indicated by circles. As can be seen from these figures, when the pedestrian T is projected in the frame image, the position close to the position is usually the gazing point (t = 10, 20, 30). However, even if the pedestrian T is displayed in the frame image, if the importance d is reduced due to the pedestrian T moving to the end of the frame image or the like, the point of gazing will move away from the position of the pedestrian T, It moves to a position corresponding to the coupling weight vector W ^l _{p, q} of the storage SOM. As described above, according to the present embodiment, the pedestrian T can be detected at an appropriate location in the frame image in consideration of current and past knowledge as if they were humans.

1 is a configuration diagram of an image analysis apparatus according to an embodiment of the present invention. It is a figure which shows an example of moving image data. It is a figure which shows an example of the frame image data cut out from moving image data. It is a figure which shows an analysis object area | region. It is a figure explaining the process which calculates the importance of an attention position. It is a block diagram of an analysis area judgment part. It is a figure which shows SOM for short-term memory. It is a figure which shows SOM for medium term memory | storage. It is a figure which shows SOM for long-term memory | storage. It is a figure which shows the operation example of the image analysis apparatus which concerns on this embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Image analysis apparatus, 12 Moving image camera, 14 Frame image acquisition part, 16 Image analysis part, 18 Analysis area judgment part, 18s SOM management part for short term memory, 18m SOM management part for medium term memory, 18l SOM management part for long term memory , 20 Importance calculation part, 22 Warning part.

Claims (7)

Image acquisition means for sequentially acquiring images;
Attention position vector determination means for determining an attention position vector in the images acquired sequentially,
First connection weight vector storage means for storing a plurality of connection weight vectors respectively corresponding to units arranged in a competitive layer of the first self-organizing map;
Second connection weight vector storage means for storing a plurality of connection weight vectors respectively corresponding to units arranged in the competitive layer of the second self-organizing map;
Based on the target position vector in the image determined by the target position vector determining unit, at least a part of the plurality of combined weight vectors stored in the first combined weight vector storage unit according to the target position vector. The plurality of connection weight vectors stored in the first connection weight vector storage means as the plurality of connection weight vectors stored in the second connection weight vector storage means as time elapses. A first connection weight vector updating means for approaching;
First output vector generation means for generating an output vector of the first self-organizing map based on the plurality of connection weight vectors stored in the first connection weight vector storage means;
Based on the output vector of the first self-organizing map generated by the first output vector generation means, according to the output vector among the plurality of connection weight vectors stored in the second connection weight vector storage means. Second connection weight vector updating means for bringing at least a part close to the output vector,
A predetermined image analysis process is performed on a portion of the image corresponding to the plurality of connection weight vectors stored in the first connection weight vector storage unit.
An image analysis apparatus characterized by that. The image analysis apparatus according to claim 1,
Third connection weight vector storage means for storing a plurality of connection weight vectors respectively corresponding to units arranged in the competitive layer of the third self-organizing map;
Based on the output vector of the first self-organizing map generated by the first output vector generation means, according to the output vector among the plurality of connection weight vectors stored in the third connection weight vector storage means. Third connection weight vector updating means for bringing at least a part close to the output vector;
Further comprising third output vector generation means for generating an output vector of the third self-organizing map based on the plurality of connection weight vectors stored in the third connection weight vector storage means;
The second connection weight vector updating unit is configured to store the plurality of the plurality of second combination weight vector storage units stored in the second connection weight vector storage unit based on the output vector of the third self-organizing map generated by the third output vector generation unit. At least part of the combination weight vector corresponding to the output vector is brought close to the output vector;
An image analysis apparatus characterized by that. In the image analysis device according to claim 1 or 2,
The predetermined image analysis process is a process of determining an attention position vector in the image by the attention position vector determination means.
An image analysis apparatus characterized by that. In the image analysis device according to any one of claims 1 to 3,
The attention position vector determination means determines the importance of the attention position vector together with the attention position vector in the image,
The first combination weight vector updating unit determines the importance of determining at least a part of the plurality of combination weight vectors stored in the first combination weight vector storage unit according to the target position vector together with the target position. Depending on the target position vector,
An image analysis apparatus characterized by that. The image analysis apparatus according to any one of claims 1 to 4,
The first output vector generation means calculates a density of the connection weight vectors corresponding to surrounding units for each of the plurality of connection weight vectors stored in the first connection weight vector storage means, Generating the connection weight vector selected according to the density as an output vector of the first self-organizing map;
An image analysis apparatus characterized by that. An image acquisition step for sequentially acquiring images;
An attention position vector determination step for determining an attention position vector in the images sequentially obtained;
Corresponding to each unit arranged in the competitive layer of the first self-organizing map stored in the first combined weight vector storage means based on the target position vector in the image determined by the target position vector determination step The plurality of connection weight vectors stored in the first connection weight vector storage means as time elapses while at least part of the plurality of connection weight vectors is close to the target position vector. A first connection weight vector update step of approaching a plurality of connection weight vectors respectively corresponding to units arranged in the competitive layer of the second self-organizing map, stored in the second connection weight vector storage means;
A first output vector generation step of generating an output vector of the first self-organizing map based on the plurality of connection weight vectors stored in the first connection weight vector storage means;
Based on the output vector of the first self-organizing map generated in the first output vector generating step, according to the output vector among the plurality of connection weight vectors stored in the second connection weight vector storage means. A second combined weight vector updating step for bringing at least a portion closer to the output vector,
A predetermined image analysis process is performed on a portion of the image corresponding to the plurality of connection weight vectors stored in the first connection weight vector storage unit.
An image analysis method characterized by the above. Image acquisition means for sequentially acquiring images;
Attention position vector determination means for determining an attention position vector in the images acquired sequentially,
First connection weight vector storage means for storing a plurality of connection weight vectors respectively corresponding to units arranged in a competitive layer of the first self-organizing map;
Second connection weight vector storage means for storing a plurality of connection weight vectors respectively corresponding to units arranged in the competitive layer of the second self-organizing map;
Based on the target position vector in the image determined by the target position vector determining unit, at least a part of the plurality of combined weight vectors stored in the first combined weight vector storage unit according to the target position vector. The plurality of connection weight vectors stored in the first connection weight vector storage means as the plurality of connection weight vectors stored in the second connection weight vector storage means as time elapses. A first connection weight vector updating means for approaching;
First output vector generation means for generating an output vector of the first self-organizing map based on the plurality of connection weight vectors stored in the first connection weight vector storage means; and the first output vector generation means Based on the output vector of the first self-organizing map generated by the above, at least a part of the plurality of connection weight vectors stored in the second connection weight vector storage means according to the output vector is output. Causing the computer to function as a second connection weight vector update means that approaches a vector,
Causing the computer to perform a predetermined image analysis process on a portion of the image corresponding to the plurality of connection weight vectors stored in the first connection weight vector storage unit;
A program characterized by that.