JP7321221B2 - Information processing device, program, determination method, and system - Google Patents

Description

The present invention relates to an information processing device, program, determination method, and system.

For example, by extracting feature points from consecutive frames of images taken by a TV camera of vehicles traveling on the road, determining whether or not they belong to the same vehicle, and grouping them, we can identify multiple vehicles. Techniques for separating, recognizing, and acquiring location data for each vehicle have been known (see, for example, Patent Document 1).
[Prior art documents]
[Patent Literature]
[Patent Document 1] JP-A-2002-56494

According to one embodiment of the present invention, an information processing device is provided. The information processing apparatus may include a coordinate information acquisition unit that acquires coordinate information indicating reference coordinates of a detection target in each of a plurality of time-series consecutive images. The information processing device establishes correspondence between a detection target in a first image and a detection target in a second image based on coordinate information of a first image and coordinate information of a second image among a plurality of images. You may have a department. In the information processing device, a movement line segment of a detection target connecting the reference coordinates of the detection target in the first image and the reference coordinates of the detection target in the second image corresponding to the detection target is a boundary line set in advance. A passage determination unit may be provided that determines that the detection target has passed through the boundary line segment when the detection target crosses the boundary line segment.

The matching unit performs the detection in which the Euclidean distance, which is the length of the movement line segment, is the shortest between the plurality of detection targets in the first image and the plurality of detection targets in the second image. The set of targets may be associated as before and after movement. The association unit aggregates the Euclidean distances of the associated pairs of detection targets to calculate a median value, and when the Euclidean distances of the pair of detection targets exceeds 1.5 times the median value , may not be associated as before and after movement of the detection target. The apparatus may further include an error information storage unit that stores error information including information on the detection target that has not been associated with the other detection target by the association unit. The associating unit is associated with none of the plurality of detection targets in the next image following the one image among the plurality of detection targets in one of the plurality of images. When there is the detection target, the error information may be referred to.

The error information storage unit may store the error information when the reference coordinates of the detection target that have not been associated are within a preset area. The preset area may be an area that includes the center of the image and does not include the edges of the image. The association unit may delete the error information of the detection target that has not been associated for each of the detection targets that have not been associated in a certain period of time. When it is determined that the detection target has passed through the boundary line segment, the movement vector is an angle between a movement vector that can be calculated from the movement line segment and a predetermined arbitrary point and a determination vector that can be obtained from the boundary line segment. may further include a vector determination unit that determines a passing direction with respect to the boundary line segment of .

When the angle between the determination vector and the movement vector is less than 90 degrees or greater than 270 degrees, the vector determination unit determines that the passing direction is a direction from the boundary line to the arbitrary point. good. The vector determination unit determines that the passing direction is opposite to the direction from the boundary line to the arbitrary point when an angle between the determination vector and the movement vector is greater than 90 degrees and less than 270 degrees. You may

A first count number that is the number of times that the detection target passes through the boundary line segment in the direction to the arbitrary point when the passing direction is determined to be the direction from the boundary line segment to the arbitrary point. A counting unit for addition may be further provided. When it is determined that the passing direction is opposite to the direction from the boundary line to the arbitrary point, the counting unit causes the detection target to move along the boundary line in a direction opposite to the direction to the arbitrary point. A second count number, which is the number of times of passage in the reverse direction, may be added.

When the vector determination unit determines that the passing direction is the direction from the boundary line to the arbitrary point, the count number, which is the number of times the detection target passes through the boundary line, may be added. Preferably, the count number may be subtracted when it is determined that the passing direction is opposite to the direction from the boundary line segment to the arbitrary point.

According to one embodiment of the present invention, there is provided a program for causing a computer to function as the information processing apparatus. According to a third aspect of the invention, there is provided a computer-implemented method of determination. The determination method may comprise acquiring coordinate information indicating reference coordinates of a detection target in each of a plurality of time-series consecutive images. The detection target in the first image and the detection target in the second image are associated with each other based on the coordinate information of the first image and the coordinate information of the second image among the plurality of images. It may have steps. A moving line segment of the detection target connecting the reference coordinates of the detection target in the first image and the reference coordinates of the detection target in the second image corresponding to the detection target is a predetermined boundary line. determining that the detected object has passed through the boundary segment if it crosses the segment.

According to one embodiment of the present invention, there is provided a system comprising the information processing device and an image processing device. The image processing apparatus may include a moving image acquiring section that acquires the moving image captured by an imaging section that captures a moving image including the detection target. An image conversion unit may be provided that converts each of the plurality of frames of the moving image into a processed image to be subjected to image processing in chronological order. A coordinate information calculation unit may be provided for each of the processed images, which detects the detection target in the processed image and calculates the coordinate information indicating the reference coordinates of the detection target. A transmitting unit that transmits the coordinate information to the information processing device may be provided. The information processing device may be arranged in a cloud, the image processing device may be arranged at an edge and function as an edge server, and the transmission unit may transmit the coordinate information to the information processing device via a network. can be sent to The moving image acquiring unit may delete the moving image in response to the transmitting unit transmitting the coordinate information to the information processing device.

It should be noted that the above summary of the invention does not list all the necessary features of the invention. Subcombinations of these feature groups can also be inventions.

An example system 10 is shown schematically. An example of a process flow in system 10 is shown schematically. An example of association of detection targets in the information processing apparatus 100 is schematically shown. An example of the determination that the information processing apparatus 100 has passed is schematically shown. FIG. 3 is an explanatory diagram for explaining the contents of processing by the information processing apparatus 100; Another example of association of detection targets in the information processing apparatus 100 is schematically shown. vinegar. Another example of the determination of passage in the information processing apparatus 100 is schematically shown. Another example of the determination of passage in the information processing apparatus 100 is schematically shown. 1 schematically shows an example of functional configurations of an information processing apparatus 100 and an image processing apparatus 200. FIG. 1 schematically shows an example of a hardware configuration of a computer 1200 functioning as an information processing device 100 or an image processing device 200. FIG.

There is a demand for a technique for counting the number of moving objects (eg, vehicles, people, etc.) that have passed through an arbitrarily specified line segment in a moving image. It is conceivable to realize such technology by cloud services, but there are problems such as an increase in network traffic due to transmission of moving images to the cloud and the possibility of leakage of personal information. In the system 10 according to the present embodiment, for example, for a moving image containing personal information, heavy image processing (object detection) is performed on the edge side, then the moving image is deleted, only the coordinate information is uploaded to the cloud, and the A format in which moving objects are counted using coordinate information on the side is adopted. As a result, network traffic can be reduced, and personal information protection can be achieved by not transmitting personal information over the network.

Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all combinations of features described in the embodiments are essential for the solution of the invention.

FIG. 1 schematically illustrates an example system 10 . The system 10 includes an information processing device 100 and at least one image processing device 200 .

Networks 20 include, for example, mobile communication networks and the Internet. The mobile communication network complies with any of the 3G (3rd Generation) communication method, the LTE (Long Term Evolution) communication method, the 5G (5th Generation) communication method, and the communication method after the 6G (6th Generation) communication method. good too. For example, the information processing device 100 is arranged in the cloud, and the image processing device 200 is arranged at the edge of the mobile communication network. The image processing device 200 may be a so-called edge server.

The image processing device 200 performs image processing on a moving image in which a detection target is captured. A detection target is any moving object, and examples thereof include, but are not limited to, a person, a car, and the like.

The image processing device 200 may acquire a moving image to be subjected to image processing from a camera included in the image processing device 200 . The camera included in the image processing device 200 may be a camera built into the image processing device 200 or a camera connected to the image processing device 200 . The image processing device 200 may receive a moving image to be subjected to image processing from a camera connected to the image processing device 200 via a network. The image processing device 200 may receive a moving image to be subjected to image processing from any communication device via a network.

The image processing apparatus 200 may divide a moving image obtained by imaging a detection target into images for each frame, detect information on the detection target by image processing, and transmit the information to the information processing apparatus 100 . Based on the information acquired from the image processing device 200, the information processing device 100 counts, for example, the flow rate of the detection target passing through the reference line segment.

FIG. 2 schematically shows an example of a process flow in system 10. As shown in FIG. Here, the state before the image processing apparatus 200 acquires the captured moving image of the detection target is defined as the start state.

In S102, the image processing apparatus 200 acquires the captured moving image of the detection target. In S104, the image processing apparatus 200 converts the acquired moving image into an image for each frame. In S106, the image processing apparatus 200 performs image processing on each of the plurality of images converted in S104, and calculates coordinate information of the detection target.

In S<b>108 , the image processing device 200 transmits the coordinate information calculated in S<b>106 to the information processing device 100 . In S110, the information processing apparatus 100 associates detection targets for each piece of coordinate information. The coordinate information corresponding to the image of each frame represents the coordinates of each of the multiple detection targets on each image. This is because it is necessary to associate the same detection target between matching images. In S112, the information processing apparatus 100 determines whether or not the detection target has passed through a boundary segment preset for the image, based on the result of the association in S110. In S114, the information processing apparatus 100 outputs the result of the detection target passage determination. The information processing apparatus 100 displays and outputs the result, for example.

For example, when the information processing apparatus 100 acquires coordinate information corresponding to one moving image from the image processing apparatus 200, the information processing apparatus 100 collectively outputs results corresponding to the moving image. As a specific example, the information processing apparatus 100 outputs the number of detection targets that have passed through the boundary segment in one moving image. Further, for example, when the information processing device 100 continuously acquires the coordinate information corresponding to the moving image from the image processing device 200, the information processing device 100 may output the result in real time. As a specific example, the information processing apparatus 100 outputs a message to that effect each time the detection target passes through the boundary segment.

Conventionally, for example, in order to investigate the traffic volume of people in the city, the number of people entering and exiting a specific area, and the traffic volume of vehicles on the road, etc. There was a request to count the number. In such a case, as an example, a method has been adopted in which a moving image captured at a site where a detection target exists is directly transmitted to a server on the cloud. However, such a method has a problem of increasing network traffic during video analysis on the cloud. In addition, there has been a problem that personal information is included in the moving image to be transmitted, and the personal information cannot be protected.

The system 10, for example, performs high-load video processing (target detection) in the image processing device 200 on the edge side of the network, and transmits coordinate information of the detection target to the information processing device 100 on the cloud side via the network. Prepare. The coordinate information is not a moving image and does not include in-image feature amounts or personal information. The information processing apparatus 100 has a function of counting the movement of the detection target using only the coordinate information. As a result, network traffic can be reduced compared to the case where moving images are transmitted from the image processing apparatus 200 to the information processing apparatus 100, and personal information can be protected. Further, the load of the information processing apparatus 100 can be distributed to the image processing apparatus 200, and the processing load of the information processing apparatus 100 can be reduced. The image processing apparatus 200 may delete data including personal information as soon as processing is completed. This can further contribute to the protection of personal information.

FIG. 3 schematically shows an example of association of detection targets in the information processing apparatus 100. As shown in FIG. In the example shown in FIG. 3 , the plurality of detection targets 300 are obtained by enclosing the range of detection targets in the first image with a rectangle, and the plurality of detection targets 400 are each after the first image. The range of the detection target in the second image in the time series is enclosed by a rectangle. Reference coordinates 302 represent coordinates that serve as a reference for the detection target 300 . The reference coordinates 302 may be, for example, the center coordinates of a rectangular range. Reference coordinates 402 represent coordinates that serve as a reference for the detection target 400 . The reference coordinates 402 may be, for example, the center coordinates of a rectangular range.

The movement line segment 502 may be a line segment connecting the reference coordinates 302 and 402 . A moving line segment 502 can represent a trajectory along which the detection target has moved substantially between the shooting timings of the first image and the second image. The length of the moving line segment 502 may be the Euclidean distance as an example. Euclidean distance may be the usual distance between two points as measured by a ruler.

The information processing apparatus 100 associates, for example, the reference coordinates 302 of the detection target 300 detected in the past frame with the reference coordinates 402 of the detection target 400 detected in the current frame. good.

As an example of the method of association, first, the detection target 300 on the first image and the detection target 400 on the second image are drawn in the same area, and the reference coordinates 302 and 400 are drawn from the respective areas. 402 are calculated. Next, the reference coordinates with the shortest Euclidean distance between the reference coordinates 302 and 402 are associated as the same set of detection targets.

If there is a detection target that is framed out between the first image and the second image, the detection target is included in the first image but not included in the second image. . On the other hand, if the reference coordinates with the shortest Euclidean distance are associated with each other, the corresponding detection targets will be left over, and it is possible to prevent erroneous detection targets from being associated with each other. If there is a detection target framed in between the first image and the second image, the detection target is not included in the first image, but is included in the second image. . In this case as well, if the reference coordinates with the shortest Euclidean distance are associated with each other, the corresponding detection target will remain, and it is possible to prevent erroneous detection targets from being associated. In this way, by performing correspondence using the Euclidean distance, it is possible to improve the detection accuracy of the detection target.

As an example, the information processing apparatus 100 may not associate detection targets whose Euclidean distance exceeds 1.5 times the median value of the Euclidean distance. For example, the information processing apparatus 100 aggregates the Euclidean distances of the pairs of detection targets that have already been associated to calculate the median, and the Euclidean distances of the pair of detection targets for which association is newly determined is equal to the median. is more than 1.5 times, the set is not associated with before and after movement of the detection target. As a result, it is possible to prevent an association in which the Euclidean distance becomes an abnormal value, and it is possible to improve the detection accuracy of the detection target.

FIG. 4 schematically shows an example of determination that the information processing apparatus 100 has passed. Here, in the first image, the tracked detection target 300 is at the position of the tracked detection target 400 in the second image captured in time series after the first image due to movement. will be described as an example.

In the example shown in FIG. 4, a moving line segment 502 connecting the reference coordinates 302 in the first image and the reference coordinates 402 corresponding to the reference coordinates 302 intersects across a preset boundary line segment 506. there is In this case, the information processing apparatus 100 determines that the detection target 300 has passed through the boundary line segment 506 . Conversely, if the movement line segment 502 connecting the reference coordinates 302 in the first image and the reference coordinates 402 corresponding to the detection target 300 does not cross the preset boundary line segment 506, It is determined that the detection target does not pass through the boundary line segment 506 .

FIG. 5 is an explanatory diagram for explaining the contents of processing by the information processing apparatus 100. As shown in FIG. In the image processing in the image processing apparatus 200, even if detection of the detection target fails due to occlusion or detection omission occurs for some reason, the information processing apparatus 100 side can cope with this process. This process is for Occlusion is a situation in which an object to be detected is hidden by a cover or the like in a moving image and cannot be detected. Failure to detect may occur due to, for example, low object detection accuracy of the image processing apparatus 200 or too low resolution of moving images.

A target image outer edge 504 represents the outer edge of the target image. The information processing apparatus 100 identifies an assumed detection area 604 within the outer edge 504 of the target image. The assumed detection area 604 is near the center of the target image outer edge 504 and may be an area in which the detection target 300 is assumed to be detected. The information processing apparatus 100 sets, for example, the assumed detection area 604 within 10% of the outer circumference of the entire outer edge 504 of the target image.

Four detection targets 300 illustrated in FIG. 5 indicate the positions of specific detection targets in the 1st, 2nd, 5th, and 6th frames in images of 6 consecutive frames. The two failed targets 600 are the actual positions of the detection targets 300 in the third and fourth frames that have disappeared due to occlusion or the like due to tracking detection failure. The coordinate information acquired by the information processing device 100 from the image processing device 200 does not include the coordinates of the two failure targets 600 . Here, it is assumed that the information processing apparatus 100 attempts to obtain the reference coordinates 302 of the detection target 300 from the lower leftmost position for the first time to the upper right position for the sixth time with reference to the direction of FIG. 5 .

The information processing apparatus 100 successfully acquires the reference coordinates 302 on the first and second attempts, fails to acquire the reference coordinates 302 on the third and fourth attempts (referred to as reference coordinates 602 in the figure), and fails to acquire the reference coordinates 302 on the fifth and fourth attempts. Acquisition of the reference coordinates 302 is successful with the sixth time. In such a case, the information processing apparatus 100 attempts to associate the reference coordinates 302 acquired the second time with the reference coordinates 602 not detected the third time.

However, since the failure target 600 has not been detected, the reference coordinates 602 are not included in the coordinate information. Then, since the second reference coordinates 302 cannot be associated, the movement line segment 502 cannot be calculated. However, even in such a case, if the detection target 300 for which association has failed is in the assumed detection area 604, the reference coordinates 302 should be saved as error information, and the association should not be regarded as failure immediately. good.

The information processing apparatus 100 may set the number of times the association is not failed based on an arbitrary time in the moving image or an arbitrary parameter such as the number of associations, and allow the detection target 300 to survive. If the number of times the association is not considered unsuccessful is exceeded, the information processing apparatus 100 may delete the corresponding reference coordinates 302 from the error information and assume that they did not exist. As the association proceeds, the association between the third reference coordinate 602 and the fourth reference coordinate 602 does not fail because the reference coordinate 602 does not exist for each other.

However, the association between the fourth reference coordinates 602 and the fifth reference coordinates 302 fails again. Here, the information processing apparatus 100 may associate the second reference coordinates 302 saved in the error information with the fifth reference coordinates 302 by referring to the error information.

With this association, even if acquisition of the reference coordinates 302 of some of the detection targets 300 fails, if the movement line segment 502 intersects the preset boundary line segment 506, the detection target 300 has passed the boundary line segment 506. As a result, in the present embodiment, the uncorrelated reference coordinates 302 are stored as error information. The passage of 300 boundary line segments 506 can be determined.

A detection target positioned near the outer edge of the image may have gone out of the frame even if it disappeared in the middle. On the other hand, if the detection target located in the center of the image disappears in the middle, the possibility of frame-out is low and the possibility of occlusion or detection failure occurring is high. Therefore, by setting the assumed detection area 604 as an area that includes the center of the image but does not include the edges of the image, error information is stored only when the image disappears for reasons other than legitimate reasons such as frame-out. can be done. Further, by deleting the reference coordinates 302 that have not been detected for a certain period of time from the error information, the accuracy of the error information can be improved.

FIG. 6 schematically shows another example of association of detection targets in the information processing apparatus 100. As shown in FIG. The information processing apparatus 100 may use vectors to associate detection targets. The information processing apparatus 100 may determine the intersection with the boundary line segment 506 by the movement line segment 502 and determine the intersection direction using the movement vector 507 . A movement vector 507 is a vector connecting the reference coordinates 302 of the detection target 300 and the reference coordinates 402 of the detection target 400 , and may be calculated from the movement line segment 502 .

FIG. 7 schematically shows another example of determination that the information processing apparatus 100 has passed. FIG. 8 schematically shows another example of determination that the information processing apparatus 100 has passed.

The determination by the information processing apparatus 100 that the vehicle has passed may be made using a vector. An arbitrary point 508 is an arbitrarily set point. The arbitrary point 508 may be set to arbitrary coordinates as long as they are commonly used in determination processing in the information processing apparatus 100 . The arbitrary point 508 is set, for example, in the IN side area of the IN side area and the OUT side area separated by the boundary line segment 506 . Note that the arbitrary point 508 may be set in the area on the OUT side. A determination vector 509 is a vector that extends from the boundary line segment 506 toward an arbitrary point 508 so as to be perpendicular to the boundary line segment 506 and moves to a position that intersects the movement vector 507 .

For example, when the information processing apparatus 100 determines that the detection target 300 has passed through the boundary line segment 506, first, the information processing apparatus 100 identifies the movement vector 507 obtained from the movement line segment 502, and uses a preset arbitrary point 508 to to identify the decision vector 509 . Next, information processing apparatus 100 calculates angle 510 between movement vector 507 and determination vector 509 . Then, the information processing apparatus 100 determines the passing direction of the movement vector 507 with respect to the boundary line segment 506 based on the angle 510 . As illustrated in FIG. 7, the information processing apparatus 100 may determine that the passing direction is the direction from the boundary line segment 506 to the arbitrary point 508 when the angle 510 is less than 90 degrees or greater than 270 degrees. . As illustrated in FIG. 8, when the angle 510 is greater than 90 degrees and less than 270 degrees, the information processing apparatus 100 determines that the passing direction is opposite to the direction from the boundary line segment 506 to the arbitrary point 508. You can

As described above, the information processing apparatus 100 can determine the passing direction of the detection target with respect to the boundary line segment 506 by calculating the angle 510 between the movement vector 507 and the determination vector 509 . For example, when the information processing apparatus 100 determines that the passing direction is the direction from the boundary line segment 506 to the arbitrary point 508, the number of times the detection target 300 passes the boundary line segment 506 in the direction to the arbitrary point 508 is added, and when it is determined that the passing direction is opposite to the direction from the boundary line segment 506 to the arbitrary point 508, the detection object 300 moves the boundary line segment 506 to the arbitrary point A second count is added, which is the number of times the passage has passed in the direction opposite to the direction. This makes it possible to count the number of detection targets that have passed through the boundary line segment 506 toward the IN side and the number of detection targets that have passed through the boundary line segment 506 toward the OUT side.

When the information processing apparatus 100 determines that the passing direction is the direction from the boundary line segment 506 to the arbitrary point 508, the information processing apparatus 100 adds the count number, which is the number of times the detection target 300 has passed the arbitrary point 508, and determines the passing direction. However, if it is determined that the direction is opposite to the direction from the boundary line segment 506 to the arbitrary point 508, the count number may be subtracted. This makes it possible to count the difference between the number of detection targets that have passed through the boundary line segment 506 toward the IN side and the number of detection targets that have passed through the boundary line segment 506 toward the OUT side. For example, when the IN side is inside the room and the OUT side is outside the room, it is possible to count how many people are in the room.

FIG. 9 schematically shows an example of functional configurations of the information processing device 100 and the image processing device 200 . The image processing apparatus 200 includes a moving image acquisition section 202 , an image conversion section 204 , a coordinate information calculation section 206 and a transmission section 208 .

A moving image acquisition unit 202 acquires a moving image including a detection target. The moving image acquiring unit 202 may acquire moving images from a camera included in the image processing apparatus 200 . The moving image acquisition unit 202 may receive moving images from a camera connected to the image processing apparatus 200 via a network. The moving image acquisition unit 202 may receive moving images from any communication device via a network. The moving image acquisition unit 202 may store the acquired moving image in the moving image storage unit of the image processing device 200 .

The image conversion unit 204 converts each of the plurality of frames of the moving image into a processed image to be subjected to image processing in chronological order. The coordinate information calculation unit 206 detects a detection target in each processed image converted by the image conversion unit 204, and calculates coordinate information 103 indicating reference coordinates of the detection target. The transmission unit 208 transmits the coordinate information 103 calculated by the coordinate information calculation unit 206 to the information processing device 100 . The moving image acquiring unit 202 may delete the moving image in response to the transmitting unit 208 transmitting the coordinate information 103 to the information processing apparatus 100 . For example, when the transmitting unit 208 transmits the coordinate information 103 to the information processing apparatus 100, the moving image acquisition unit 202 deletes the moving image that is the source of the coordinate information 103 from the moving image storage unit.

The information processing apparatus 100 has a coordinate information acquisition unit 102 , a correspondence unit 104 , a passage determination unit 106 , an output control unit 108 , an error information storage unit 110 , a vector determination unit 114 and a count unit 116 . Note that it is not essential that the information processing apparatus 100 have all of these.

The coordinate information acquisition unit 102 acquires coordinate information 103 indicating reference coordinates 302 of a detection target 300 in each of a plurality of time-series consecutive images. The coordinate information acquisition section 102 may acquire the coordinate information 103 transmitted by the transmission section 208 .

Based on the coordinate information 103 acquired by the coordinate information acquisition unit 102, the association unit 104 associates each of a plurality of consecutive images in time series with a detection target. For example, the associating unit 104 selects the first image based on the reference coordinates 302 of the first image among the plurality of images and the reference coordinates 402 of the second image that is chronologically continuous with the first image. , and the detection target 400 in the second image are associated with each other.

The matching unit 104 selects the detection target whose Euclidean distance, which is the length of the moving line segment 502, is the shortest for the plurality of detection targets 300 in the first image and the plurality of detection targets 400 in the second image. The set may be associated as before and after movement of the detection target. The matching unit 104 aggregates the Euclidean distances of the matched pairs of detection targets to calculate a median value, and when the Euclidean distance of the pairs of detection targets exceeds 1.5 times the median value, Before and after movement may not be associated.

A passage determination unit 106 determines whether or not the detection target has passed through a boundary line segment 506 preset for the image. For example, the passage determination unit 106 determines that the movement line segment 502 connecting the reference coordinates 302 of the detection target 300 in the first image and the reference coordinates 402 of the detection target 400 in the second image corresponding to the reference coordinates 302 is If the detection target 300 crosses a preset boundary line segment 506 , it is determined that the detection target 300 has passed through the boundary line segment 506 .

The output control unit 108 controls to output the determination result by the passage determination unit 106 . The output control unit 108 causes the display provided in the information processing apparatus 100 to display and output, for example, the determination result by the passage determination unit 106 . The output control unit 108 may transmit the determination result by the passage determination unit 106 to another device and display it on the display of the device.

The output control unit 108 may output the result each time the passage determination unit 106 determines that the detection target has passed the boundary line segment 506 . For example, when the moving image is captured by a surveillance camera, it is possible to immediately grasp that the detection target has entered or left a predetermined area. The method of outputting the result is not limited to display output, and may be voice, light, data transmission, or the like.

The error information storage unit 110 stores error information 112 including information on the detection target 300 that has not been associated with another detection target 400 by the association unit 104 . The error information storage unit 110 stores the error information 112, for example, when the reference coordinates 302 of the detection target 300 that have not been associated are within a preset area.

The association unit 104 selects, among the plurality of detection targets 300 in one image among the plurality of images, the detection targets that are not associated with any of the plurality of detection targets 300 in the next image following the one image. 300 , the error information 112 may be referred to and associated with the detection target 300 . The associating unit 104 may delete the error information 112 of the detection target 300 that has not been associated for each detection target 300 that has not been associated in a certain period of time.

When the passage determination unit 106 determines that the detection target 300 has passed through the boundary line segment 506, the vector determination unit 114 performs a movement vector 507 that can be calculated from the movement line segment 502, an arbitrary point 508 set in advance, and a boundary line segment. The passing direction of the movement vector 507 with respect to the boundary line segment 506 is determined from the angle 510 with the determination vector 509 obtained from the line segment 506 .

For example, when the angle 510 between the determination vector 509 and the movement vector 507 is less than 90 degrees or greater than 270 degrees, the vector determination unit 114 determines that the passing direction is the direction from the boundary line segment 506 to the arbitrary point 508. If the angle 510 between the determination vector 509 and the movement vector 507 is larger than 90 degrees and smaller than 270 degrees, it is determined that the passing direction is opposite to the direction from the boundary line segment 506 to the arbitrary point 508 .

The counting unit 116 counts the number of times the detection target passes through the boundary segment 506 . For example, when the passing direction is determined to be the direction from the boundary line segment 506 to the arbitrary point 508, the counting unit 116 adds the count number that is the number of times the detection object 300 has passed the boundary line segment 506. , the passing direction is determined to be opposite to the direction from the boundary line segment 506 to the arbitrary point 508, the count number is subtracted.

Further, for example, when the passing direction is determined to be the direction from the boundary line segment 506 to the arbitrary point 508, the counting unit 116 determines that the detection object 300 passes the boundary line segment 506 in the direction to the arbitrary point 508. Add the first count number, which is the number of times When it is determined that the passing direction is opposite to the direction from the boundary line segment 506 to the arbitrary point 508 , the counting unit 116 counts the detection object 300 from moving the boundary line segment 506 to the arbitrary point 508 . A second count is added, which is the number of times the passage has passed in the direction opposite to the direction.

The output control unit 108 may control to output the result of counting by the counting unit 116 . For example, when the coordinate information acquisition unit 102 collectively acquires the coordinate information corresponding to one moving image, the output control unit 108 collectively outputs the results corresponding to the moving image. As a specific example, the output control unit 108 outputs the count number counted by the counting unit 116 for coordinate information corresponding to one moving image, or outputs the count number counted by the counting unit 116 for coordinate information corresponding to one moving image. controls to output the first count number and the second count number counted by .

FIG. 10 schematically shows an example of a hardware configuration of a computer 1200 functioning as the information processing device 100 or the image processing device 200. As shown in FIG. Programs installed on the computer 1200 cause the computer 1200 to function as one or more "parts" of the apparatus of the present embodiments, or cause the computer 1200 to operate or perform operations associated with the apparatus of the present invention. Multiple "units" can be executed and/or the computer 1200 can be caused to execute the process or steps of the process according to the present invention. Such programs may be executed by CPU 1212 to cause computer 1200 to perform certain operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

Computer 1200 according to this embodiment includes CPU 1212 , RAM 1214 , and graphics controller 1216 , which are interconnected by host controller 1210 . Computer 1200 also includes input/output units such as communication interface 1222 , storage device 1224 , DVD drive 1226 , and IC card drive, which are connected to host controller 1210 via input/output controller 1220 . DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, and the like. Storage devices 1224 may be hard disk drives, solid state drives, and the like. Computer 1200 also includes legacy input/output units, such as ROM 1230 and keyboard, which are connected to input/output controller 1220 via input/output chip 1240 .

The CPU 1212 operates according to programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. Graphics controller 1216 retrieves image data generated by CPU 1212 into a frame buffer or the like provided in RAM 1214 or itself, and causes the image data to be displayed on display device 1218 .

Communication interface 1222 communicates with other electronic devices over a network. Storage device 1224 stores programs and data used by CPU 1212 within computer 1200 . DVD drive 1226 reads programs or data from DVD-ROM 1227 or the like and provides them to storage device 1224 . The IC card drive reads programs and data from IC cards and/or writes programs and data to IC cards.

ROM 1230 stores therein programs that are dependent on the hardware of computer 1200, such as a boot program that is executed by computer 1200 upon activation. Input/output chip 1240 may also connect various input/output units to input/output controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, and the like.

The program is provided by a computer-readable storage medium such as DVD-ROM 1227 or IC card. The program is read from a computer-readable storage medium, installed in storage device 1224 , RAM 1214 , or ROM 1230 , which are also examples of computer-readable storage media, and executed by CPU 1212 . The information processing described within these programs is read by computer 1200 to provide coordination between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing information operations or processing according to the use of computer 1200 .

For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 executes a communication program loaded into the RAM 1214 and sends communication processing to the communication interface 1222 based on the processing described in the communication program. you can command. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as a RAM 1214, a storage device 1224, a DVD-ROM 1227, or an IC card, and transmits the read transmission data. Data is transmitted to the network, or received data received from the network is written in a receive buffer area or the like provided on the recording medium.

In addition, the CPU 1212 causes the RAM 1214 to read all or necessary portions of files or databases stored in external recording media such as the storage device 1224, DVD drive 1226 (DVD-ROM 1227), IC card, etc. Various types of processing may be performed on the data. CPU 1212 may then write back the processed data to an external recording medium.

Various types of information, such as various types of programs, data, tables, and databases, may be stored on recording media and subjected to information processing. CPU 1212 performs various types of operations on data read from RAM 1214, information processing, conditional decisions, conditional branching, unconditional branching, and information retrieval, which are described throughout this disclosure and are specified by instruction sequences of programs. Various types of processing may be performed, including /replace, etc., and the results written back to RAM 1214 . In addition, the CPU 1212 may search for information in a file in a recording medium, a database, or the like. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 selects the first attribute from among the plurality of entries. search for an entry that matches the specified condition of the attribute value of the attribute, read the attribute value of the second attribute stored in the entry, and thereby determine the first attribute that satisfies the predetermined condition An attribute value of the associated second attribute may be obtained.

The programs or software modules described above may be stored in a computer-readable storage medium on or near computer 1200 . Also, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, whereby the program can be transferred to the computer 1200 via the network. offer.
The blocks in the flowcharts and block diagrams in this embodiment may represent steps in the process in which the operations are performed or "parts" of the apparatus responsible for performing the operations. Certain steps and "sections" may be provided with dedicated circuitry, programmable circuitry provided with computer readable instructions stored on a computer readable storage medium, and/or computer readable instructions provided with computer readable instructions stored on a computer readable storage medium. It may be implemented by a processor. Dedicated circuitry may include digital and/or analog hardware circuitry, and may include integrated circuits (ICs) and/or discrete circuitry. Programmable circuits, such as Field Programmable Gate Arrays (FPGAs), Programmable Logic Arrays (PLAs), etc., perform AND, OR, EXCLUSIVE OR, NOT AND, NOT OR, and other logical operations. , flip-flops, registers, and memory elements.

A computer-readable storage medium may comprise any tangible device capable of storing instructions to be executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon may be illustrated in flowchart or block diagram form. It will comprise an article of manufacture containing instructions that can be executed to create means for performing specified operations. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer readable storage media include floppy disks, diskettes, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory) , electrically erasable programmable read only memory (EEPROM), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick , integrated circuit cards, and the like.

The computer readable instructions may be assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state configuration data, or instructions such as Smalltalk, JAVA, C++, etc. any source or object code written in any combination of one or more programming languages, including object-oriented programming languages, and conventional procedural programming languages such as the "C" programming language or similar programming languages; may include

Computer readable instructions are used to produce means for a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or programmable circuits to perform the operations specified in the flowchart or block diagrams. A general purpose computer, special purpose computer, or other programmable data processor, locally or over a wide area network (WAN) such as the Internet, etc., to execute such computer readable instructions. It may be provided in the processor of the device or in a programmable circuit. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, and the like.

In the above embodiment, the case where the information processing device 100 is arranged in the cloud and the image processing device 200 is arranged at the edge of the mobile communication network has been mainly described as an example, but the present invention is not limited to this. The information processing device 100 may be located in a mobile communication network, in which case the network 20 may be a mobile communication network. In this case, the information processing device 100 is arranged on the upper side of the mobile communication network, and the image processing device 200 is arranged on the lower side of the mobile communication network. Also, the information processing device 100 and the image processing device 200 may be arranged on the Internet, and in this case, the network 20 may be the Internet. Also, the information processing apparatus 100 and the image processing apparatus 200 may be arranged in a LAN (Local Area Network), and in this case, the network 20 may be a LAN.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is obvious to those skilled in the art that various modifications or improvements can be made to the above embodiments. It is clear from the description of the scope of claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The execution order of each process such as actions, procedures, steps, and stages in the devices, systems, programs, and methods shown in the claims, the specification, and the drawings is etc., and it should be noted that they can be implemented in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the specification, and the drawings, even if the description is made using "first," "next," etc. for convenience, it means that it is essential to carry out in this order. not a thing

10 system, 20 network, 100 information processing device, 102 coordinate information acquisition unit, 103 coordinate information, 104 association unit, 106 passage determination unit, 108 output control unit, 110 error information storage unit, 112 error information, 114 vector determination unit , 116 counting unit, 200 image processing device, 202 moving image acquisition unit, 204 image conversion unit, 206 coordinate information calculation unit, 208 transmission unit, 300 detection target, 302 reference coordinates, 400 detection target, 402 reference coordinates, 502 movement line Minute, 504 Target Image Outer Edge, 506 Boundary Line Segment, 507 Movement Vector, 508 Arbitrary Point, 509 Judgment Vector, 510 Angle, 600 Failure Target, 602 Reference Coordinates, 604 Assumed Detection Area, 1200 Computer, 1210 Host Controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input/output controller, 1222 communication interface, 1224 storage device, 1226 DVD drive, 1227 DVD-ROM, 1230 ROM, 1240 input/output chip

Claims (13)

a coordinate information acquisition unit that acquires coordinate information indicating reference coordinates of a detection target in each of a plurality of time-series consecutive images;
The detection target in the first image and the detection target in the second image are associated with each other based on the coordinate information of the first image and the coordinate information of the second image among the plurality of images. A correspondence unit, wherein, for the plurality of detection targets in the first image and the plurality of detection targets in the second image, reference coordinates of the detection targets in the first image and the detection targets. The set of detection targets with the shortest Euclidean distance, which is the length of the movement line segment of the detection target connecting the reference coordinates of the detection target in the second image corresponding to a mapping part that maps as
A moving line segment of the detection target connecting the reference coordinates of the detection target in the first image and the reference coordinates of the detection target in the second image corresponding to the detection target is a predetermined boundary line. a passage determination unit that determines that the detection target has passed through the boundary line segment when the detection target crosses the boundary line segment;
with
The association unit aggregates the Euclidean distances of the associated pairs of detection targets to calculate a median value, and when the Euclidean distances of the pair of detection targets exceeds 1.5 times the median value and an information processing apparatus that does not associate the detection target before and after the movement . 2. The information processing apparatus according to claim 1 , further comprising an error information storage unit that stores error information including information on said detection target that has not been associated with said other detection target by said association unit. a coordinate information acquisition unit that acquires coordinate information indicating reference coordinates of a detection target in each of a plurality of time-series consecutive images;
The detection target in the first image and the detection target in the second image are associated with each other based on the coordinate information of the first image and the coordinate information of the second image among the plurality of images. a correspondence part;
A moving line segment of the detection target connecting the reference coordinates of the detection target in the first image and the reference coordinates of the detection target in the second image corresponding to the detection target is a predetermined boundary line. a passage determination unit that determines that the detection target has passed through the boundary line segment when the detection target crosses the boundary line segment;
an error information storage unit that stores error information including information on the detection target that is not associated with the other detection target by the association unit;
Information processing device. The associating unit is associated with none of the plurality of detection targets in the next image following the one image among the plurality of detection targets in one of the plurality of images. 4. The information processing apparatus according to claim 2 , wherein when there is said detection target, said error information is referenced and said detection target is associated. 5. The error information storage unit according to any one of claims 2 to 4, wherein the error information is stored when the reference coordinates of the detection target that have not been associated are within a preset area. information processing equipment. 6. The information processing apparatus according to claim 5 , wherein said preset area is an area that includes the center of said image and does not include edges of said image. 7. The information according to any one of claims 2 to 6 , wherein the associating unit deletes the error information of the detection target that has not been associated for each of the detection targets that have not been associated in a certain period of time. processing equipment. A program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 7 . A computer-implemented determination method comprising:
Acquiring coordinate information indicating reference coordinates of a detection target in each of a plurality of time-series consecutive images;
The detection target in the first image and the detection target in the second image are associated with each other based on the coordinate information of the first image and the coordinate information of the second image among the plurality of images. a step of: regarding the plurality of detection targets in the first image and the plurality of detection targets in the second image, reference coordinates of the detection targets in the first image and correspondence to the detection targets; The pair of detection targets with the shortest Euclidean distance, which is the length of the movement line segment of the detection target connecting the reference coordinates of the detection target in the second image, is associated as before and after the movement of the detection target. stages and
A moving line segment of the detection target connecting the reference coordinates of the detection target in the first image and the reference coordinates of the detection target in the second image corresponding to the detection target is a predetermined boundary line. determining that the detection object has passed through the boundary segment if it crosses the boundary segment ;
Further comprising a step of calculating a median value by aggregating the Euclidean distances of the pairs of detection targets that have already been matched,
In the determination method, in the step of associating, when the Euclidean distance of the pair of detection targets exceeds 1.5 times the median value, the association is not performed as before and after movement of the detection target. A computer-implemented determination method comprising:
Acquiring coordinate information indicating reference coordinates of a detection target in each of a plurality of time-series consecutive images;
The detection target in the first image and the detection target in the second image are associated with each other based on the coordinate information of the first image and the coordinate information of the second image among the plurality of images. stages and
A moving line segment of the detection target connecting the reference coordinates of the detection target in the first image and the reference coordinates of the detection target in the second image corresponding to the detection target is a predetermined boundary line. determining that the detection target has passed through the boundary line segment if the detection target crosses the boundary line segment;
with
An error information storage step of storing error information including information of the detection target that was not associated with the other detection target in the matching step.
further comprising
judgment method. an information processing apparatus according to any one of claims 1 to 7 ;
comprising an image processing device and
The image processing device is
a moving image acquiring unit that acquires the moving image captured by an imaging unit that captures a moving image including the detection target;
an image conversion unit that converts each of the plurality of frames of the moving image into a processed image to be subjected to image processing in chronological order;
a coordinate information calculation unit that detects the detection target in the processed image for each of the processed images and calculates the coordinate information indicating the reference coordinates of the detection target;
and a transmitting unit configured to transmit the coordinate information to the information processing device. The information processing device is arranged in the cloud,
The image processing device is located at an edge and functions as an edge server,
12. The system according to claim 11 , wherein said transmission unit transmits said coordinate information to said information processing device via a network. The system according to claim 11 or 12 , wherein said moving image acquisition unit deletes said moving image in response to said transmission unit transmitting said coordinate information to said information processing device.

Images