JP2007201556A - Congestion information providing system and method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a congestion information providing system that can determine a congestion situation by one camera, has processing speed that is faster than that when an individual is recognized, and can ensure real-time properties; and to provide a congestion information providing method. <P>SOLUTION: There are provided a fixed-point camera 10 for observing the congestion situation, a congestion information arithmetic unit 20 connected to the fixed-point camera 10, and an information display 40 for displaying the congestion information. The congestion information arithmetic unit 20 comprises a person detection means for detecting a person region contained in an image photographed by the fixed-point camera 10; a measurement means for measuring the number of persons in the image, based on the person region detected by the person detection means; and a providing means for providing a user with congestion information, based on a result measured by the measurement means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a congestion information providing system and method, and more particularly to a technology for providing a user with congestion information of a place where people gather.

Conventionally, a fixed point camera has been installed at an observation point, and the amount of traffic, passage time, and the like are grasped based on the image. For example, in Patent Document 1, a camera is installed at each measurement start point and measurement end point, a person is randomly extracted from an image taken at the measurement start point, and the facial features are stored in the storage unit. Moreover, the pedestrian extracted from the image photographed with the camera at the measurement end point is extracted, and the facial feature is extracted. Then, the passage information is output based on the image acquisition time at the measurement start point and the measurement end point of the pedestrian whose face feature matches with the face feature recorded in the storage unit.
JP 2003-346279 A

  However, Patent Document 1 has the following problems. First, the face overlaps in the image, and the feature points of the face cannot be extracted sufficiently, and the discrimination rate of the same person decreases. Secondly, a storage capacity for storing feature point information of a plurality of persons for a certain period of time is required, and privacy problems are caused by using the face feature information. Thirdly, processing time increases due to personal identification and lacks real-time properties. Fourth, a plurality of cameras are required for measurement, and measurement cannot be performed with a single camera.

  The present invention has been made in view of the above problems, and can provide a congestion information providing system that can determine the congestion status with a single camera, and can achieve higher real-time performance and a higher processing speed than a case of personal recognition. It aims to provide a method.

  In order to solve the above problems, the present invention provides a fixed point camera for observing a congestion situation, a person detection unit for detecting a person region included in an image taken by the fixed point camera, and the person detection unit And measuring means for measuring the number of persons in the image based on the person area detected by the user, and providing means for providing the user with congestion information based on the measurement result of the measuring means.

  As a result, it is possible to detect the person from the captured image of the fixed point camera and grasp the congestion situation.

  Further, it may further comprise a determining means for determining that the person is congested if the number of persons is equal to or greater than a predetermined threshold, and for determining that the person is not congested if the number of persons is less than the predetermined threshold.

  Thereby, it can be determined whether or not it is congested based on the comparison between the number of persons and the threshold value, and the result can be provided.

  The image processing apparatus further includes detection means for detecting an area of a walkable area in the image, and setting means for setting an area of a person occupation area occupied by a person in the image. The predetermined threshold value may be changed according to a ratio of the area of the person-occupied area to the area of the walkable area.

  Thus, when the shooting area is changed, for example, when wide-area shooting or zoom shooting is performed, a threshold value corresponding to the scale can be set.

  The present invention also provides a fixed point camera for observing the congestion situation, storage means for storing a reference image taken in a situation where there is no person at the observation point by the fixed point camera, and observation of the congestion situation by the fixed point camera. And a difference unit that performs a difference process between the observation-time image captured at the time and the reference image, and a providing unit that provides a user with congestion information based on a result of the difference process.

  Thereby, the congestion information based on the difference result between the reference image and the observation image can be provided.

  Further, a difference value indicating a difference amount between the reference image and the observation image is calculated based on a result of the difference processing, and if the difference value is equal to or greater than a predetermined threshold, it is determined that the area is congested. If the difference value is less than a predetermined threshold value, it may further include a determination unit that determines that the difference is not congested.

  Thereby, the result which discriminate | determined the congestion condition based on the difference result can be provided.

  Further, the reference image is an image obtained by photographing a traffic surface of the observation point, the difference unit generates a difference image obtained by performing a difference process on the reference image from the observation time image, and the determination unit includes the reference unit The respective areas of the traffic surface area where the traffic surface in the image is photographed and the difference area included in the difference image are measured, and the traffic surface exposure area obtained by subtracting the difference area from the traffic surface area is the difference value. May be calculated as

  The providing means receives the congestion information transmission request information from the terminal device operated by the user, and transmits / receives the congestion information to the terminal apparatus according to the transmission request information; Means for unilaterally delivering the congestion information to a terminal device operated by a person, means for posting the congestion information on a homepage, and a public display device for displaying the congestion information installed at a fixed position Or at least one of them.

  As a result, when providing congestion information on a terminal device or a homepage, it is possible to provide congestion status to users in remote locations, and when displaying on a public display device, the public display device is visually recognized. Congestion information can be provided to all who can.

  Further, a time measuring unit that generates time information by measuring the time at which the congestion information is generated, and a prediction unit that calculates prediction information of congestion status at a certain time based on the time information and the congestion information. The provision unit may further provide the prediction information.

  Thereby, congestion can be estimated and congestion reduction can be aimed at.

  Further, the fixed point camera is installed for each section dividing the wide area space into a plurality of sections, and distribution information generating means for generating distribution information indicating the distribution of the congestion situation in the wide area space based on the congestion information for each section. Further, it may be provided.

  Thereby, the user can grasp | ascertain the congestion distribution of wide area space.

  The congestion information providing method according to the present invention includes a step of capturing an image of an observation point of a congestion situation with a fixed point camera, a step of detecting a person area included in the image, and And a step of providing congestion information based on the measurement result to the user.

  The congestion information providing method according to the present invention includes a step of capturing an image of an observation point of a congestion situation with a fixed point camera under a situation where there is no person at the observation point and storing it as a reference image, and the fixed point camera. Based on the result of the difference processing, the step of generating an image at the time of observation of the observation point at the time of observation of the congestion state, the step of reading the reference image and performing a difference process between the reference image and the image at the time of observation Providing congestion information to a user.

  According to the present invention, it is possible to grasp the congestion situation with one camera. Furthermore, since the congestion state is observed without face recognition processing for detecting a specific person, the processing speed is higher than that in the case of face recognition processing, and real-time performance can be secured.

  Hereinafter, a congestion information providing system and method according to the present invention will be described in detail with reference to the accompanying drawings.

<First embodiment>
FIG. 1 is a conceptual diagram of a congestion information providing system 1 to which the present invention is applied. The congestion information providing system 1 installs the fixed point camera 10 at the observation point of the congestion situation, measures the number of passersby within the photographing area 11, and determines the congestion situation based on this. The image 12 is an image photographed by the fixed point camera 10. The image 12 includes a region 13 that cannot be walked due to a structure such as a building or a wall, and a walkable sidewalk region 14. Yes. A dotted circle indicates a detected person. By installing the fixed-point camera 10 at a ticket gate of a station, a platform, an amusement park attraction, a ticket booth, a shopping street restaurant, or the like, a user at a remote location can grasp the congestion situation of each place.

  FIG. 2 is a block diagram showing a schematic configuration of the congestion information providing system 1 according to the first embodiment. The congestion information providing system 1 includes a fixed point camera 10, a congestion information calculation device 20 that determines a congestion state based on images of the fixed point camera 10, and an information display device 40 that displays congestion information by the congestion information calculation device 20. . The information display device 40 may be a device that can receive and display distributed congestion information, such as a terminal device operated by a user, such as a mobile phone, a PDA, a PC, a display device installed in a street or a facility, or the like. The information may be posted on the homepage, and the user may browse the homepage by operating the terminal device. The fixed point camera 10, the congestion information calculation device 20, and the information display device 40 are connected to each other by an existing communication network 41 such as the Internet, a public line network, and a wireless communication network.

  In the present embodiment, the fixed point camera 10 and the congestion information calculation device 20 are connected via the communication network 41. However, the fixed point camera 10 and the congestion information calculation device 20 may be connected by wire without using the communication network. Good. In this case, the congestion information calculation device 20 includes a camera I / F and acquires an image via the camera I / F.

  FIG. 3 is a functional block diagram of the congestion information calculation device 20 according to the first embodiment. The congestion information calculation device 20 includes a communication interface (hereinafter referred to as “communication I / F”) 21 for transmitting and receiving image data and congestion information via the communication network 41, an image acquisition unit 22 that reads an image of the fixed point camera 10, and A person detection unit 23 that executes a person detection algorithm for detecting a person from the image of the fixed point camera 10, a measurement unit 24 that measures the number of detected persons, and a level of congestion based on the measurement result of the measurement unit 24 A determination unit 25 for detecting the image, a detection unit 26 for detecting a walkable region included in the photographed image, a setting unit 27 for setting an area occupied by one person in the photographed image, and an area of the walkable region The calculation unit 28 that calculates a threshold value used by the determination unit 25 based on a value obtained by dividing the area occupied by one person from the date and time information that the determination unit 25 has determined the congestion situation A time measuring unit 29 for measuring the time, a log information storing unit 30 for storing log information in which the date and time measured by the time measuring unit 29 and the congestion information output by the determination unit 24 are associated, and the congestion situation is predicted based on the log information A prediction unit 31 that generates prediction information; and a provision unit 32 that provides congestion information and prediction information to a user.

  Next, based on FIG. 4, the observation of the congestion situation which the congestion information provision system 1 which concerns on this Embodiment performs is demonstrated. FIG. 4 is a flowchart showing a flow of congestion state observation processing according to the first embodiment.

  In step S1, the congestion information calculation device 20 acquires an image captured by the fixed point camera 10 via the network 41 and the communication I / F 21 (S1). The image acquisition unit 22 reads a captured image.

  In step S2, a person detection algorithm is applied to the image read by the person detection unit 23 in S1 to perform person detection (S2). As an example of the person detection algorithm, for example, when the fixed point camera 10 is a visible light camera, a skin color area of a photographed image is detected, and the detected area has a predetermined size or shape that can be assumed to be a human face area. Person detection may be performed based on whether or not it has. Further, contour extraction processing may be performed on the captured image, and a person may be detected using a rectangular shape circumscribing the extracted contour line. Alternatively, an infrared thermography camera that detects the surface temperature of an object and displays a difference in surface temperature as a color image may be used as the fixed point camera 10 to extract a region indicating a human body temperature and perform person detection. In addition, a known person detection algorithm may be used.

  In step S3, the measuring unit 24 measures the number of persons detected in S3 (S3).

  In step S4, the determination unit 25 determines the congestion status based on the number of persons detected in S3 (S4). For example, when the congestion status is determined by three levels, “congested”, “normal”, and “vacant”, a first threshold value and a second threshold value smaller than the first threshold value are set in advance. The determination unit 25 determines “congested” if the number of persons in S3 is equal to or greater than the first threshold, determines “normal” if the number is greater than or equal to the second threshold, and is less than the second threshold. If there is, it is determined as “free”. The determination unit 25 generates congestion information indicating the determined content. The first threshold value and the second threshold value may be arbitrarily set by the number of people who are considered to be congested by the administrator of the congestion information providing system 1, or may be automatically set by threshold setting processing described later. Good.

  In step S5, the timing unit 29 measures the date and time information on which the congestion information was generated in S4, generates log information that associates the date and time information with the congestion information, and stores it in the log information storage unit 30 (S5). .

  In step S6, the providing unit 32 provides the user with the congestion information generated in S4 (S6). The providing unit 32 may transmit the congestion information in response to a request from the information terminal device operated by the user, or may unilaterally distribute the congestion information to the information terminal device. In addition, congestion information may be displayed on a public display device installed in a street, a station, an amusement park, or may be posted on a homepage.

  In step S7, it is determined whether or not the observation is completed. If the observation is not completed, the process proceeds to “No”, returns to S1, and repeats the processes of S1 to S7 again. In the case of termination, the process proceeds to “Yes”, and the congestion state observation process is terminated.

  According to this congestion state observation process, since the person identification process is not performed only by extracting the person, the misperception rate is lower than that in the case of performing the identification process, and there is no risk of privacy infringement. In addition, since the determination process uses only the threshold value, the process is easy and fast, and the real-time nature of providing congestion information is excellent.

  Next, a threshold setting process executed by the congestion information providing system 1 will be described with reference to FIG. FIG. 5 is a flowchart for illustrating the flow of the threshold setting process.

  In step S11, the detection unit 26 detects a walkable area from the image acquired by the image acquisition unit 22 (S11). The detection unit 26 may automatically detect the walkable area 14 based on the color information of the walking surface, or a monitor and a mouse (not shown) are connected to the congestion information calculation device 20 so that the manager is displayed on the monitor. It is also possible to trace the edge of the walkable area in the image with the mouse and detect the area within the edge as the walkable area.

  In step S12, the setting unit 27 sets an area occupied by one person in the captured image (hereinafter referred to as “person occupied area”) (S12). Depending on the altitude and position (shooting angle) of the fixed point camera 10 with respect to the walking surface, the person occupation area in the shot image varies. This is because the higher the altitude, the smaller the person occupied area because the person is photographed smaller, and the lower the altitude, the larger the person occupied area because the person is photographed larger. In addition, when shooting from above the pedestrian's head and when shooting from the pedestrian's side or front and back, in general, shooting from the head is more than shooting from the side or front and back. The person occupation area is reduced. The administrator may input a numerical value of the person occupation area when it is assumed that an average person is photographed, and the setting unit 27 may set the person occupation area based on the input numerical value. The setting unit 27 may calculate the average value of the areas of the person regions detected by the person detection unit 23 and automatically set the person occupation area based on this value.

  In step S13, the calculation unit 28 calculates the area of the walkable region detected in S11, and calculates a threshold used by the determination unit 25 according to the value obtained by dividing the area of the walkable region by the person occupation area ( S13). The calculation unit 28 calculates a larger first threshold value and second threshold value because the larger the value obtained by dividing the area of the walkable area by the area occupied by the person, the greater the number of people that can pass through the walkable area. Conversely, the smaller the value obtained by dividing the area of the walkable area by the area occupied by the person, the smaller the number of people that can pass through the walkable area, so the smaller first threshold value and second threshold value are calculated.

  In step S14, the discrimination | determination part 25 sets the 1st threshold value and 2nd threshold value which were calculated by S13, and uses it for discrimination | determination of a congestion condition (S14).

  With this threshold setting process, when the fixed point camera 10 changes the shooting area such as wide area shooting or zoom shooting, the threshold corresponding to the scale is automatically set, so that the administrator of the congestion information providing system 1 is saved. Can do.

  Next, the prediction information provision process which the congestion information provision system 1 which concerns on this Embodiment performs based on FIG. 6 is demonstrated. FIG. 6 is a flowchart showing the flow of the prediction information providing process.

  In step S21, the prediction unit 31 reads the log information stored in the log information storage unit 30 (S21).

  In step S22, an approximate function that associates the time with the congestion status is generated based on the log information read by the prediction unit 31 (S22). The prediction unit 31 generates congestion prediction information at a certain time based on the generated approximate function. As an approximation function calculation method, as shown in FIG. 7A, a graph is prepared in which time is plotted on the horizontal axis and the degree of congestion is plotted on the vertical axis, and the read log information is plotted. Then, the approximate function may be calculated using a least square method. Further, as shown in FIG. 7B, the degree of congestion at a certain time may be obtained from the degree of concentration of plotted points without obtaining an approximate function. The algorithm used by the prediction unit 31 to generate the prediction information is not limited to the above, and any known algorithm can be used as appropriate. The prediction unit 31 may calculate the expected waiting time information using an approximate function.

  In step S23, the providing unit 32 provides the prediction information generated in S22 to the information display unit 40 (S23).

  In step S24, it is determined whether or not the prediction information providing process has ended. If not, the process proceeds to “No”, returns to step S21, and repeats the processes of S21 to S24 again. In the case of termination, the process proceeds to “Yes”, and the prediction information providing process is terminated.

  By this prediction information provision processing, it is possible to notify the user of a busy time or a non-crowded time, which can contribute to the suppression of the inflow of people at the time of congestion and the relief of congestion.

  Moreover, in the said embodiment, although the discrimination | determination part 25 discriminate | determined the congestion condition, the provision part 32 may provide the number of people which the measurement part 24 measured.

<Second embodiment>
In the second embodiment, the fixed point camera 10 generates a reference image in which a person is not present at the observation point in the congestion state in advance, and the congestion is based on a difference value between the reference image and the observation image taken at the time of observation. Determine the status.

  FIG. 8 is a functional block diagram of the congestion information calculation device 20 according to the second embodiment. The congestion information calculation device 20 according to the second embodiment replaces the person detection unit 23 and the measurement unit 24 of the congestion information calculation device 20 of the first embodiment, and a reference image storage unit 33 that stores a reference image, and a reference image And a difference unit 34 for performing a difference process between the image and the observation image.

  Next, based on FIGS. 9 and 10, the observation of the congestion status executed by the congestion information providing system 1 according to the present embodiment will be described. FIG. 9 is a flowchart showing a flow of congestion state observation processing according to the second embodiment. FIG. 10 is a schematic diagram showing the reference image 100, the observation image 101, and the difference image 102.

  In this process, the fixed-point camera 10 takes an image in a situation where there is no person at the observation point, and this image is stored as the reference image 100 in the reference image storage unit 33, and starts from this state.

  In step S31, the image acquisition unit 22 acquires the observation-time image 101 (same as the captured image of S1) from the fixed point camera 10 as in S1.

  In step S <b> 32, the difference unit 34 reads the reference image 100 from the reference image storage unit 33.

  In step S <b> 33, the difference unit 34 performs a difference process between the reference image 100 and the observation-time image 101 acquired in S <b> 31 to generate a difference image 102. Then, a difference value is calculated based on the pixel value of the difference image 102.

  In step S34, the determination unit 25 determines the congestion status based on the difference value in S33, as in S1. Since the difference value with the reference image 100 becomes larger as the observation image 101 has more people, the determination unit 24 compares the difference value with a predetermined threshold value, and is more crowded if the difference value is equal to or greater than the predetermined threshold value. If it is less than the predetermined threshold value, it is determined that it is more free.

  Steps S35 to S37 are the same processes as steps S5 to S7.

  According to the present embodiment, it is possible to determine the congestion state with only a simple difference process without using a person detection algorithm, and the processing speed can be increased.

  In step S <b> 33, the determination unit 25 subtracts the area of the difference area remaining in the difference image 102 from the area of the walkable area of the reference image 100 to obtain a traffic surface exposure area. You may judge that

<Third embodiment>
The third embodiment is an embodiment in which a wide area is divided into a plurality of sections and a fixed point camera 10 is arranged in each section.

  In FIG. 11, the wide area is divided into three sections of A block, B block, and C block in the east-west direction. Then, the fixed point camera 10 is installed in each block, and congestion information for each block is transmitted to the congestion information calculation device 20.

  In addition to the components of the congestion information calculation device 20 according to the first or second embodiment, the congestion information calculation device 20 according to the present embodiment is based on the congestion information for each section and the congestion status of the section. A distribution information generation unit that generates distribution information indicating Then, the distribution information generation unit arranges the congestion information of blocks A (congestion), B block (normal), and C block (empty) in FIG. 11 in the east-west direction, and the degree of congestion gradually from east to west. The distribution information that decreases is generated. The providing unit 32 provides congestion status for each section and / or distribution information of the congestion status.

  FIG. 12 is a screen display example in which the congestion status and the prediction information are displayed on the mobile phone used by the user as the information display device 40. The mobile phone 40 has a congestion status “block, normal, vacant” and waiting time information “waiting 15 minutes, waiting 5 minutes, waiting time 0” 121 and congestion distribution information “West exit is relatively free. 122 and prediction information “A block is expected to be free around 13:00” 123 are displayed.

  According to this embodiment, it is possible to provide congestion information as a wide area.

Conceptual diagram of a congestion information providing system 1 to which the present invention is applied The block diagram which shows schematic structure of the congestion information provision system 1 which concerns on 1st embodiment. Functional block diagram of the congestion information computing device 20 according to the first embodiment The flowchart which shows the flow of the observation process of the congestion condition which concerns on 1st embodiment. Flow chart for showing the flow of threshold setting processing Flow chart showing the flow of prediction information provision processing The schematic diagram which shows the processing content of the estimation part 31 Functional block diagram of the congestion information calculation device 20 according to the second embodiment The flowchart which shows the flow of the observation process of the congestion condition which concerns on 2nd embodiment. FIG. 10 is a schematic diagram showing a reference image 100, an observation time image 101, and a difference image 102. Schematic diagram of the congestion information providing system 1 according to the third embodiment Illustration of screen display example

Explanation of symbols

1: Congestion information providing system, 10: Fixed point camera, 20: Congestion information calculation device, 40: Information display device

Claims (11)

A fixed-point camera for observing congestion,
Person detecting means for detecting a person region included in an image captured by the fixed point camera;
Measuring means for measuring the number of persons in the image based on the person area detected by the person detecting means;
Providing means for providing the user with congestion information based on the measurement result of the measuring means;
A congestion information providing system characterized by comprising: A determination unit that determines that the person is congested if the number of persons is equal to or greater than a predetermined threshold, and further determines that the person is not congested if the number of persons is less than the predetermined threshold;
The congestion information providing system according to claim 1. Detecting means for detecting the area of the walkable area in the image;
Setting means for setting an area of a person occupation area occupied by a person per person in the image,
The determining means changes the predetermined threshold according to a ratio of the area of the person-occupied area to the area of the walkable area,
The congestion information providing system according to claim 2. A fixed-point camera for observing congestion,
Storage means for storing a reference image taken under a situation where there is no person at the observation point by the fixed point camera;
Difference means for performing a difference process between the image at the time of observation taken at the time of observation of the congestion situation by the fixed point camera and the reference image,
Providing means for providing congestion information based on the result of the difference processing to the user;
A congestion information providing system characterized by comprising: Based on the result of the difference processing, a difference value indicating a difference amount between the reference image and the observation image is calculated. If the difference value is equal to or greater than a predetermined threshold, it is determined that the image is congested. If it is less than a predetermined threshold, it further comprises a determination means for determining that it is not crowded.
The congestion information providing system according to claim 4. The reference image is an image obtained by photographing the traffic surface of the observation point,
The difference means generates a difference image obtained by performing a difference process on the reference image from the observation time image,
The discriminating means measures each area of a traffic surface area where a traffic surface in the reference image is photographed and a difference region included in the difference image, and subtracts the difference region area from the traffic surface area. A surface exposure area is calculated as the difference value;
The congestion information providing system according to claim 5. The providing means receives the congestion information transmission request information from a terminal device operated by the user, and transmits / receives the congestion information to the terminal apparatus according to the transmission request information; and the user Means for unilaterally delivering the congestion information to a terminal device to be operated; means for posting the congestion information on a homepage; and a public display device for displaying the congestion information installed at a fixed position. Composed of at least one of them,
The congestion information providing system according to any one of claims 1 to 6. A time measuring means for generating time information by measuring the time at which the congestion information is generated;
Prediction means for calculating prediction information of congestion status at a certain time based on the time information and the congestion information;
The providing means further provides the prediction information;
The congestion information providing system according to any one of claims 1 to 7. The fixed point camera is installed for each section that divides a wide area into a plurality of sections,
Distribution information generating means for generating distribution information indicating the distribution of the congestion status in the wide area based on the congestion information for each section;
9. The congestion information providing system according to any one of claims 1 to 8. A step of taking an image of a congestion observation point with a fixed point camera;
Detecting a human region included in the image;
Measuring the number of persons in the image based on the person region;
Providing congestion information based on the measured result to the user;
A method for providing congestion information, comprising: Taking an image of the observation point of the congestion situation by a fixed point camera under the situation where there is no person at the observation point, and storing as a reference image;
Generating an observation-time image in which the observation point is photographed by the fixed-point camera when observing the congestion situation;
Reading the reference image and performing a difference process between the reference image and the observation image;
Providing congestion information based on the result of the difference processing to the user;
A method for providing congestion information, comprising:

Images