JP7353918B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to information processing technology.

2. Description of the Related Art Conventionally, there is a technology that estimates the degree of crowding of people in a predetermined area by detecting people included in an image captured by an imaging device that captures an image of a predetermined area and counting the number of detected people. Patent Document 1 discloses a technique in which the number of people in a determination area is calculated, and the ratio of the number of people in the determination area to the maximum number of people expected in the determination area is calculated as the degree of congestion.

JP 2018-42049 Publication

However, when more people are located in an area that does not have seats, it is not possible to detect a person who is in a seat that is a blind spot due to the multiple people located in the area on the image, and as a result, the person is In some cases, it was not possible to accurately estimate the degree of congestion.

SUMMARY OF THE INVENTION Therefore, the present invention aims to suppress the decline in the accuracy of estimating the degree of crowding even when a person located in a seat is not visible in the image due to multiple people located in an area that does not have a seat. There is.

In order to solve the above problems, an information processing device of the present invention has the following configuration. That is, a detection means for detecting a person from an image captured by an imaging means in a predetermined area, and a detection means for calculating the number of people located in a first area without seats in the predetermined area based on the result of the detection means. 1 calculation means, and a second calculation means for calculating the number of people located in a second area which is an area different from the first area and which has seats in the predetermined area, The calculation means sets the number of people located in the second area to a predetermined number when the number of people calculated by the first calculation means is greater than or equal to a first threshold.

According to the present invention, even if a person located in a seat is not visible on the image due to a plurality of people located in an area that does not have a seat, it is possible to suppress a decrease in the accuracy of estimating the degree of crowding.

FIG. 1 is a diagram showing an example of a system configuration. FIG. 2 is a diagram showing functional blocks of an information processing device. FIG. 2 is a schematic diagram of the inside of a vehicle. FIG. 3 is a diagram for explaining a process of calculating the number of people in a vehicle. This is a table showing information on setting areas. This is a table showing size ranges corresponding to each setting area. It is a flowchart which shows the flow of processing which calculates the number of people in a vehicle. FIG. 3 is a diagram showing the hardware configuration of each device.

Embodiments of the present invention will be described below with reference to the accompanying drawings. Note that the configuration shown in the following embodiments is only an example, and the configuration is not limited to the illustrated configuration.

(Embodiment 1)
FIG. 1 is a diagram showing the system configuration in this embodiment. The system 10 in this embodiment includes, for example, an information processing device 100, an imaging device 110 installed on a vehicle 20 to image an area (predetermined area) inside a vehicle 20 such as a train or a bus, and a recording device 120. , and a display device 130. In the following description, the location where the imaging device 110 is installed will be described as the vehicle 20, but it may be any space where areas with seats and areas without seats coexist, such as a theater or stadium. It may be a spectator seat.

The information processing device 100, the imaging device 110, and the recording device 120 are interconnected via a network 140. The network 140 is realized, for example, by a plurality of routers, switches, cables, etc. that comply with communication standards such as ETHERNET (registered trademark).

Note that the network 140 may be realized by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like.

The information processing device 100 is realized, for example, by a personal computer or the like in which a program for realizing information processing functions described below is installed.

The imaging device 110 is a device that captures images. The imaging device 110 associates image data of the captured image, information on the imaging time at which the image was captured, and identification information that is information for identifying the imaging device 110, and connects the information processing device 100 and the recording device via the network 140. The information is sent to an external device such as device 120. Note that in the system 10 according to this embodiment, there is one imaging device 110, but there may be a plurality of imaging devices 110. For example, the imaging device 110 may be installed in each of the plurality of vehicles 20. In this case, each of the imaging devices 110 installed in each of the plurality of vehicles 20 is connected to the information processing device 100, the recording device 120, etc. via the network 140.

The recording device 120 records image data of an image captured by the imaging device 110, information on the imaging time at which the image was captured, and identification information for identifying the imaging device 110 in association with each other. Then, in accordance with a request from the information processing device 100, the recording device 120 transmits the recorded data (image, identification information, etc.) to the information processing device 100.

The display device 130 is configured with an LCD (Liquid Crystal Display) or the like, and displays results of information processing by the information processing device 100, images captured by the imaging device 110, and the like. The display device 130 is connected to the information processing device 100 via a display cable compliant with communication standards such as HDMI (registered trademark) (High Definition Multimedia Interface).

Further, the display device 130 functions as a display means, and displays images captured by the imaging device 110, results of information processing to be described later, and the like. Note that at least any two or all of the display device 130, the information processing device 100, and the recording device 120 may be provided in a single housing.

Note that the results of information processing by the information processing device 100 and the images captured by the imaging device 110 are not limited to the display device 130 connected to the information processing device 100 via a display cable, but can be stored in external devices such as the following. It may be displayed on a display included in the device. That is, it may be displayed on a display of an information device such as a smartphone or a tablet terminal connected via the network 140.

Next, information processing of the information processing apparatus 100 according to the present embodiment will be described with reference to functional blocks of the information processing apparatus 100 according to the present embodiment shown in FIG.

In this embodiment, each function shown in FIG. 2 is realized as follows using a ROM (Read Only Memory) 820 and a CPU (Central Processing Unit) 800, which will be described later with reference to FIG. shall be carried out. Each function shown in FIG. 2 is realized by the CPU 800 of the information processing device 100 executing a computer program stored in the ROM 820 of the information processing device 100.

The communication unit 200 can be realized by an I/F (Interface) 840, which will be described later with reference to FIG. 8, and communicates with the imaging device 110 and the recording device 120 via the network 140. The communication unit 200 receives, for example, image data of an image captured by the imaging device 110, and transmits a control command for controlling the imaging device 110 to the imaging device 110. Note that the control command includes, for example, a command that instructs the imaging device 110 to capture an image.

The storage unit 201 can be implemented by a RAM (Random Access Memory) 810, an HDD (Hard Disk Drive) 830, etc., which will be described later with reference to FIG. 8, and stores information and data related to information processing by the information processing device 100. For example, the storage unit 201 stores information regarding the position of a person detected from an image.

The output control unit 202 outputs images captured by the imaging device 110, a setting screen for making settings related to information processing according to this embodiment, information indicating the results of information processing, etc. to an external device, or displays them on the display device 130. Let me do it. Note that external devices to which information is output by the output control unit 202 include, for example, another information processing device (not shown) and the recording device 120.

The operation reception unit 203 receives an operation performed by a user via an input device (not shown) such as a keyboard or a mouse. For example, the operation accepting unit 203 accepts an operation that specifies an area on an image.

The setting unit 204 sets a setting area on the image to be used for determining the position of the person. The operation reception unit 203 in this embodiment accepts a user operation to specify a setting area in the image, and the setting unit 204 specifies the setting area in the image based on the information indicating the position of the setting area received by the operation reception unit 203. Set. Note that the position of the setting area is determined by the X and Y coordinates of each vertex of the setting area when the upper left end point of the image is the origin. The operation reception unit 203 receives information on the X and Y coordinates of each vertex in the setting area designated by the user, and the setting unit 204 receives information on the X and Y coordinates of each vertex in the setting area specified by the user. Set the settings area. Note that a detailed explanation regarding the setting area will be given later.

The detection unit 205 detects a person included in the image. The detection unit 205 detects a person included in the image by, for example, performing processing such as pattern matching using a matching pattern (dictionary). In this case, the person may be detected from the image using a plurality of matching patterns, such as a matching pattern when the person is facing forward and a matching pattern when the person is facing sideways. In this way, by performing detection processing using a plurality of matching patterns, it is expected that detection accuracy will be improved. Note that it is not always necessary to prepare a matching pattern (dictionary) that indicates the characteristics of the whole body, and matching patterns may be prepared for parts of the person such as the upper body, lower body, head, face, and feet.

The position determination unit 206 determines the area where the person detected by the detection unit 205 is located. Note that in this embodiment, an image captured by the imaging device 110 is an area corresponding to a passage area where a person can pass and a seat where a person can sit, and where more people are standing in the passage area. It is assumed that there is a back seat area that can be a blind spot at the top. In other words, the back seat area is an area located on the back side of the aisle area when starting from the installation position of the imaging device 110 in the vehicle 20. The position determination unit 206 in this embodiment determines whether the person detected by the detection unit 205 is located in the aisle area or the back seat area. Note that a detailed explanation of the position determination unit 206 will be given later.

The calculation unit 207 calculates the number of people who are determined to be located in an aisle area that does not have a seat, based on the detection result of the detection unit 205 with respect to the captured image. Further, the calculation unit 207 calculates the number of people located in the back seat area, which is an area having seats, based on the number of people determined to be located in the aisle area. A detailed explanation of the calculation process of calculating the number of people by the calculation unit 207 will be described later.

The crowding determination unit 208 determines the degree of crowding of people based on the results obtained by the calculation unit 207. For example, the congestion determination unit 208 compares the number of people in the vehicle 20 calculated by the calculation unit 207 with a threshold value, and determines that there is congestion when the number of people in the vehicle 20 is equal to or greater than the threshold value, and If the number is less than the threshold, it is determined that there is no congestion. Note that the determination result of the degree of congestion is not limited to two types, ie, whether or not it is crowded, but may be, for example, three or more types. For example, assume that a first threshold and a second threshold larger than the first threshold are set. At this time, the congestion determining unit 208 determines that it is not crowded if the number of people calculated by the calculating unit 207 is less than the first threshold, and if the number of people is equal to or greater than the first threshold and less than the second threshold. If the number of people is equal to or greater than the second threshold, the state is determined to be crowded.

Note that the congestion determination unit 208 determines the degree of crowding of people in the vehicle 20 based on the number of people calculated by the calculation unit 207 for each of the plurality of images taken inside the vehicle 20 by the imaging device 110. Good too. For example, the congestion determination unit 208 may determine the degree of congestion in the vehicle 20 by comparing the average value of the number of people calculated for each of the plurality of images with a threshold value. Further, the degree of congestion in the vehicle 20 may be determined by comparing not only the average value but also the median value of the number of people calculated for each of a plurality of images with a threshold value.

Note that the output control unit 202 outputs information indicating the determination result by the congestion determination unit 208 to an external device. For example, the output control unit 202 causes the display device 130 to display information indicating the determination result by the congestion determination unit 208. Note that the threshold value used by the congestion determination unit 208 to determine the degree of congestion is set by the setting unit 204, for example, based on an operation by the user to specify the threshold value. Further, assume that the imaging device 110 is installed in each of the plurality of vehicles 20. At this time, the congestion determination unit 208 executes a congestion degree determination for each of the plurality of vehicles 20, and the output control unit 202 outputs information indicating the congestion degree determination result for each of the plurality of vehicles 20. You can also do this. For example, the output control unit 202 may transmit information indicating which vehicles 20 are crowded among the plurality of vehicles 20 or information indicating which vehicles 20 are not crowded to the public transportation manager or vehicle concerned with the plurality of vehicles. It is also possible to notify 20 users.

Here, the information processing of the information processing apparatus 100 in this embodiment will be described in more detail with reference to FIGS. 3 to 6. FIG. 3 shows an example of the interior of the vehicle 20 when the vehicle 20 is assumed to be a bus. As shown in FIG. 3A, the vehicle 20 includes a driver's seat 301, a plurality of seats 302, and an entrance/exit 303.

Further, FIG. 3(b) shows three regions inside the vehicle 20 of FIG. 3(a). A passageway area 313 in FIG. 3(b) is an area corresponding to a passageway inside the vehicle 20, and indicates an area that does not have a seat. Further, the back seat area 312 is an area that has seats in the vehicle 20, and is an area that may become a blind spot on the image captured by the imaging device 110 due to many people standing in the traffic area. As shown in FIG. 3B, the back seat area 312 is an area located farther from the aisle area 313 starting from the installation position of the imaging device 110. Further, the front seat area 313 is an area that has seats in the vehicle 20, and is an area that does not become a blind spot on the image captured by the imaging device 110 even if many people are standing in the traffic area. . As shown in FIG. 3B, the front seat area 313 is an area located on the front side of the aisle area 311 from the installation position of the imaging device 110 as a starting point.

An image 400 shown in FIG. 4A shows an image of the inside of the vehicle 20 captured by the imaging device 110. Further, FIG. 4(a) shows a plurality of people sitting on seats in the vehicle 20. Further, the passage area 411 shown in FIG. 4(b) corresponds to the passage area 311 shown in FIG. 3(b). Similarly, the back seat area 412 and the front seat area 413 shown in FIG. 4(b) correspond to the back seat area 312 and the front seat area 313 shown in FIG. 3(b), respectively.

Further, FIG. 4C shows a situation where the number of passengers in the vehicle 20 has increased compared to the state shown in FIG. 4A, and a plurality of people are standing upright in the passage area 411. In the present embodiment, the calculation unit 207 performs the following process when the number of people determined to be located in the aisle area 411 and the number of people determined to be located in the front seat area 413 satisfy a predetermined condition. Execute. That is, the calculation unit 207 sets the number of people located in the back seat area 412 to a predetermined number. Specifically, the calculation unit 207 calculates that the number of people determined to be located in the aisle area 411 is at least a first threshold, and the number of people determined to be located in the front seat area 413 is at least a second threshold. In this case, the number of people in the back seat area 412 is set to a predetermined number. In this case, when the number of people determined to be located in the aisle area 411 is less than the first threshold, or the number of people determined to be located in the front seat area 413 is less than the second threshold, Perform processing such as the following: That is, the calculation unit 207 calculates the number of people determined by the position determination unit 206 to be located in the back seat area 412 as the number of people located in the back seat area 412.

As shown in FIG. 4C, when many people are located in the aisle area 411, the multiple people standing in the aisle area 411 make it difficult for the person located in the back seat area 412 to be seen on the image. . In this case, since it is difficult to detect people located in the back seat area 412 from the image, the number of people located in the back seat area 412 may not be appropriately reflected in the counting result. Therefore, in the calculation unit 207 in this embodiment, if the number of people located in the aisle area 411 is equal to or greater than the first threshold and the number of people located in the front seat area 413 is equal to or greater than the second threshold, Let the number of people located in the area 412 be a preset number. Generally, in a space such as a vehicle where seats and aisles coexist, if many people are standing in the aisle, it is likely that other people are already seated in the seats. In particular, if there are many people standing in the aisle and there are as many people as possible sitting in the seats that are not in the blind spot, may already be occupied by another person. Therefore, if the number of people located in the aisle area 411 is greater than or equal to the first threshold and the number of people located in the front seat area 413 is greater than or equal to the second threshold, the calculation unit 207 in this embodiment calculates the number of people located in the back seat area 412 It is assumed that the seats in are filled with people. Then, the calculation unit 207 in this embodiment sets the preset number as the number in the back seat area 412. Note that the preset number here is, for example, the number of people who can be seated in the back seat area 412, and is the number specified in advance by the user. Note that the preset number here is not limited to this, and may be a number obtained by multiplying the number of people who can be seated in the back seat area 412 by a predetermined multiplier.

FIG. 4D is a diagram for explaining the process of determining the position of a person by the position determination unit 206. The setting unit 204 in this embodiment sets a plurality of areas for the image, as shown in FIG. 4(d). In this embodiment, the setting unit 204 sets an area (setting area) for the image based on an operation by the user to specify an area on the image. Furthermore, the setting unit 204 sets the type for each of the plurality of setting areas based on the user's designation. In this embodiment, the types for the setting area are "aisle" which is a type corresponding to the aisle area 411, "seat (back)" which is a type corresponding to the back seat area 412, and a type corresponding to the front seat area 413. It is assumed that there is a type "seat (front)". At this time, for example, the operation reception unit 203 receives a user operation to specify the first setting area 441 with respect to the image 400 displayed on the display device 130, and further specifies "passage" as the type of the first setting area 441. Accepts the specified user operation. The operation reception unit 203 also accepts a user operation to specify the second setting area 442 with respect to the image 400 displayed on the display device 130, and further specifies “seat (back)” as the type of the second setting area 442. Accepts the specified user operation. The operation reception unit 203 also accepts a user operation to specify the third setting area 443 for the image 400 displayed on the display device 130, and further specifies "seat (front)" as the type of the third setting area 443. Accepts the specified user operation. The setting unit 204 sets a plurality of setting areas and a type corresponding to each of the plurality of setting areas based on the user operation received by the operation receiving unit 203.

A table 500 shown in FIG. 5 is a table showing information set by the setting unit 204 based on information received by the operation receiving unit 203. The area ID 501 is information that identifies the setting area. “1” in the area ID 501 corresponds to the first setting area 441 , “2” in the area ID 501 corresponds to the second setting area 442 , and “3” in the area ID 501 corresponds to the third setting area 443 . Furthermore, a position 502 indicates the position on the image of each of the plurality of setting areas. Note that the position of the setting area on the image is indicated by the X and Y coordinates of each vertex of the setting area, with the upper left end point of the image as the origin. Further, a type 503 indicates a type associated with each of the plurality of setting areas.

The position determining unit 206 in this embodiment specifies a setting area including the person detected by the detecting unit 205, and determines the position of the person based on the type corresponding to the specified setting area.

For example, the detection unit 205 detects a region of the upper body of a person (hereinafter referred to as a human region) from the image, and the position determination unit 206 specifies a set region including the center of gravity of the human region. At this time, for example, if the center of gravity position of the person area is included in the third setting area 443, the type of the third setting area 443 is "seat (front)", so the position determination unit 206 determines the position of the center of gravity of the person area. It is determined that the person is located in the front seat area 413. Further, if the center of gravity of the person area is included in the first setting area 441, the type of the first setting area 441 is "passage", so it is determined that the person in the person area is located in the passage area 411. . Furthermore, if the center of gravity position of the person area is included in the second setting area 442, the type of the second setting area 442 is "seat (back)", so the person in the person area is located in the back seat area 412. Then it is determined.

Note that multiple setting areas may overlap, such as the first setting area 441 and the second setting area 442 shown in FIG. May be included. In this case, the position determining unit 206 in this embodiment determines the area where the person in the detected person area is located based on the size of the detected person area. For example, the setting unit 204 presets a threshold value for the size of the human region for each of the two setting regions. Here, if the first setting area 441 and the second setting area 442 overlap as shown in FIG. ), set the range from 500 pixels to 1000 pixels. Similarly, the setting unit 204 sets the size range corresponding to the second setting area 442 from 100 pixels to 499 pixels. A table 600 shown in FIG. 6 is information set by the setting unit 204, and shows the size range for each of the first setting area 441 and the second setting area 442. Then, if the overlapping area between the first setting area 441 and the second setting area 442 includes the centroid position of the detected person area, the position determination unit 206 determines that the size of the person area in the vertical direction of the image is determined by the table. It is determined which of the size ranges shown in 600 it falls within. For example, if the size of the person area is between 100 pixels and 499 pixels, the position determining unit 206 specifies the second setting area 441, and the type corresponding to the second setting area 441 is "seat (back)". Therefore, it is determined that the person in the person area is located in the back seat area 412. Similarly, when the size of the person area is between 500 pixels and 1000 pixels, the position determination unit 206 specifies the first setting area 441, and since the type of the first setting area 441 is "passage", the size of the person area is It is determined that the person is located in the passage area. Note that, if the center of gravity of the person area is included in the overlapping area between the first setting area 441 and the second setting area 442, the position determination unit 206 determines that the size of the person area in the vertical direction of the image is as shown in the table 600. If it is not included in any of the size ranges, perform the following processing. That is, the position determination unit 206 excludes the person area from the detection results of the detection unit 205. As explained above, when a person area is detected in an area that overlaps with a plurality of setting areas, the position determination unit 206 in this embodiment determines the area where the person in the person area is located based on the size of the person area. Determine.

Next, information processing for determining the degree of congestion by the information processing apparatus 100 in this embodiment will be described in more detail with reference to the flow shown in FIG. 7 . By executing the process of the flow shown in FIG. 7, the number of people in the vehicle 20 can be calculated and the degree of congestion can be determined. It is assumed that the processing of the flow shown in FIG. 7 is executed by the functional blocks shown in FIG. 2, which are realized by the CPU 800 of the information processing device 100 executing a computer program stored in the ROM 820 of the information processing device 100. .

First, in S701, the communication unit 200 acquires an image captured by the imaging device 110 installed in the vehicle 20 as an image to be processed.

Next, in S702, the detection unit 205 detects a human region included in the processing target image acquired in S701. For example, it is assumed that the detection unit 205 in this embodiment detects a region of a person's upper body from an image as a person region using a matching pattern of the person's upper body.

Next, in S703, the position determining unit 206 determines as a target of interest one human region that has not yet been determined as a target of interest in the image, among the human regions detected from the image to be processed in S702. At this time, it is assumed that, for example, the position determination unit 206 determines, among the human regions on the image to be processed, the human region located at the upper part of the image as the object of interest.

Next, in S704, the position determining unit 206 determines the type of area where the person in the person area of interest is located. In the example shown in FIG. 4D, it is assumed that the center of gravity of the human region of interest is included in the third setting region 443. At this time, the position determining unit 206 specifies the third setting area 443 and determines that the person in the person area is located in the front seat area 413 based on the type of the third setting area 443, "seat (front)". do. Furthermore, in the example shown in FIG. 4D, a case is assumed in which the overlapping area between the first setting area 441 and the second setting area 442 includes the centroid position of the human area of interest. At this time, the position determination unit 206 refers to the table 600 and determines which of the size range corresponding to the first setting area 441 and the size range corresponding to the second setting area 442 includes the size of the person area of interest. Determine. Then, for example, when determining that the size of the person area is included in the size range corresponding to the first setting area 441, the position determining unit 206 determines the size of the person area based on the “passage” type of the first setting area 441. It is determined that the person in the person area is located in the passage area 411. Similarly, if the position determination unit 206 determines that the size of the person area is included in the size range corresponding to the second setting area 442, the position determination unit 206 determines that the type of the second setting area 442 is "seat (back)". Based on this, it is determined that the person in the person area is located in the back seat area 412. Note that if the size of the person area of interest is not included in either the size range corresponding to the first setting area 441 or the size range corresponding to the second setting area 442, the person area is determined by the detection result of the detection unit 205. Exclude from

Next, in S705, the position determining unit 206 determines whether all human areas in the image currently being processed have been determined as the target of interest. If it is determined that all human areas on the image to be processed have been determined as objects of interest (Yes in S705), the process moves to S706. On the other hand, if it is determined that all the human areas on the image to be processed have not been determined as the target of interest (No in S705), the process moves to S703, and the position determining unit 206 selects the next human area of the target of interest. Determine.

Next, in S706, the calculation unit 207 calculates the number of people for each area within the vehicle 20. That is, the calculation unit 207 calculates the number of people located in each of the aisle area 411, the back seat area 412, and the front seat area 413 based on the determination result by the position determination unit 206. For example, in the example shown in FIG. 4B, the calculation unit 207 calculates that the number of people located in the aisle area 411 is 0, the number of people located in the back seat area 412 is 3, and the number of people located in the front seat area 413 is 2. Calculate as a person. On the other hand, in the example shown in FIG. 4C, the calculation unit 207 calculates that the number of people located in the aisle area 411 is 5, the number of people located in the back seat area 412 is 0, and the number of people located in the front seat area 413 is 4. It is calculated as a person. In this way, in the example shown in FIG. 4(c), since a plurality of people are located in the aisle area 411, the person seated in the seat in the back seat area 412 is difficult to see in the image, so The person located is not detected. As a result, although the number of people located in the back seat area 412 is four, if only the detection result by the detection unit 205 is used, the number of people located in the back seat area is calculated as zero.

Next, in S707, the calculation unit 207 compares the number of people in the passage area 411 with the first threshold value. If the calculation unit 207 determines that the number of people in the passage area 411 is less than the first threshold (No in S707), the process moves to S710. On the other hand, if the calculation unit 207 determines that the number of people in the passage area 411 is equal to or greater than the first threshold (Yes in S707), the process moves to S708.

Next, in S708, the calculation unit 207 compares the number of people in the front seat area 413 with a second threshold value. If the calculation unit 207 determines that the number of people in the front seat area 413 is less than the second threshold (No in S708), the process moves to S710. On the other hand, if the calculation unit 207 determines that the number of people in the front seat area 413 is equal to or greater than the second threshold (Yes in S708), the process moves to S709.

When the process moves to S709, in S709, the calculation unit 207 presets the number of people located in the back seat area 412, which is an area located further from the imaging device 110 among the areas adjacent to the aisle area 411. Correct to a predetermined number.

Next, in S710, the calculation unit 207 calculates the total number of people in the vehicle 20 (total number of people). In the example shown in FIG. 4, the calculation unit 207 calculates the total number of people located in each of the aisle area 411, the back seat area 412, and the front seat area 413. Note that if the answer is Yes in S708, the calculation unit 207 in S710 calculates the number of people in each of the aisle area 411 and the front seat area 413 calculated in S706, and the number of people in the back seat area 412 corrected in S709. Calculate the total number of people. On the other hand, if the answer is No in S707 or No in S708, the calculation unit 207 in S710 calculates the sum of the number of people for each area calculated in S706 as the total number of people.

Next, in S711, the congestion determination unit 208 determines the degree of congestion in the vehicle 20 based on the number of people for each area calculated in S706. At this time, the congestion determination unit 208 compares, for example, the total number of people in the aisle area 411, the back seat area 412, and the front seat area 413 with a threshold value. Then, the congestion determination unit 208 determines that the area is crowded when the number of people is equal to or greater than the threshold, and determines that the area is not crowded when the number of people is less than the threshold.

Next, in S712, the output control unit 202 outputs information indicating the determination result of the degree of congestion determined by the congestion determination unit 208 in S711 to an external device (such as the display device 130). For example, the output control unit 202 causes the display device 130 to display information indicating the determination result by the congestion determination unit 208.

Next, in S713, if there is an instruction to end the process shown in FIG. 7 (Yes in S713), the process in the flow shown in FIG. Then, in step S701, the communication unit 200 acquires the next image to be processed.

As described above, in the information processing apparatus 100 according to the present embodiment, the number of people located in the aisle area 411 is greater than or equal to the first threshold, and the number of people located in the front seat area 413 is greater than or equal to the second threshold. In this case, the number located in the back seat area 412 is a predetermined number set in advance. Generally, in a closed space such as a vehicle where seats and aisles coexist, if many people are standing in the aisle, it is likely that other people are already seated in the seats. In particular, if there are many people standing in the aisle and there are as many people as possible sitting in the seats that are not in the blind spot, may already be occupied by another person. Therefore, if the number of people located in the aisle area 411 is at least the first threshold and the number of people located in the front seat area 413 is at least the second threshold, the seats in the back seat area 412 are filled with people. The number of seats in the back seat area 412 is set in advance. By doing this, by increasing the number of people located in the aisle area 411, the number of people located in the back seat area 412 can be appropriately reduced even if the person located in the back seat area 412 is not visible on the image. This can be reflected in the counting results. Therefore, even if a person located in a seat is not visible on the image due to multiple people located in an area that does not have a seat, it is possible to suppress a decrease in the accuracy of estimating the degree of crowding.

In the present embodiment, the image captured by the imaging device 110 installed in the vehicle 20 includes three areas: the aisle area 412, the back seat area 412, and the front seat area 413, but the present invention is not limited to this. For example, there is a passageway area 411 where a person passes, and a back seat area which is an area corresponding to a seat where a person can sit, which is adjacent to the passageway area 411 and located further from the imaging device 110. Assume that the interior of the vehicle 20 is configured as follows. In other words, assume that the front seat area 413 does not exist in the example shown in FIG. At this time, if the number of people located in the aisle area 411 is greater than or equal to the first threshold, the calculation unit 207 sets the number of people located in the back seat area to a preset number. On the other hand, when the number of people located in the aisle area 411 is less than the first threshold, the calculation unit 207 calculates the number of people located in the back seat area from the number of people determined to be located in the back seat area by the position determination unit 206. be the number of By doing this, by increasing the number of people located in the aisle area 411, the number of people located in the back seat area 412 can be appropriately reduced even if the person located in the back seat area 412 is not visible on the image. This can be reflected in the counting results. Therefore, even if a person located in a seat is not visible on the image due to a plurality of people located in an area that does not have a seat, it is possible to suppress a decrease in the accuracy of estimating the degree of crowding.

(Other embodiments)
Next, with reference to FIG. 8, the hardware configuration of the information processing apparatus 100 for realizing each function of each embodiment will be described. Note that although the hardware configuration of the information processing device 100 will be described in the following description, it is assumed that the recording device 120 and the imaging device 110 are also realized by the same hardware configuration.

The information processing device 100 in this embodiment includes a CPU 800, a RAM 810, a ROM 820, an HDD 830, and an I/F 840.

The CPU 800 is a central processing unit that centrally controls the information processing device 100. RAM 810 temporarily stores computer programs executed by CPU 800. Further, the RAM 810 provides a work area used when the CPU 800 executes processing. Further, the RAM 810 functions, for example, as a frame memory or as a buffer memory.

The ROM 820 stores programs for the CPU 800 to control the information processing device 100 and the like. The HDD 830 is a storage device that records image data and the like.

The I/F 840 communicates with external devices via the network 140 according to TCP/IP, HTTP, or the like.

In addition, in the description of each embodiment mentioned above, an example will be described in which the CPU 800 executes the processing, but at least a part of the processing of the CPU 800 may be performed by dedicated hardware. For example, the process of displaying a GUI (GRAPHICAL USER INTERFACE) or image data on the display device 130 may be executed by a GPU (GRAPHICS PROCESSING UNIT). Further, the process of reading the program code from the ROM 820 and expanding it to the RAM 801 may be executed by a DMA (DIRECT MEMORY ACCESS) functioning as a transfer device.

Note that the present invention can also be implemented by a process in which one or more processors read and execute a program that implements one or more of the functions of the embodiments described above. The program may be supplied to a system or device having a processor via a network or a storage medium. The present invention can also be implemented by a circuit (eg, an ASIC) that implements one or more of the functions of the embodiments described above. Further, each part of the information processing apparatus 100 may be realized by the hardware shown in FIG. 8, or may be realized by software.

Note that another device may have one or more functions of the information processing device 100 according to each embodiment described above. For example, the imaging device 110 may have one or more functions of the information processing device 100 according to each embodiment. Note that the above-described embodiments may be combined, for example, any combination of the above-described embodiments may be implemented.

Although the present invention has been described above along with the embodiments, the above embodiments are merely examples of implementation of the present invention, and the technical scope of the present invention is interpreted to be limited by these embodiments. It's not a thing. That is, the present invention can be implemented in various forms without departing from its technical idea or main features. For example, a combination of each embodiment is also included in the disclosure content of this specification.

100 Information processing device 110 Imaging device 200 Communication unit 201 Storage unit 202 Output control unit 203 Operation reception unit 204 Setting unit 205 Detection unit 206 Position determination unit 207 Calculation unit 208 Congestion determination unit

Claims (19)

detection means for detecting a person from an image captured by the imaging means in a predetermined area;
a first calculation means for calculating the number of people located in a first area without seats in the predetermined area based on the result of the detection means;
a second calculation means for calculating the number of people located in a second area, which is an area different from the first area and has seats in the predetermined area;
The second calculation means is characterized in that when the number of people calculated by the first calculation means is equal to or greater than a first threshold, the second calculation means sets the number of people located in the second area to a predetermined number set in advance. Information processing equipment. The second calculation means calculates the number of people located in the second area based on the result of the detection means when the number of people calculated by the first calculation means is less than the first threshold value. The information processing device according to claim 1, characterized in that: 3. The apparatus according to claim 1, further comprising congestion determining means for determining the degree of crowding of people in the predetermined area based on the number of people calculated by the first calculating means and the second calculating means. information processing equipment. It is characterized by further comprising a setting means for setting a first setting area that is an area on the image corresponding to the first area, and a second setting area that is an area on the image corresponding to the second area. The information processing device according to any one of claims 1 to 3. If the person detected by the detection means is included in the first setting area on the image, it is determined that the person is located in the first area, and the person is included in the second setting area on the image. 5. The information processing apparatus according to claim 4, further comprising position determination means for determining that the person is located in the second area if the person is located in the second area. The information processing device according to any one of claims 1 to 5, wherein the second area is an area located on the back side compared to the first area with the position of the imaging means as a starting point. . Third calculation means for calculating the number of people located in a third area, which is an area located on the near side compared to the first area, with the position of the imaging means as a starting point, and is an area having seats in the predetermined area. The information processing apparatus according to claim 6, further comprising: an information processing apparatus according to claim 6; When the second calculation means calculates that the number of people calculated by the first calculation means is equal to or more than the first threshold, and the number of people calculated by the third calculation means is equal to or more than the second threshold, 8. The information processing apparatus according to claim 7, wherein the number of people located in the second area is a predetermined number. 9. The information processing apparatus according to claim 1, wherein the predetermined number corresponds to the number of people who can sit on the seats in the second area. a detection step of detecting a person from an image captured by an imaging means in a predetermined area;
a first calculation step of calculating the number of people located in a first region that does not have a seat in the predetermined region based on the result of the detection step;
a second calculation step of calculating the number of people located in a second region that is a region different from the first region and that has seats in the predetermined region;
In the second calculation step, if the number of people calculated in the first calculation step is greater than or equal to a first threshold, the number of people located in the second area is set to a predetermined number. information processing method. In the second calculation step, if the number of people calculated in the first calculation step is less than the first threshold, calculate the number of people located in the second area based on the result in the detection step. 11. The information processing method according to claim 10. 12. The method according to claim 10, further comprising a crowding determination step of determining the degree of crowding of people in the predetermined area based on the number of people calculated in the first calculation step and the second calculation step. information processing methods. The method further comprises a setting step of setting a first setting area that is an area on the image corresponding to the first area and a second setting area that is an area on the image corresponding to the second area. The information processing method according to any one of claims 10 to 12. If the person detected in the detection step is included in the first setting area on the image, it is determined that the person is located in the first area, and the person is included in the second setting area on the image. 14. The information processing method according to claim 13, further comprising a position determining step of determining that the person is located in the second area if the person is located in the second area. The information processing method according to any one of claims 10 to 14, wherein the second area is an area located further back than the first area with the position of the imaging means as a starting point. . a third calculation step of calculating the number of people located in a third area, which is an area located on the near side compared to the first area with the position of the imaging means as a starting point, and is an area having seats in the predetermined area; 16. The information processing method according to claim 15, further comprising: In the second calculation step, when the number of people calculated in the first calculation step is greater than or equal to the first threshold, and the number of persons calculated in the third calculation step is greater than or equal to the second threshold; 17. The information processing method according to claim 16, wherein the number of people located in the second area is a predetermined number. 18. The information processing method according to claim 10, wherein the predetermined number corresponds to the number of people who can sit on the seats in the second area. A program for causing a computer to function as each means of the information processing apparatus according to claim 1.

Images