JP2023079291A - Image processing device, image processing method, and program - Google Patents

Abstract

To estimate the degree of possible danger to a person.SOLUTION: An image processing device provided herein identifies a behavior pattern of a person in a captured image, acquires feature information indicative of features of a shooting location, determines the degree of possible danger to the person using the identified behavior pattern and the acquired feature information, and outputs information on the determined degree of danger.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to technology for estimating the degree of danger that may occur due to human behavior in a facility or the like.

In parks and other facilities, people become absorbed in their own activities, such as playing and exercising, neglecting observation of their surroundings, or are only able to move slowly to avoid dangers caused by their own actions. failure to do so may result in serious accidents. For example, when a child falls while playing on a playground equipment in a park, he or she may not be able to avoid the danger, or may be injured by coming into contact with the playground equipment even though he/she performed the action to avoid the danger in time.

Various techniques have been developed to issue a warning before such danger occurs. Patent Document 1 specifies the age of a child who uses the playground equipment using an image of a place where the playground equipment for which the target age is set is installed, determines whether the specified age is included in the target age, and specifies Disclosed is a technique for issuing a warning when the specified age is not included in the target age.

Japanese Patent Application Laid-Open No. 2020-27320

However, in the technique of Patent Literature 1, the determination as to whether to issue a warning is based on the target age of the playground equipment, and it is not determined whether or not the child's behavior poses a danger. Therefore, even if a child is in danger when playing without using playground equipment such as tag or playing with playground equipment of a target age, the child cannot recognize the danger.

An object of the present disclosure is to provide a technique for estimating the degree of danger that may occur to humans.

An image processing apparatus according to the present disclosure that solves the above problems includes identifying means for identifying a behavior pattern of a person reflected in a captured image, first obtaining means for obtaining first characteristic information indicating characteristics of an imaging location, determining means for determining the degree of possible danger to the person using the identified behavior pattern and the acquired first characteristic information; and output means for outputting information about the determined degree of danger. , is characterized by having

According to the present disclosure, the degree of possible danger to humans can be estimated.

It is a figure which shows the usage example of a sensor system. It is a figure which shows the hardware configuration example of a sensor system and a terminal. It is a block diagram which shows the software structural example of a sensor system. It is a figure which shows the data table example of an action pattern. It is a figure which shows the data table example of the characteristic of a park. It is a figure which shows the data table example for risk calculation. It is a figure which shows the data table example for risk determination. 4 is a flowchart showing the flow of risk prediction processing; 7 is a flowchart showing a detailed flow of risk calculation processing; It is a flow chart which shows a flow of learning processing of action pattern DB. 7 is a flowchart showing the flow of learning processing for a risk calculation table; It is a figure which shows the example of a notification method. It is a figure which shows the usage example of a sensor system. It is a block diagram which shows the software structural example of a sensor system. FIG. 10 is a diagram showing an example of a data table of characteristics of moving objects; 4 is a flowchart showing the flow of risk prediction processing; 4 is a flow chart showing the flow of risk calculation processing. It is a figure which shows the other example of the notification method.

Hereinafter, embodiments of the technology of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that the following embodiments do not limit the technology of the present disclosure according to the scope of claims, and all combinations of features described in the embodiments are essential to the solution of the technology of the present disclosure. Not necessarily. In addition, the same reference numbers are given to the same components, and the description thereof is omitted.

<Embodiment 1>
In this embodiment, the behavior pattern of the child reflected in the captured image and the characteristic information of the imaging location are used to estimate the degree of risk (degree of risk) that indicates the likelihood of the child taking risky behavior. will be described with reference to the drawings. FIG. 1 is a diagram showing a usage example of the sensor system according to this embodiment. FIG. 1(a) shows an outline of a usage example of the sensor system, and FIG. 1(b) shows a case where grid lines are given to the imaging area. Since the sensor system processes images, it can be said that the sensor system is an image processing device.

The sensor system 101 is installed at a location where the entire park can be included in the field of view. The installation location of the sensor system 101 may be, for example, inside the park or outside the park. The sensor system 101 is, for example, a device capable of acquiring temporally continuous images (still images or moving images) obtained by imaging and audio. It has the function of recognizing the behavioral pattern of the person who is the target of the The sensor system 101 also has a function of recognizing the characteristics of the park where the target person is. Furthermore, the sensor system 101 determines the degree of danger (degree of danger) that may occur to the child based on the information obtained by recognition, and sends information about the determination result to the guardian or leader who is related to the target child. It also has a function to notify the terminal such as

Assume that a plurality (specifically, six) of children 131 to 136 are playing in the park 110 . The sensor system 101 analyzes the captured images to recognize each of the children 131 to 136, and determines the behavior patterns of the target child, such as which playground equipment each child is currently playing with, and which direction each child is moving. Get information about Information about the direction of movement of the child is obtained using known techniques such as optical flow. Information about the child's direction of movement is obtained from the frame-to-frame variation, focusing on specific parts, such as the position of the face or the position of the torso. Note that images under the same imaging conditions are used when optical flow is used.

It is also assumed that the park 110 has facilities related to the characteristics of the park 110 . The facilities include, for example, a fence 111 that divides the area of the park 110 and its surroundings, a horizontal bar 112, a sandbox 113, a jungle gym 114, and an entrance 115 of the park 110. FIG. The sensor system 101 analyzes the captured image, recognizes the facilities of the park 110 such as the fence 111, the horizontal bar 112, the sandbox 113, the jungle gym 114, and the entrance/exit 115, and includes information about the facilities of the park based on the recognition results. Get park property information. The sensor system 101 analyzes the captured image and obtains characteristic information of the park including information such as the shape and height of the fence 111, for example.

Suppose a child 135 is playing with a ball 121 . Information about the ball 121 is used as part of the behavioral patterns of the child 135 to obtain information about what the child is playing with and whether the child can see the surroundings. For information on whether the surroundings can be seen, for example, a condition is determined as to whether the surroundings cannot be seen. This information is obtained by For example, it may be determined that "the surroundings cannot be seen" when the target person's face does not face the same direction for a certain period of time based on the passage of time of the target person's face image. Further, when the horizontal orientation of the subject's body and the horizontal orientation of the subject's head continue for a certain period of time, it may be determined that "the surroundings cannot be seen." The horizontal orientation of the subject's head and the horizontal orientation of the subject's body are determined by known techniques. As a known technique, for example, an open pose technique may be used, or a three-dimensional model may be prepared in advance and obtained by fitting.

FIG. 1(b) is a diagram showing an example when an area is recognized with respect to FIG. 1(a). Area information indicating areas A1, A2, A3, B1, B2, B3, C1, C2, and C3 is included in the behavior pattern information of children and the park characteristic information. The areas are recognized by analyzing captured images obtained by the sensor system 101 . Although this embodiment shows an example in which the entire park is divided into nine areas, the present invention is not limited to this. The entire park may be divided into smaller pieces, such as 16 areas. Also, specific areas such as the horizontal bar 112, the sandbox 113, and the jungle gym 114 may be divided more finely than other areas. By dividing the park more finely, it is possible to obtain more accurate information about the behavior patterns of children and the characteristics of the park, so that the accuracy of estimating the degree of risk can be increased.

The sensor system 101 determines the possibility of the child taking dangerous behavior as a risk level from the acquired behavior pattern information of the child and the property information of the park. etc.

For example, suppose child 132 is facing fence 111 and has an active personality. An active child is, for example, a child with a high activity level who moves around or climbs high places. Based on the child's character and movement, it is determined that the child 132 is likely to take a dangerous action of "climbing over the fence". Therefore, the information regarding the degree of danger obtained based on the result of this determination is notified to the leader of the child 132 . In addition, since the child 135 is vigorously chasing the ball 121 toward the entrance 115, it is determined that the possibility of the child 135 taking a dangerous action of "jumping out of the entrance" is high. Therefore, the information about the degree of risk obtained based on the result of this determination is notified to the child 135's guardian or leader.

(Sensor system and terminal hardware)
FIG. 2 is a diagram illustrating a hardware configuration example of a sensor system and a terminal; The sensor system 101 has a CPU 201 , a ROM 202 , a RAM 203 , a storage device 204 , an input section 206 , an output section 207 and a communication section 208 . These components are coupled to transmit and receive data to each other via bus 205 .

The CPU 201 is a microprocessor, and controls access to various devices connected to the bus 205 based on control programs and the like stored in the ROM 202 . The CPU 201 performs various controls such as acquisition of behavior patterns of children and park characteristics, estimation of risk, notification after estimating risk, and controls each device in the sensor system 101 .

A ROM (Read Only Memory) 202 is a read-only non-volatile memory and stores a boot program and the like. The CPU 201 executes a boot program when the system is started, loads an OS (Operating System) and various control programs from a storage device 204, which is a boot device, into a RAM 203 and executes them. The control program includes, for example, a program that implements the processes shown in FIGS. 8-11.

A RAM 203 is a readable and writable random access memory. The RAM 203 is used for temporary storage of various data used by each device. Various work areas for storing data necessary for processing and various data whose values are changed during processing are stored. Various data used in the processing flow described later are loaded into the RAM 203 .

The storage device 204 is storage including removable media such as a non-volatile memory or an SD card, and stores a large amount of data in a changeable manner. The storage device 204 stores a control program for estimating the degree of danger and various management data used by this program, and updates the contents as needed.

A bus 205 transfers an address signal and a control signal that indicate each device to be controlled by the CPU 201 . A bus 205 also transfers data between devices.

An input unit 206 includes a touch panel, buttons, an imaging device, a microphone, and the like, and the user gives various instructions to estimate the degree of risk. Note that an imaging device is a device capable of obtaining a moving image or a still image by imaging. The imaging device may be of a type in which the imaging direction is fixed, or may be of a type having a function of moving the imaging direction in pan and tilt directions or a zoom function.

The output unit 207 is a liquid crystal monitor, a speaker, or the like, and performs notification by screen display or voice when it is determined that the risk is high. Accordingly, when it is determined that the degree of danger is high, it is possible to alert the leader or the like. The output unit 207 includes a device having an input function such as a touch panel. The communication unit 208 connects to the terminal 210 via a wireless communication path and exchanges data.

A terminal 210 is a device to which information obtained by the sensor system 101 is notified. The terminal 210 has a CPU 211 , a communication section 212 , an output section 214 and a storage device 215 . These components are coupled to transmit and receive data to each other via bus 213 .

The CPU 211 is a microprocessor, and controls display, notification, etc. based on connection with the sensor system 101 and data received from the sensor system 101 .

The communication unit 212 connects to the sensor system 101 via a wireless communication path and exchanges data.

A bus 213 transfers an address signal and a control signal that indicate each device to be controlled by the CPU 211 . A bus 213 also transfers data between devices.

The output unit 214 is a liquid crystal monitor, a speaker, or the like, and performs notification by screen display or voice when it is determined that the risk is high. Accordingly, when it is determined that the degree of danger is high, it is possible to alert the leader or the like. The output unit 214 also includes a device with an input function such as a touch panel.

The storage device 215 is storage including removable media such as a non-volatile memory or an SD card, and stores a large amount of data in a changeable manner. The storage device 215 stores a program for controlling communication with the sensor system 101 and various management data used by this program, and updates the contents as needed.

(Functional configuration of sensor system)
FIG. 3 is a block diagram showing a functional configuration example of the sensor system according to this embodiment. The sensor system 101 has an image acquisition unit 301 , a park characteristic acquisition unit 302 , an action pattern acquisition unit 303 , an action pattern DB 304 and a learning unit 305 . The sensor system 101 further includes a park characteristic (park characteristic data table) 306 , an action pattern table 307 , a risk determination section 308 , a risk calculation table 309 , a risk determination table 310 and a notification section 311 . Hereinafter, the database will be referred to as DB.

The image acquisition unit 301 acquires an image (moving or still image) of an imaging location by imaging, and transmits the acquired image to the park characteristic acquisition unit 302 , behavior pattern acquisition unit 303 , and learning unit 305 .

The park property acquisition unit 302 recognizes park properties from the image received from the image acquisition unit 301, and creates a park property data table (park property information) 306 based on the recognition result. The park characteristic acquisition unit 302 creates a park characteristic (park characteristic data table) 306 based on map information or pre-input information. An example of the park characteristic data table will be described later with reference to the drawings.

The behavior pattern acquisition unit 303 recognizes a person to be monitored, such as a child, from the image received from the image acquisition unit 301, and acquires a behavior pattern table 307 of the person to be monitored (child), including the data in the behavior pattern DB 304. create. An example of the behavior pattern table of the person to be monitored (child) will be described later with reference to the drawings.

The learning unit 305 recognizes the child from the image received from the image acquisition unit 301, and updates the behavior pattern DB 304 of the person to be monitored (child). Also, the learning unit 305 updates the risk calculation table 309 based on the history of past risky actions. The risk calculation table 309 is a table in which the degree of risk is specified in association with the type of action pattern, and is stored in advance for each type of characteristic. An example of the degree-of-risk calculation table will be described later with reference to the drawings.

The risk determining unit 308 calculates the risk of dangerous behavior for each child based on the park characteristic data table (characteristics) 306 , the behavior pattern table 307 of the person being monitored (child), and the risk calculation table 309 . That is, among the stored risk determination tables, the risk determination table corresponding to the characteristics of the park is determined, and the action pattern of the target person identified from the determined risk determination table is associated with the target person's behavior pattern. Derive the degree of danger. It can be said that the degree of danger is determined. Then, the risk determination unit 308 creates a risk determination table 310 based on the calculation results. An example of the risk determination table will be described later with reference to the drawings.

Based on the risk determination table 310, when the notification unit 311 obtains a determination result that the child has taken a risky action whose risk exceeds a certain level, the notification unit 311 sends a message to a terminal such as a leader of the target child. On the other hand, it notifies information that there is a high possibility that the target person's child will take dangerous behavior. The information to be notified is not limited to this, and may be tabular information showing a list of information indicating the likelihood of the target child taking risky behavior for each risky behavior.

(children's behavior patterns)
A child's behavior pattern will be described with reference to the drawings. FIG. 4 is a diagram showing an example data table of behavior patterns of children according to the present embodiment. The behavior pattern data of the child is classified into two patterns, one in which information on the target child is registered in advance and one in which the information on the target child is not registered in advance. FIG. 4(a) shows a case where information about a target child is registered in advance, and FIG. 4(b) shows a case where information about a target child is not registered in advance.

First, a data table 400 of behavior patterns of children in which information about target children is registered in advance will be described. The data table 400 includes the items of name 401, face image 402, whole body image 403, height 404, position 405, moving direction 406, moving speed 407, current play 408, and surroundings visible 409. has information on The data table 400 also has information on each item of personality (activity level) 410 and athletic ability (physical ability) 411 regarding the target child. Information on the child's name is registered in the item of name 401 .

A child's face image is registered in the item of face image 402 . A child's whole body image is registered in the item of the whole body image 403 . Recognition of a registered child from an image captured by the sensor system 101 is performed based on the face image of the child registered in the face image 402 item. Recognition of the registered child is not limited to this, and may be performed based on the whole body image registered in the item of whole body image 403 instead of the face image. Alternatively, the face image registered in the item of face image 402 and the whole body image registered in the item of whole body image 403 may be combined. By combining the face image and the whole body image, the number of target parts increases compared to when only the face image or only the whole body image is used, so the recognition accuracy of the registered child can be improved.

Information about the height of the child is registered in the item of height 404 . For the height 404 item, the user may manually input information regarding the child's height in advance. Alternatively, the height of the subject's child may be estimated from the whole-body image obtained by imaging with the imaging apparatus of this embodiment, and the height 404 item may be automatically input based on the estimation result.

The item of position 405 registers information about the position (place) where the child is in the park. When the park is divided into a plurality of areas such as areas A1 to C3 in FIG. 1(b), which area the target child is in is shown. Here, it is expressed as an area, but it may be expressed as a coordinate position (x, y, z) instead of an area. Information about the position is obtained from images captured by the sensor system 101 .

Information about the child's moving direction is registered in the item of moving direction 406 . From the image captured by the sensor system 101, information indicating from which area the child is heading to which area among the areas A1 to C3 is acquired and registered. If the moving speed 407, which will be described later, is "stationary", the child is not moving, so the moving direction data is not acquired. In the example of FIG. 4, "-" indicates not yet acquired.

Information about the child's moving speed is registered in the item of moving speed 407 . Based on the image captured by the sensor system 101, the moving distance of the child per unit time is estimated, the moving speed is derived based on the estimated result, and information about the moving speed is registered. Based on the acquired moving speed and a predetermined threshold value, it is classified into one of "fast", "slow", and "stationary" and registered. In this embodiment, the moving speed is expressed in three stages of "fast", "slow", and "stationary". You may As for the resolution of the moving speed, the resolution of the speed that can be measured from the number of pixels of the imaging device and the imaging interval (frames/s) may be the maximum resolution. Also, the information about the moving speed may be expressed using a specific numerical value. For example, a target child is extracted using a plurality of frames of captured images acquired by the input unit, and information about the movement speed is set based on the difference in the coordinates of the same part of the extracted child and the threshold value. become.

The current play 408 field contains information about the child's current play. By recognizing the behavior of the child from the image captured by the sensor system 101, information about the child's current play is acquired and registered. Current play may be determined, for example, based on the play equipment in the area where the child is. In addition, the current play recognizes the state where the child is putting his hand on the horizontal bar, the state where the child is sitting in the sandbox, and the state where the child is putting his hand on the jungle gym. may be determined.

Information indicating whether or not the child can see the surroundings (accurately recognizes the surroundings and acts accordingly) is registered in the item 409 that the surroundings can be seen. For example, from the image captured by the sensor system 101, the child's behavior is recognized, and a state such as "chasing the ball" and "head only facing the ball for a certain period of time" is detected, When this state is detected, it is determined that "the surroundings cannot be seen". When this state is not detected, it is determined that "the surroundings are visible". For example, if the face image of a target child is extracted using multiple frames of captured images acquired by the input unit, and the condition is that the face direction has been facing the same direction for a certain period of time, It is determined that it is in the state of If the above conditions are not met, it is determined that the user is in a state of "seeing the surroundings". Alternatively, when the condition that the child's horizontal orientation of the body and the horizontal orientation of the child's head continues for a certain period of time, it is determined that the child is in a state of "the surroundings cannot be seen." If the above conditions are not met, it is determined that the user is in a state of "seeing the surroundings". Then, based on the determination result, one of a plurality of preset items such as "visible" and "not visible" is set.

Information on the personality (activity level) of the child is registered in the item of personality 410 . Data pre-registered in the behavior pattern DB is stored in the item of character 410, and in this embodiment, either "gentle" or "active" is stored. "Quiet" is registered for children who are calm and not active in nature and have relatively low activity levels. “Active” refers to a child who is active and is registered as a child whose activity level is relatively high. In the present embodiment, an example of classifying into two types of "gentle" and "active" is shown, but personality may be classified and registered in more detail. Personality information may be determined without using a learning model, may be determined using a learning model, or may be entered directly by a user. When the learning model is not used, the personality is determined based on the behavior history of the child appearing in the image data within a predetermined period. For example, if the amount of movement for a certain period of time exceeds a certain amount, it may be determined to be "active", and if it does not exceed a certain amount, it may be determined to be "gentle". In addition, it recognizes the child's behavior and judges it as "active" if the time spent talking to another child (in a relatively close position and facing each other) exceeds a certain amount of time, and as "gentle" if it does not exceed a certain amount of time. You may When a learning model is used, for example, the "personality" may be determined using a model learned by using "child's behavior" as an input and "personality" at that time as correct data. The behavior pattern DB 304 is updated by learning by the learning unit 305 . The learning operation of the action pattern DB 304 will be described with reference to FIG. 10 .

Information about the child's athletic ability is registered in the item about athletic ability 411 . Data pre-registered in the action pattern DB is stored in the item of athletic ability 411, and in the present embodiment, for example, one of five levels from 1 to 5 is stored. The numerical value "1" indicates the lowest athletic ability, the higher the numerical value, the higher the athletic ability, and the numerical value "5" indicates the highest athletic ability. Information about athletic performance may be determined without using a learning model, may be determined using a learning model, or may be entered directly by a user. When the learning model is not used, the athletic ability is determined based on the child's action history and the like appearing in the image data within a predetermined period. For example, by recognizing behavior from the amount of movement, recognizing states such as "I can do a reverse climb on a horizontal bar" or "I can climb to the top on a jungle gym", I judge (evaluate) that my athletic ability is high. may Alternatively, it may be determined from the height of the child, the maximum moving speed within the period, or the like. If the height is high and the maximum movement speed is high, the kinetic lift may be set to a higher value. When a learning model is used, for example, the "athletic ability" may be determined using a model learned by using "child's behavior" as an input and "athletic ability" at that time as correct data.

Next, the data table 420 of the child's behavior pattern when the information about the child of the target person is not registered in advance will be described. The data table 420 includes ID 421, face image 422, whole body image 423, height 424, position 425, movement direction 426, movement speed 427, current play 428, and surroundings visible 429 for children in the target park. It has information for each item. The data table 420 also has information on items of personality 430 and athletic ability 431 regarding children in the target park.

In the item of ID 421, information on an ID unique to each child for identifying unregistered children is registered. An unregistered child's face image is registered in the face image 422 item. If there is a face image that is not registered in the behavior pattern and is not registered in association with the ID 421 among the recognized child face images from the image obtained by the sensor system 101, an ID is issued and associated with the ID. Then, the target face image is registered in the data table 420 . The full-body image 423 field registers a full-body image of an unregistered child. When there is data in the item of ID 421 and the item of face image 422 and there is no data in the item of full-body image 423, the face image registered in the item of face image 422 is selected from the image obtained by imaging by the sensor system 101. A full-body image of the child is recognized and registered in the data table 420 . Recognition of a child from an image captured by the sensor system 101 is performed based on the face image of the child registered in the face image 422 item. Recognition of the child is not limited to this. may be performed in combination with whole-body images registered in . By combining the face image and the whole body image, the number of target parts increases compared to the case where only the face image or only the whole body image is used, so it is possible to improve the recognition accuracy of the child.

In the item of height 424, information regarding the height of a child whose ID is issued and registered is registered. The height of the child may be estimated from the whole-body image of the child captured by the imaging device of this embodiment, and the height 424 may be automatically input based on the estimation result.

The position 425, moving direction 426, moving speed 427, and current play 428 items contain the same information as the information registered in the above-described position 405, moving direction 406, moving speed 407, and current play 408 items. is stored. In addition, the same information as the information registered in the above-described item 409 of which the surroundings are visible is stored in the item of 429 which is the surroundings can be seen.

Information about the character of an unregistered child is registered in the item of character 430 . When the ID 421 is assigned (issued) to the item of personality 430, the learning operation of the action pattern DB shown in FIG. 10 starts, and when the learning operation is completed, data is input based on the learning result. Until the learning operation is completed, the state is "learning" and no data is input. Personality data handled in this embodiment is registered with either "gentle" (low activity level) or "active" (high activity level), like the personality 410 . Further, similar to the information entered in the item of personality 410, the personality may be classified and registered in more detail.

Information about the athletic ability (physical ability) of the unregistered child is registered in the item about the athletic ability 431 . Similar to the item of personality 430, the item of athletic ability 431 starts learning operation of the action pattern DB shown in FIG. 10 when the ID 421 is assigned (issued). Data is entered based on Until the learning operation is completed, the state is "learning" and no data is input. As for the athletic performance data handled in this embodiment, any value among five levels from 1 to 5, for example, is registered, like the athletic performance 411 .

(Characteristics of the park)
Next, the characteristics of parks will be explained with reference to figures. FIG. 5 is a diagram showing an example of a data table of park characteristics according to the present embodiment. It is assumed that the data table shown in FIG. 5 is created for the park shown in FIG. The park characteristics data table 500 has information on each item of position 501 , attribute 502 , attribute characteristics 503 , and information 504 . The information of each item may be information directly input by the user.

In the item of position 501, for example, when the park is divided into areas A1, A2, A3, B1, B2, B3, C1, C2, and C3 shown in FIG. Information indicating whether there is a characteristic (attribute) of is registered. The information registered in the item of the position 501 is obtained from the captured image and the captured information obtained by capturing the image of the sensor system 101 . Moreover, you may acquire from map information. The map information may be acquired in advance from the outside and registered in the database within the system, or may be acquired via the Internet.

The item of the attribute 502 is information indicating the attribute of the characteristics of the park, which is information about the elements of the park necessary when estimating the risky behavior of the child. Information acquired using object recognition is registered. Information about park elements includes, for example, fence 1, fence 2, horizontal bar, sandpit, jungle gym, doorways, and outside roads adjacent to the park.

In the item of attribute characteristics 503, information on attribute characteristics is registered. Information about attribute features may be obtained from map information. When the attribute is a road outside, the traffic volume may be determined from the type of the road, or may be determined from the number of vehicles captured in images captured for a certain period of time.

In the item of information 504, information regarding the characteristics of attributes is registered. As with the attribute 502, the information 504 contains information acquired by object recognition from an image captured by the sensor system 101. FIG.

(Data for risk calculation)
The data for risk calculation will be described with reference to the drawings. FIG. 6 is a diagram showing an example of a data table used for risk calculation according to this embodiment.

The data table for calculating the degree of risk is based on the action pattern data table shown in FIG. 4 and the park characteristic data table shown in FIG. is a table for calculating the degree of risk indicating The numerical value of the degree of risk indicates the degree of danger, and a large value indicates a high degree of risk, and a small value indicates a low degree of risk.

FIG. 6(a) is a data table for calculating the degree of risk set for the risky behavior of "jumping out of the doorway" among multiple types of risky behaviors. A data table 600 for calculating the degree of risk includes information on items such as the child's moving speed 601, the surroundings being visible 602, the presence or absence of jump-out prevention stakes 603, and the degree of risk 604. FIG.

The child's moving speed 601 field contains information about the child's moving speed, which is an element of the child's behavior pattern data table. Information indicating a plurality of set speeds is shown.

An item 602 that the surroundings are visible shows information as to whether the child can see the surroundings, which is an element of the child's behavior pattern data table. For example, one of a plurality of preset options such as "visible", "not visible", and "unknown" is displayed.

The item of presence/absence of jump-out prevention stakes 603 is information on the presence or absence of jump-out prevention stakes possessed by the attribute of the entrance of the park characteristics. "None" is set when there is no

The item of risk level 604 stores information about the risk level set based on the information of the child's moving speed 601 , the surroundings being visible 602 , and the presence or absence of jump-out prevention stakes 603 . The information about the degree of danger is not limited to being set based on three pieces of information: the child's moving speed 601, the surroundings being visible 602, and the presence or absence of jump-out prevention stakes 603. The information about the degree of danger may be set based on two pieces of three pieces of information, for example, the child's movement speed 601, the surroundings being visible 602, and the presence or absence of jump-out prevention stakes 603, or may be set based on one piece of information. may be set to the original.

The item of risk level 604 stores information quantifying the risk level for each combination of the child's moving speed 601, the surroundings being visible 602, and the presence/absence of jump-out prevention stakes 603. FIG. Also, this numerical value may be learned based on past examples of children's risky behaviors to improve the accuracy of the degree of risk. A learning flow of the risk calculation table will be described with reference to FIG.

FIG. 6(b) is a data table for calculating the degree of risk set for a risky action of "falling off a fence" among multiple types of risky actions. A data table 610 for calculating the degree of danger includes information on each item of child's athletic ability 611 , fence height 612 , fence type 613 , and degree of danger 614 .

The child's athletic ability 611 item contains information on the child's athletic ability, which is an element of the child's behavior pattern data table. Information of one of five stages of 2, 3, 4, and 5 is indicated.

The item of height of fence 612 shows information about the height of the fence, which is possessed by the fence attribute of the park characteristics. If there is a high fence of 5m or more, "high (5m or more)" is set, if there is a fence of 2m-5m, "medium (2m-5m)" is set, and if there is a fence of 1m-2m, "low ( 1m-2m)” is set, and “None” is set when there is no fence.

In the field of fence type 613, information about the type of fence possessed by the park characteristic fence attribute is set. If the fence has a shape or height that is easy to climb, "easy to climb" is set. "Unknown" is set.

The risk level 614 field stores information about the risk level set based on the child's athletic ability 611, fence height 612, and fence type 613 information. The information about the degree of risk is not limited to being set based on the three pieces of information of the child's athletic ability 611, the height 612 of the fence, and the type 613 of the fence. The information about the degree of risk may be set based on two pieces of information out of three pieces of information, for example, the child's athletic ability 611, the fence height 612, and the fence type 613, or may be set based on one piece of information. may be set.

Similar to the item of the degree of risk 604, the item of the degree of risk 614 stores information that quantifies the degree of risk for each combination of the data of the child's athletic ability 611, the height of the fence 612, and the type of fence 613. Also, this numerical value may be learned based on past examples of children's risky behaviors to improve the accuracy of the degree of risk. A learning flow of the risk calculation table will be described with reference to FIG.

(Data for judging risk)
The data for judging the degree of risk will be described with reference to the drawings. FIG. 7 is a diagram showing an example of a data table used for determining the degree of risk according to this embodiment. FIG. 7 shows the result of judging the risk from the child behavior pattern data table shown in FIG. 4A, the park characteristic data table shown in FIG. 5, and the risk calculation table shown in FIG. . The risk determination table 700 has information on each item of child 701 , risky behavior 702 , and risk 703 .

The child 701 item stores the name of a child whose risk level can be calculated among the child names 401 and whose risk level is to be calculated. Whether or not the degree of risk can be calculated depends on whether or not the information stored in the items of position 405 and moving direction 406 in FIG. 4A and the information stored in the item of position 501 in FIG. It is obtained by determining whether If the result of determination that there is a commonality is obtained, the table shown in FIG. 6 is checked, and if the degree of risk can be calculated from the table shown in FIG. 6, the following processing is executed. That is, as a risk determination result, the child's name (the name stored in the name 401 field shown in FIG. 4A) is added to the child 701 field in the table shown in FIG.

The item of risky behavior 702 stores risky behavior that a child may take. There are as many types of dangerous behaviors 702 as there are risk calculation tables shown in FIG.

The risk level 703 field stores information about the risk level for each child stored in the child 701 field. In the item of the degree of risk 703, the numerical value stored in the item of the degree of risk 604 and the item of the degree of risk 614 is input as data as it is.

(Danger prediction processing)
FIG. 8 is a flowchart showing the overall operation of the sensor system according to this embodiment. This processing procedure is stored as a program in the ROM 202 and executed by the CPU 201 . Also, some or all of the functions of the steps in FIG. 8 may be realized by hardware such as ASIC and electronic circuits. Note that the symbol "S" in the description of each process means a step in the flowchart, and the same applies to subsequent flowcharts.

The sensor system according to this embodiment is divided into four steps: an image acquisition process S801, a child or park recognition process S802, a risk calculation process S803, and a dangerous behavior notification process S804. These four steps are executed at regular time intervals.

In S801, a process of acquiring an image (moving image or still image) captured by the sensor system 101 is executed.

In S802, recognition processing of children and parks is executed using image recognition from the image acquired in the processing of S801. Note that image recognition is performed using a known technique.

In S803, processing for calculating the degree of risk is executed based on the data acquired in the processing of S802. Details of the processing for calculating the degree of risk will be described later with reference to the drawings.

In S804, based on the degree of risk obtained in the process of S803, if the degree of risk is equal to or greater than a predetermined threshold, information regarding the child's risky behavior is notified to a terminal such as the leader of the target child. .

(Risk calculation processing)
Details of the above-described risk degree calculation processing in S803 will be described with reference to the drawings. FIG. 9 is a flowchart showing the detailed flow of the risk calculation process according to the present embodiment.

In S901, the result of recognition in the process of S802 is obtained. That is, the child recognition result and the park recognition result are acquired.

In S902, park characteristic information (park characteristic data table) is acquired based on the park recognition result acquired in S901. In S903, a child behavior pattern data table is obtained from the child recognition result obtained in S901 and pre-registered behavior pattern data. That is, the child's behavior pattern data table is specified using the child's position, the child's movement speed, the child's current behavior type, and the child's characteristics that are analyzed and detected. The child's characteristics include the child's personality (activity level) and the child's motor skills (physical performance). The identification of the child's behavior pattern data table is not limited to the child's position, the child's moving speed, the child's current behavior type, and the child's feature detection results. For example, identifying the child's behavior pattern data table may use detection results of the child's location, the child's movement speed, and the child's current behavior type.

At S904, the child of interest is identified. In S905, using the park characteristic information (park characteristic data table) acquired in S902, the child behavior pattern data table acquired in S903, and the risk degree calculation table registered in advance, the park specified in S904 is used. The risk level of the target child is calculated. The risk calculation table is, for example, the table shown in FIG.

In S906, it is determined whether or not the risk calculated in S905 is equal to or greater than a predetermined threshold. If it is determined that the degree of risk is greater than or equal to the predetermined threshold (YES in S906), the process proceeds to S907. On the other hand, if it is determined that the degree of risk is not equal to or greater than the threshold (NO in S906), S907 is skipped and the process proceeds to S908.

In S907, the information is added to and stored in a risk notification list for notifying the guardian of the child that the risk is high. Based on this risk level notification list, the process of notifying the guardian of the target child or the like is executed in the process of S804.

In S908, it is determined whether or not the risk calculation has been completed for all recognized children. If it is determined that the risk calculation has not been completed for all children (NO in S908), the process returns to S904, and the child whose risk has not been calculated is processed. will be If it is determined that the calculation of the degree of risk has been completed for all children (YES in S908), the flow shown in FIG. 9 ends.

Therefore, the degree of risk is calculated for all children, and the child who satisfies the above conditions is added to the degree of risk notification list.

(Learning processing of action pattern DB)
Learning processing of the action pattern DB will be described with reference to the drawings. FIG. 10 is a flowchart showing the flow of learning processing of the action pattern DB according to this embodiment. This processing procedure is stored as a program in the ROM 202 and executed by the CPU 201 . Also, some or all of the functions of the steps in FIG. 10 may be realized by hardware such as ASIC and electronic circuits.

In S<b>1001 , an image (moving image or still image) is obtained from the image obtaining unit 301 .

In S1002, the image acquired in S1001 is analyzed to detect children. The child detection method may be, for example, a method of performing matching processing using pre-registered child face data, or a method of designation by an administrator or the like for the captured image.

In S1003, it is determined whether or not a child has been detected in the processing of S1002. If it is determined that the child cannot be detected (NO in S1003), the flow shown in FIG. 10 ends. On the other hand, if it is determined that a child has been detected (YES in S1003), the process proceeds to S1004.

At S1004, the target child is identified. In S1005, the amount of movement of the target child identified in S1004 is acquired. The amount of movement of the child may be derived, for example, based on the coordinates of a predetermined location of the child based on the captured images for a plurality of frames.

In S1006, the child's character ("active", "gentle", etc.) is determined from the amount of movement of the child acquired in S1005. In S1007, the physical ability is determined from the amount of movement of the child acquired in S1005.

In S1008, the personality determined in S1006 and the physical ability determined in S1006 are learned, and the action pattern DB 304 is updated.

In S1009, it is determined whether or not processing has been completed for all recognized children. If it is determined that the process has not been completed for all children (NO in S1009), the process returns to S1004, and the child whose personality and physical ability have not been determined is processed. If it is determined that the processing has been completed for all children (YES in S1009), the flow shown in FIG. 10 ends.

Therefore, the character (activity level) and physical ability are determined for all children, and the behavior pattern DB is updated by learning the character and physical ability of all children.

(Learning processing of risk calculation table)
Learning processing of the risk calculation table will be described with reference to the drawings. FIG. 11 is a flowchart showing the flow of learning processing for the risk calculation table according to this embodiment. The learning process of the risk calculation table includes two processes, for example, when a child takes a risky action and when it is based on past examples of risky action. FIG. 11(a) shows a case where a child took a risky action, and FIG. 11(b) shows a case of a past risky action. This processing procedure is stored as a program in the ROM 202 and executed by the CPU 201 . Also, some or all of the functions of the steps in FIG. 10 may be realized by hardware such as ASIC and electronic circuits.

First, learning processing of the risk calculation table when a child takes a dangerous action will be described with reference to FIG. 11(a).

In S<b>1101 , an image (moving image or still image) is obtained from the image obtaining unit 301 .

In S1102, if the child appearing in the image has taken a dangerous action, the action of the child appearing in the image is detected from the image.

In S1103, it is determined whether or not a dangerous behavior has been detected in the processing of S1102. If it is determined that no dangerous behavior has been detected (NO in S1103), the flow shown in FIG. 11A is terminated. On the other hand, if it is determined that dangerous behavior has been detected (YES in S1103), the process proceeds to S1104.

In S1104, the target risky behavior is specified among the detected risky behaviors. In S1105, the behavior pattern data table of the child who took the target dangerous behavior identified in S1104 is acquired. In S1106, park property information (park property data table) is acquired.

In S1107, learning is performed based on the behavior pattern table of the target child taking the dangerous behavior acquired in S1105 and the park characteristic data table acquired in S1105, and the risk calculation table 309 is updated.

In S1108, it is determined whether or not the learning process for all detected dangerous behaviors has been completed. If it is determined that the learning process has not been completed for all the detected dangerous behaviors (NO in S1108), the process returns to S1104, and the dangerous behaviors for which the learning process has not been performed are processed. will be On the other hand, when the determination result that the learning process for all the detected dangerous behaviors has been completed is obtained (YES in S1108), the flow shown in FIG. 11(a) ends.

Next, the learning process of the risk calculation table based on past cases of dangerous behavior will be described with reference to FIG. 11(b). Note that this flowchart assumes that dangerous behavior data, children's behavior pattern data, and park characteristic data directly input by the user are used.

In S1111, the dangerous behavior that the child has taken in the past is acquired from the dangerous behavior data. Note that the risky behavior data is data that is directly input by the user and includes information about risky behavior that children have taken in the past. In S1112, the behavior pattern of the child who took the risky behavior acquired in S1111 is acquired. In S1113, the characteristic information of the park subject to the dangerous behavior acquired in S1111 is acquired.

In S1114, the child's past risky behavior acquired in S1111, the behavior pattern of the child when the child took the risky behavior acquired in S1112, and the characteristic information of the park acquired in S1113 are learned, and a risk calculation table is created. to update.

(Notification method)
A notification method will be described with reference to the drawings. FIG. 12 is a diagram illustrating an example of a notification method according to this embodiment. In the method of notifying a risk estimation result according to the present embodiment, it is assumed that a leader of a child who is likely to take a risky action has a portable terminal 1201 . The mobile terminal 1201 may be, for example, a terminal device capable of receiving information transmitted by the sensor system and displaying the received information, such as a smart phone or a tablet terminal.

When it is determined that there is a high possibility of taking a risky action in the risk calculation process of S803 shown in FIG. do. For example, the sensor system notifies the mobile terminal 1201 of the leader of the message 1202 that "Man E is about to run out of the exit."

In the example shown in FIG. 7, the risk level of the risky behavior of "man E" to "jump out of the exit" is 90, so if the threshold is less than 90, it indicates that man E is about to run out of the exit. Notify me of a message. For example, a predetermined message "E man is about to jump out of the exit" 1202 is notified. For example, the predetermined message may have a format prepared in advance, and may be created by associating the name of the child identified as the target with the child's risky behavior.

As described above, according to the present embodiment, it is possible to accurately estimate the possibility of taking a risky action.

<Embodiment 2>
In this embodiment, in addition to the behavior patterns of children and the characteristics of the park, a mode of estimating the degree of risk using the characteristics of moving objects in the park will be described with reference to the drawings. FIG. 13 is a diagram showing a usage example of the sensor system according to this embodiment. FIG. 13(a) shows an outline of a usage example of the sensor system, and FIG. 13(b) shows a case where grid lines are added to the imaging area.

As in the first embodiment, the sensor system of this embodiment recognizes children 1331 to 1336 and facilities 1311 to 1315 related to park characteristics from images captured by the sensor system 1301, Acquire behavior pattern information and park characteristic information.

In addition to these, the sensor system 1301 recognizes the ball 1321 that the subject's child is playing with as a moving object (moving object) in the park, and acquires the position and moving speed of the ball 1321 as property information of the moving object in the park. do. Note that the moving object is not limited to the ball. For example, it may be a non-subject such as a frisbee, an animal, or a child different from the subject child.

Using the child's behavior pattern information, the property information of the park, and the property information of moving objects in the park, the degree of danger is estimated and notified to the guardian or leader.

In FIG. 13, a child 1335 is chasing a ball 1321 with a jungle gym 1314 in the chasing direction. Therefore, since there is a possibility that the child 1335 will "collide with the playground equipment" as a dangerous behavior, the leader of the child 1335 is notified of the possibility that the child 1335 will "collide with the playground equipment".

FIG. 13(b) is a diagram showing an example of recognizing an area with respect to FIG. 13(a). Area information indicating areas A1, A2, A3, B1, B2, B3, C1, C2, and C3 is included in the behavior pattern information of children, the property information of parks, and the property information of moving objects in parks. Areas are recognized by analyzing captured images obtained by the sensor system 1301 . Although the present embodiment shows an example in which the entire park is divided into nine areas as in the first embodiment, the present invention is not limited to this. The entire park may be divided into smaller pieces, such as 16 areas. Certain areas may be divided more finely than other areas. By dividing the data more finely, it is possible to obtain more accurate information about the behavior patterns of children, the characteristics of parks, and the characteristics of moving objects, and thus the accuracy of estimating the degree of risk can be increased.

A hardware configuration example of the sensor system and the terminal according to the present embodiment is the same as that of the first embodiment, and the description thereof will be omitted.

(Functional configuration of sensor system)
FIG. 14 is a block diagram showing a functional configuration example of the sensor system according to this embodiment. Similar to the sensor system 101 , the sensor system 1301 has an image acquisition unit 1401 , a park property acquisition unit 1403 , an action pattern acquisition unit 1404 , an action pattern DB 1405 and a learning unit 1406 . The sensor system 1301 has a park characteristic (characteristic data table) 1408 , an action pattern table 1409 , a risk determination unit 1410 , a risk calculation table 1411 , a risk determination table 1412 , and a notification unit 1413 . The sensor system 1301 further has a moving object property acquisition unit 1402 and a moving object property 1407 . That is, the sensor system 1301 further has a moving object property acquisition unit 1402 and a moving object property 1407 in addition to the functional units of the sensor system 101 . A detailed description of the same functional units as those of the sensor system 101 is omitted.

A moving object property acquisition unit 1402 recognizes the property of a moving object in the park from the image (moving or still image) received from the image acquiring unit 1401, and calculates the property of the moving object (property data table) 1407 based on the recognition result. to create An example of a data table of characteristics of moving objects will be described later with reference to the drawings.

The risk determination unit 1410 calculates the risk of dangerous behavior for each child based on the characteristics 1407 of the moving object, the characteristics 1408 of the park, the behavior pattern table 1409 of the monitored person (child), and the risk calculation table 1411. . Then, the risk determination unit 1410 creates a risk determination table 1412 based on the calculation result.

(Characteristics of moving objects)
The characteristics of moving objects will be described with reference to the drawings. FIG. 15 is a diagram showing an example data table of characteristics of moving objects according to the present embodiment. But we assume moving thing in park. It is assumed that the data table shown in FIG. 15 is created for the park shown in FIG. The data table 1500 has information on each item of moving object name 1501 , position 1502 , moving direction 1503 , and moving speed 1504 .

The moving object name 1501 field stores information about the name of the moving object. The item of position 1502 registers information about the position (place) where the moving object exists in the park. When the park is divided into a plurality of areas such as areas A1 to C3 shown in FIG. 13(b), it is shown in which area the target moving object exists. Here, it is expressed as an area, but it may be expressed as a coordinate position (x, y, z) instead of an area. Information about the position is obtained from images captured by the sensor system 1301 .

Information about the moving direction of the moving object is registered in the item of moving direction 1503 . From the image captured by the sensor system 1301, information indicating from which area to which area the moving object is heading from among the areas A1 to C3 is acquired and registered. If the moving speed 1504, which will be described later, is "stationary", the moving object is not moving, so the moving direction data is not acquired.

Information on the moving speed of a moving object is registered in the moving object moving speed 1504 item. Based on the image captured by the sensor system 1301, the moving distance of the moving object per unit time is estimated, the moving speed is derived based on the estimation result, and information about the moving speed is registered. Based on the acquired moving speed and a predetermined threshold value, it is classified into one of "fast", "slow", and "stationary" and registered. In this embodiment, the moving speed is expressed in three stages of "fast", "slow", and "stationary". You may Also, the information about the moving speed may be expressed using a specific numerical value.

(Danger prediction processing)
FIG. 16 is a flow chart showing the overall operation of the sensor system according to this embodiment. This processing procedure is stored as a program in the ROM 202 and executed by the CPU 201 . Also, some or all of the functions of the steps in FIG. 16 may be realized by hardware such as ASIC and electronic circuits. In this embodiment, the processing of S802 shown in FIG. 8 is changed to processing S1602 for recognizing children, parks, and moving objects.

In S1601, processing for acquiring an image (moving image or still image) captured by the sensor system 1301 is executed.

In S1602, recognition processing of children, parks, and moving objects is executed using image recognition from the image acquired in the processing of S1601. Note that image recognition is performed using a known technique.

In S1603, processing for calculating the degree of risk is executed based on the data acquired in the processing of S1602. As a result, in S1603, processing that takes into account the characteristics of the moving object is performed. Details of the processing for calculating the degree of risk will be described later with reference to the drawings.

In S1604, based on the degree of risk obtained in the processing of S1603, if the degree of risk is equal to or greater than a predetermined threshold, information regarding the child's risky behavior is notified to a terminal such as the leader of the target child. .

(Risk calculation process)
Details of the above-described risk degree calculation processing in S1603 will be described with reference to the drawings. FIG. 17 is a flowchart showing the detailed flow of the risk calculation process according to this embodiment.

In S1701, the result of recognition in the processing of S1602 is acquired. That is, recognition results of children, parks, and moving objects are acquired.

In S1702, park characteristic information (park characteristic data table) is acquired based on the park recognition result acquired in S1701. In S1703, moving object property information (moving object property data table) is acquired based on the moving object recognition result acquired in S1701. In S1704, a child behavior pattern data table is obtained from the child recognition result obtained in S1701 and pre-registered behavior pattern data.

At S1705, the target child is identified. In S1706, the risk level of the target child identified in S1705 is calculated using the following data. That is, as the data, the park characteristics data table acquired in S1702, the moving object characteristics data table acquired in S1703, the child behavior pattern data table acquired in S1704, and the pre-registered degree of risk A calculation table is used.

In S1707, it is determined whether the degree of risk calculated in S1706 is equal to or greater than a predetermined threshold. If it is determined that the degree of risk is greater than or equal to the predetermined threshold (YES in S1707), the process proceeds to S1708. On the other hand, if it is determined that the degree of risk is not equal to or greater than the predetermined threshold (NO in S1707), S1708 is skipped and the process proceeds to S1709.

In S1708, the information is added to and stored in a risk notification list for notifying the guardian of the child that the risk is high. Based on this risk notification list, the process of notifying the guardian of the target child in the process of S1705 is executed.

In S1709, it is determined whether or not the risk calculation has been completed for all recognized children. If it is determined that risk calculation has not been completed for all children (NO in S1709), the process returns to S1705, and processing is performed for children whose risk levels have not yet been calculated. will be If it is determined that the calculation of the degree of risk has been completed for all children (YES in S1709), the flow shown in FIG. 17 ends.

Therefore, the degree of risk is calculated for all children, and the child who satisfies the above conditions is added to the degree of risk notification list.

As described above, according to the present embodiment, it is possible to accurately estimate the possibility of taking a risky action.

<Other embodiments>
In the above description, as an example of the method of notifying the degree of risk, "Notification to leader's terminal" has been described, but the method is not limited to this, and other methods may be used. Another notification method will be described with reference to the drawings. FIG. 18 is a diagram showing an example of a notification method. FIG. 18(a) shows an example of notifying the leader by voice, and FIG. 18(b) shows an example of notifying the leader's HMD.

As shown in FIG. 18(a), a child (Takashi) 1801 to be notified is moving as if he is about to jump out of the entrance of the park. The sensor system 1803 detects the movement of the child 1801, recognizes that the child 1801 is Takashi-kun, and notifies the child 1801 with a voice message urging him to stop as a countermeasure to prevent dangerous behavior. do. The message is, for example, “Takashi, stop!” 1802 .

Also, as shown in FIG. 18(b), a child (Takashi) 1807 is moving as if he is about to jump out of the doorway. A sensor system (not shown) detects the movement of the child 1807 and recognizes that the child 1807 is Takashi. Then, the sensor system notifies a head-mounted display (HMD) 1805 worn by a leader 1804 of the child 1807 of a message that the child 1807 is likely to perform a dangerous action. The message is, for example, "Takashi is about to leave the exit" 1806 .

In the above description, a child is used as an example of a person who behaves dangerously, but the person is not limited to this. For example, an adult may be targeted as a person who performs dangerous behavior.

Further, in the above, an example of acquiring the position information from the imaging device in order to acquire the characteristics of the park has been presented, but the position information of the characteristics of the park may be acquired using GPS or map information.

In the above description, an example in which a series of processes for risk determination is performed inside the sensor system 101 or 1301 has been described, but the present invention is not limited to this. A series of processes for determining the degree of risk may be performed by being divided between the sensor system and another server, edge device, or the like. For example, only the image acquisition units 301 and 1401 in the functional configuration shown in FIGS. 3 and 14 may be performed inside the imaging apparatus, and the subsequent processing by the functional units may be performed by a server, an edge device, or the like. Subsequent processes are park property acquisition units 302 and 1403 , action pattern acquisition units 303 and 1404 , moving object property acquisition unit 1402 , learning units 305 and 1406 , and risk determination units 308 and 1410 .

Although a mode in which one sensor system 101, 1301 is installed in a park has been described above, the number of installed sensor systems is not limited to this. The number of installed sensor systems may be multiple, such as two. In this way, by setting multiple sensor systems and setting the imaging area of the multiple imaging devices over the entire park, blind spots from the imaging devices can be eliminated and the accuracy of estimating the degree of risk can be improved. . In this case, it is preferable that a plurality of imaging devices perform synchronous imaging so as not to lower the accuracy of human recognition processing and park characteristic recognition processing. Moreover, the sensor systems 101 and 1301 may be mobile instead of stationary.

In the above description, a park is described as an example of a place to be applied, but it is not limited to this. For example, it can be applied to school playgrounds, amusement parks, gymnasiums, swimming pools, and the like.

In the above description, the mode of notifying the guardian or leader of the target child of the information about the degree of risk has been described, but the present invention is not limited to this. For example, information about the degree of danger may be notified to the park manager. Moreover, when the target child has a terminal, the information about the degree of risk may be notified to the terminal that the target child has.

In the above description, a mode in which the degree of risk is calculated from the behavior pattern of the child and the characteristics of the park, or from the behavior pattern of the child, the characteristics of the park, and the characteristics of the moving object has been described, but the present invention is not limited to this. For example, the obtained degree of risk may be weighted based on prestored data such as environment such as weather and time. When the weather is rainy, the horizontal bars and jungle gyms are more slippery than when the weather is sunny. Weighting may be performed when the weather is rainy. In addition, when there is a strong wind, it is more difficult to stabilize the body than when there is no wind. Weighting may be performed. In addition, in the evening and at night, it is difficult to see the surroundings compared to the daytime. may

The present disclosure provides a program that implements one or more functions of the above-described embodiments to a system or device via a network or storage medium, and one or more processors in a computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

Claims (26)

an identifying means for identifying a behavior pattern of a person reflected in a captured image;
a first acquisition means for acquiring first characteristic information indicating characteristics of an imaging location;
a determining means for determining a degree of possible danger to the person using the identified behavioral pattern and the acquired first characteristic information;
output means for outputting information about the determined degree of risk;
An image processing apparatus characterized by comprising: A table in which the degree of risk is specified in association with the type of behavior pattern is held in advance for each type of characteristic,
The determining means determines a table corresponding to the acquired first characteristic information from among the held tables, and determines a risk associated with the identified behavior pattern from the determined table. 2. The image processing apparatus according to claim 1, wherein the degree of is determined. the person's behavior pattern includes the person's location, the person's movement speed, and the person's current behavior type;
The identifying means identifies the person's behavior pattern based on the person's position, the person's moving speed, and the person's current behavior type obtained by analyzing the captured image. 3. The image processing apparatus according to claim 1 or 2, wherein the person's behavioral pattern includes characteristics of the person;
4. The image processing apparatus according to claim 3, wherein the identifying means identifies the behavior pattern of the person based on the characteristics of the person detected by analyzing the captured image. 5. The image processing apparatus according to claim 4, wherein the person's features include the person's activity level and the person's physical ability. further comprising second acquisition means for acquiring second characteristic information indicating characteristics of a moving object present at the imaging location;
5. The image processing apparatus according to any one of claims 1 to 4, wherein the determining means determines the degree of risk also using the obtained second characteristic information. the second characteristic information includes the position of the moving object, the moving speed of the moving object, and the moving direction of the moving object;
The second acquisition means acquires the second characteristic information based on the position of the moving object, the moving speed of the moving object, and the moving direction of the moving object obtained by analyzing the captured image. The image processing apparatus according to claim 6, characterized in that: 8. The image processing apparatus according to claim 6, wherein said moving object is a toy used by said person or an animal. 9. The image processing apparatus according to any one of claims 1 to 8, wherein the determining means determines the degree of risk by weighting data stored in advance. The image processing apparatus according to claim 9, wherein the weighting is determined based on the environment of the imaging location. 11. The image processing apparatus according to any one of claims 1 to 10, wherein the output means notifies the information based on the determined degree of risk and a predetermined threshold value. said person is a child;
The output means notifies a portable terminal possessed by the child, the child's guardian, or the child's leader of the information, and informs the portable terminal of the degree of danger and the dangerous behavior of the child. 12. The image processing apparatus according to claim 11, wherein the image is displayed. 11. The image processing apparatus according to claim 10, wherein said output means notifies said information by voice. said person is a child;
12. The method according to claim 11, wherein the output means displays information about the child's dangerous behavior on a head-mounted display worn by the child's guardian or the child's leader. image processing device. 15. The image processing apparatus according to claim 12, wherein said output means notifies said child of a dangerous action that is highly likely to occur. 15. The image processing apparatus according to claim 12, wherein said output means notifies information regarding countermeasures for preventing dangerous actions that said person is highly likely to take. 3. The image processing apparatus according to claim 2, further comprising learning means for learning said table. The learning means, when a risky behavior is detected from the captured image, the behavior pattern of the subject who took the risky behavior acquired from the captured image, and the first characteristic information acquired from the captured image. 18. The image processing apparatus according to claim 17, wherein the table is learned using and. 19. The image processing according to claim 18, wherein the learning means learns the table using a database in which action patterns of people who have taken risky actions and the first characteristic information are associated with each other. Device. The first characteristic information includes information about facilities located at the imaging location,
20. The first characteristic information according to any one of claims 1 to 19, wherein the first acquisition means acquires the first characteristic information based on information about the facility obtained by analyzing the captured image. The image processing device according to . The first characteristic information includes information on the entrance/exit of the facility and the position and shape of equipment provided in the facility,
The first acquisition means acquires the first characteristic information based on information relating to the entrance/exit of the facility and the position and shape of equipment provided in the facility obtained by analyzing the captured image. 21. The image processing apparatus according to claim 20, characterized by: the facility is a park;
The first characteristic information includes information about the entrance and exit of the park, the fence provided in the park, and the position and shape of the playground equipment provided in the park,
Based on the information on the position and shape of the entrance/exit of the park, the fence provided in the park, and the playground equipment provided in the park, obtained by analyzing the captured image, the first acquisition means acquires the information obtained by analyzing the captured image. 22. The image processing apparatus according to claim 21, wherein 1 characteristic information is obtained. The image processing apparatus according to any one of claims 1 to 22, further comprising imaging means for imaging the person and acquiring the captured image. 24. The image processing apparatus according to claim 23, wherein said imaging means obtains temporally continuous images as said captured images. an identifying step of identifying a behavioral pattern of a person reflected in the captured image;
a first acquisition step of acquiring first characteristic information indicating characteristics of an imaging location;
a determination step of determining a degree of danger that may occur to the person using the identified behavior pattern and the acquired first characteristic information;
an output step of outputting information about the determined degree of risk;
An image processing method comprising: A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 24.

Images