WO2016002408A1 - 画像処理装置、監視システム、画像処理方法、及びプログラム - Google Patents
画像処理装置、監視システム、画像処理方法、及びプログラム Download PDFInfo
- Publication number
- WO2016002408A1 WO2016002408A1 PCT/JP2015/065725 JP2015065725W WO2016002408A1 WO 2016002408 A1 WO2016002408 A1 WO 2016002408A1 JP 2015065725 W JP2015065725 W JP 2015065725W WO 2016002408 A1 WO2016002408 A1 WO 2016002408A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- monitoring target
- index value
- degree
- captured
- image processing
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/188—Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/292—Multi-camera tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B13/00—Burglar, theft or intruder alarms
- G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
- G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
- G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
- G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
- G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
Definitions
- the present invention relates to image processing technology.
- Patent Document 1 discloses an abnormal behavior detection device that detects abnormal behavior. This apparatus divides the congestion level into a plurality of stages, and obtains a normal movement pattern according to the congestion level. And it is determined whether it is abnormal behavior by determining whether the movement pattern of a target object matches the normal movement pattern according to the congestion level at that time.
- Patent Document 2 discloses a monitoring system having a function of presenting a state of a monitoring target on a video displayed on a monitor. Specifically, a numerical value indicating the common degree of the movement direction of the crowd and the movement direction of the crowd is presented on an image obtained by imaging the crowd.
- the surveillance staff needs to keep watching the video of the surveillance camera for a certain amount of time if he wants to know whether the person shown in the surveillance camera is a person who passes the spot or a person who is ashamed of the spot. There is.
- An object of the present invention has been made in view of the above problems.
- An object of the present invention is to provide a technique by which a monitoring person monitoring a monitoring camera can immediately grasp the current state of a monitoring target.
- the first image processing apparatus uses a plurality of captured images captured at different times by a camera to calculate an index value indicating a degree of change in a state of a monitoring target reflected in the captured image.
- the second image processing apparatus includes a calculation unit that calculates a degree of change in a state of a monitoring target reflected in the captured image using a plurality of captured images captured at different times by a camera. Presenting means for changing the color of the area representing the monitoring target to a color based on the degree of change on the captured image captured by the camera.
- a third image processing apparatus includes a calculation unit that calculates a degree of change in a state of a monitoring target reflected in the captured image using a plurality of captured images captured at different times by a camera. Presenting means for emphasizing the monitoring target based on the degree of change on the captured image captured by the camera.
- the monitoring system provided by the present invention includes a camera, an image processing device, and a display screen.
- the image processing apparatus is an image processing apparatus provided by the above-described present invention.
- the display screen displays the first captured image on which a display based on the index value is presented by the presenting unit.
- the image processing method provided by the present invention is executed by a computer.
- the image processing method includes an index value calculation step of calculating an index value indicating a degree of change in a state of a monitoring target reflected in the captured image, using a plurality of captured images captured at different times by a camera; And a presentation step of presenting a display based on the index value on the first captured image captured by the camera.
- the program provided by the present invention has the function of operating as the image processing apparatus provided by the present invention by causing the computer to have the function of each functional component included in the image processing apparatus provided by the present invention. Make it.
- a technology that enables a monitoring person who monitors a monitoring camera to immediately grasp the current state of the monitoring target.
- FIG. 1 is a block diagram illustrating an image processing apparatus according to a first embodiment. It is a figure which illustrates notionally the process which calculates the index value of the monitoring object for every presentation object picture. It is a figure which illustrates notionally the process which presents a common display with respect to a some presentation object image. It is a block diagram which illustrates the hardware constitutions of an image processing apparatus. 3 is a flowchart illustrating a flow of processing executed by the image processing apparatus according to the first embodiment. It is a figure which illustrates notionally the process which presents the display based on an index value like animation. It is a figure which illustrates a mode that a person was piled up.
- FIG. 4 is a block diagram illustrating an image processing apparatus according to a second embodiment.
- FIG. 10 is a block diagram illustrating an image processing apparatus according to a fourth embodiment.
- FIG. 10 is a diagram for explaining a method of calculating an index value by an index value calculation unit according to the fifth embodiment.
- 10 is a flowchart for explaining a flow of processing executed by the image processing apparatus according to the fifth embodiment. It is a figure which illustrates the relationship between a user's gaze direction and a partial region. It is a figure which illustrates the information which matches the partial area
- FIG. 1 is a block diagram illustrating an image processing apparatus 2000 according to the first embodiment.
- arrows indicate the flow of information.
- each block represents a functional unit configuration, not a hardware unit configuration.
- the image processing apparatus 2000 includes an index value calculation unit 2020 and a presentation unit 2040.
- the index value calculation unit 2020 acquires a plurality of images captured by the camera 3000 (hereinafter referred to as captured images).
- the camera 3000 is, for example, a surveillance camera.
- the plurality of captured images are captured at different times.
- the plurality of captured images are each frame constituting a moving image captured by the camera 3000.
- the index value calculation unit 2020 calculates an index value indicating the degree of change in the state of the monitoring target shown in the captured image using the acquired captured image.
- the presenting unit 2040 presents a display based on the index value calculated by the index value calculating unit 2020 on the captured image captured by the camera 3000.
- the captured image may be used for calculating the index value or may not be used.
- the presentation unit 2040 presents a display based on the index value calculated using the first to (n) th captured images on the (nth) captured image.
- the presentation unit 2040 presents a display based on the index value calculated using the first to (n) th captured images on the (n + 1) th captured image.
- the captured image of the target on which the presentation unit 2040 presents the display based on the index value is also referred to as a presentation target image.
- the presentation unit 2040 calculates a monitoring target index value for each presentation target image.
- FIG. 2 is a diagram conceptually illustrating the process of calculating the monitoring target index value for each presentation target image.
- the presentation unit 2040 presents a display based on the index value calculated using the first to n th captured images on the n + 1 th captured image.
- the presentation unit 2040 presents a display based on the index value calculated using the 2nd to (n + 1) th captured images on the (n + 2) th captured image, and the 3rd to (n + 2) th captured images.
- a display based on the index value calculated using the captured image is presented on the (n + 3) th captured image.
- the presentation unit 2040 may use an index value calculated using a plurality of captured images in common for a plurality of presentation target images.
- FIG. 3 is a diagram conceptually illustrating a process of presenting a common display for a plurality of presentation target images.
- the presentation unit 2040 presents a display based on the index value calculated using the first to n th captured images on each of the n + 1 th to 2n th captured images.
- the presentation unit 2040 uses the index values calculated using the (n + 1) th to (2n) th captured images to present them on the [2n + 1] th to (3n) th captured images.
- Each functional component of the image processing apparatus 2000 may be realized by a hardware component (eg, a hard-wired electronic circuit) that implements each functional component, or a hardware component and a software component. (For example, a combination of an electronic circuit and a program for controlling the electronic circuit).
- a hardware component eg, a hard-wired electronic circuit
- a hardware component and a software component for example, a combination of an electronic circuit and a program for controlling the electronic circuit.
- FIG. 4 is a block diagram illustrating a hardware configuration of the image processing apparatus 2000.
- the image processing apparatus 2000 includes a bus 1020, a processor 1040, a memory 1060, a storage 1080, and an input / output interface 1100.
- the bus 1020 is a data transmission path through which the processor 1040, the memory 1060, the storage 1080, and the input / output interface 1100 transmit / receive data to / from each other.
- the method of connecting the processors 1040 and the like is not limited to bus connection.
- the processor 1040 is an arithmetic processing device such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit).
- the memory 1060 is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory).
- the storage 1080 is a storage device such as a hard disk, SSD (Solid State Drive), or memory card.
- the storage 1080 may be a memory such as a RAM or a ROM.
- the input / output interface 1100 is an input / output interface for the image processing apparatus 2000 to transmit / receive data to / from an external apparatus or the like.
- the image processing apparatus 2000 acquires a captured image via the input / output interface 1100.
- the image processing apparatus 2000 outputs a captured image presenting a display based on the index value via the input / output interface 1100.
- the storage 1080 includes an index value calculation module 1220 and a presentation module 1240 as programs for realizing the functions of the image processing apparatus 2000.
- the processor 1040 implements the functions of the index value calculation unit 2020 and the presentation unit 2040 by executing these modules.
- the processor 1040 may execute the modules after reading them onto the memory 1060 or without reading them onto the memory 1060.
- each module may be stored in the memory 1060.
- the image processing apparatus 2000 may not include the storage 1080.
- FIG. 5 is a flowchart illustrating the flow of processing executed by the image processing apparatus 2000 according to the first embodiment.
- the index value calculation unit 2020 acquires a captured image.
- the presentation unit 2040 calculates an index value indicating the degree of change in the state of the monitoring target shown in the captured image.
- the presentation unit 2040 presents a display based on the index value on the captured image (presentation target image) captured by the camera 3000.
- a display indicating the degree of change in the state of the monitoring target is presented on the presentation target image.
- a display indicating the degree of change in the state of the monitoring target is presented on the presentation target image. For example, assume that an image captured by the camera 3000 is displayed on the display screen 4000. In this case, an image on which the display based on the index value is superimposed is displayed on the display screen 4000. Therefore, a supervisor or the like can easily grasp how much the state of the monitoring target has changed in a short time. Therefore, a supervisor or the like can immediately and easily grasp the current state of the monitoring target.
- the method by which the index value calculation unit 2020 acquires the captured image is arbitrary.
- the index value calculation unit 2020 acquires a captured image from the camera 3000.
- the index value calculation unit 2020 may acquire a captured image stored in a storage device outside the camera 3000.
- the camera 3000 is configured to store the captured image in this storage device.
- the storage device may be provided inside the image processing apparatus 2000 or may be provided outside the image processing apparatus 2000.
- the process of acquiring the captured image may be a process in which the index value calculation unit 2020 receives the captured image output by the camera 3000 or the storage device, or the index value calculation unit 2020 may receive the camera 3000 or the storage unit. It may be a process of reading a captured image from the apparatus.
- the image processing apparatus 2000 treats an object such as a person or a thing, or a set of these objects (such as a crowd) as a monitoring target.
- the object representing the object may include a place. That is, the image processing apparatus 2000 may treat a place (area) shown in the captured image as a monitoring target.
- the index value calculation unit 2020 divides an area included in the captured image into a foreground area and a background area, and treats the foreground area as an object.
- a method for extracting an object such as a person or an object from an image is not limited to the above-described method.
- a technique for extracting an object such as a person or an object from an image is a known technique, and the index value calculation unit 2020 can use these known techniques.
- description of these known techniques is omitted.
- the image processing apparatus 2000 may set all objects extracted from the captured image as monitoring targets, or may set only specific objects as monitoring targets. For example, the image processing apparatus 2000 treats only a person or a group of people (crowd) as a monitoring target. Further, the image processing apparatus 2000 may monitor only a specific person or a crowd. In this case, the image processing apparatus 2000 acquires information (for example, a black list) indicating the monitoring target, and determines the monitoring target based on the information.
- the information indicating the monitoring target indicates a feature amount of each object to be monitored, for example.
- the information indicating the monitoring target may be information indicating the characteristics of the person to be monitored, such as “wearing a hat” or “wearing sunglasses”.
- a technique for identifying an object having a specific feature from objects included in an image is a known technique, a detailed method is omitted.
- the presentation unit 2040 presents a display based on the index value on the captured image (presentation target image) captured by the camera 3000.
- the process of presenting the display based on the index value on the presentation target image is a process of presenting the index value calculated for the monitoring target, for example, around the monitoring target in the presentation target image.
- Other examples of “a process of presenting a display based on an index value on a presentation target image” will be described in each embodiment described later.
- “presenting a display on the presentation target image” is a process of embedding or superimposing the display on the presentation target image, for example.
- the presentation unit 2040 may output the presentation target image in which the display is embedded to an output device such as the display screen 4000, or store the image in a storage device provided inside or outside the image processing device 2000. May be. In the latter case, the display target image stored in the storage device is read by the display screen 4000 and other devices and output to the display screen 4000.
- the display screen 4000 is, for example, a monitor installed in a supervisor's room, a monitor of a guard's portable terminal guarding the site, or the like.
- the presentation unit 2040 may separately generate image data representing the display without embedding the display based on the index value in the presentation target image.
- the display is presented on the presentation target image by displaying the image data and the presentation target data together.
- the presentation unit 2040 may present a display based on the index value on the map by using the map data of the facility where the camera 3000 is installed.
- This map data is displayed on a display screen 4000 or a monitor such as a guard's mobile terminal.
- the position of the monitoring target on the map can be calculated based on various parameters of the camera 3000 (such as the installation position of the camera 3000 and the orientation of the camera 3000) and the position of the monitoring target on the captured image.
- the presentation unit 2040 acquires and uses map data of a facility where the camera 3000 is installed and various parameters related to the camera 3000.
- the relationship between the various parameters of the camera 3000 and the position on the map is defined in advance by performing processing such as calibration.
- FIG. 6 is a diagram conceptually illustrating a process of presenting a display based on the index value like an animation.
- the index value calculation unit 2020 calculates an index value indicating the degree of change in the state of the monitoring target in the first to (n) th captured images, and generates a display 1 based on the index value.
- the index value calculation unit 2020 generates the display 2 using the (n + 1) th to (2n) th captured images, and generates the display 3 using the (2n + 1) th to the (3n) th captured images.
- the presentation unit 2040 presents the display 1 to the 3n th captured image, presents the display 2 to the 3n + 1 th captured image, and displays 3 to the 3n + 2 th captured image. Present. Thereby, the displays 1 to 3 are presented like an animation. Furthermore, the presentation unit 2040 may repeat the display of “display 1, display 2, display 3” for the subsequent captured images. By doing so, an animation composed of displays 1 to 3 is repeatedly presented on the captured image.
- the index value calculation unit 2020 calculates an index value indicating the degree of change in the state of the monitoring target in the captured image.
- the “status of the monitoring target” handled by the image processing apparatus 2000 is various, and the index value calculation method depends on what is handled as the monitoring target.
- the state of the monitoring target handled by the image processing apparatus 2000 and a method for calculating an index value indicating the degree of change in the state of the monitoring target will be exemplified.
- the index value calculation unit 2020 handles the position of the monitoring target as the state of the monitoring target. For example, if there is a person who has stopped for a long time in a passage through which a person passes, the person should be carefully monitored. Therefore, the index value calculation unit 2020 treats the degree of change in the position of the monitoring target as the degree of change in the state of the monitoring target. The degree of change in the position of the monitoring target can be rephrased as the degree of staying of the monitoring target. By calculating an index value based on the staying degree of the monitoring target and presenting a display based on the index value on the presentation target captured image, a monitor or the like can immediately grasp the staying degree of each monitoring target. it can.
- the degree of change in the position of the monitoring target is represented by the length of time that a certain monitoring target (such as the same person or crowd) is shown in the captured image.
- the length of time that the monitoring target appears in the captured image is, for example, how many captured images the monitoring target is captured in a captured image (each frame constituting a moving image) captured in time series.
- the index value calculation unit 2020 may represent the degree of change in the position of the monitoring target by the size of the moving range of the monitoring target.
- the size of the movement range of the monitoring target can be represented by the size of an area (such as a circle or a rectangle) that includes all the positions of the monitoring target in each of the plurality of captured images.
- the size of the region can be represented by the area of the region or the length of the side or diameter of the region.
- the index value calculation unit 2020 may calculate the degree of change in the position of the monitoring target in consideration of the degree of spatial movement such as the movement of a part of the body of the monitoring target.
- the index value calculation unit 2020 treats the frequency at which a certain monitoring target appears in the captured image as the state of the monitoring target. That is, the index value calculation unit 2020 treats the degree of change in the frequency at which a certain monitoring target appears in the captured image (the length of time that appears in the captured image) as the degree of change in the state of the monitoring target. For example, a certain monitoring target is not detected once in the captured image in the first 30 minutes, detected once in the captured image in the next 30 minutes, and detected five times in the captured image in the next 30 minutes. And In this case, the frequency that the monitoring target appears in the captured image is increasing. Therefore, the degree of change in the state of the monitoring target indicates a high degree.
- the image processing apparatus 2000 calculates an index value based on the degree of change in the frequency at which the monitoring target appears in the captured image, and presents a display based on the index value on the presentation target captured image. Thereby, the monitoring staff who saw the presentation target image can immediately grasp the degree of change in the frequency at which each monitoring target appears in the captured image.
- the index value calculation unit 2020 counts the number of times each monitoring target is detected in the captured image, for example, every predetermined period. Then, the degree of change in the frequency at which the monitoring target is detected in the captured image is calculated from the number of monitoring target detections calculated for each predetermined period. Alternatively, the time interval between detections may be obtained, and the degree of change in the length of the detection time interval may be calculated.
- the index value calculation unit 2020 treats the degree of congestion of the monitoring target as the state of the monitoring target.
- the degree of congestion of the monitoring target is a degree of crowding of people, which is also called a degree of congestion.
- the surveillance staff watching the video of the surveillance camera can immediately grasp such a situation.
- the image processing apparatus 2000 presents a display based on the degree of change in the degree of congestion of the monitoring target on the presentation target image, so that the monitor who has viewed the presentation target image selects a monitoring target whose congestion is not easily resolved over time. It can be recognized immediately.
- the degree of congestion of the monitoring target can be expressed by using, for example, the size of the monitoring target and the number of objects included in the monitoring target.
- the size of the monitoring target can be expressed by the size of the area representing the monitoring target.
- the method of expressing the size of the area is as described above.
- the index value calculation unit 2020 calculates the degree of congestion of the monitoring target using Expression (1).
- d is the degree of congestion
- n is the number of objects included in the monitoring target
- a is the area of the region representing the monitoring target.
- n may be the number of objects calculated by individually counting the objects, or may be the number of objects estimated by collectively recognizing a mass of a plurality of objects.
- the index value calculation unit 2020 calculates the degree of change in the degree of congestion by, for example, calculating the above-described degree of congestion every predetermined period.
- the index value calculation unit 2020 treats the length of the monitoring target matrix or the speed of advance as the monitoring target state. For example, in a store having a plurality of cash registers, it is assumed that only the length of a queue at a certain cash register does not change for a long time (the speed at which the queue advances is slow). In this case, there may be some trouble at the cash register.
- the index value calculation unit 2020 calculates an index value based on the length of the matrix to be monitored or the degree of change in the traveling speed.
- the length of the monitoring target matrix may be represented by the size of the area representing the matrix, or may be represented by the number of objects included in the monitoring target matrix.
- the size of the region representing the matrix is represented by the length of the side or the diameter of the region.
- the index value calculation unit 2020 calculates the direction of the matrix from the direction in which the length of the matrix changes, the direction of the object included in the matrix, and the like, and the length of the side or diameter in that direction in the region representing the matrix Is used to represent “the length of the region representing the matrix”.
- the direction of the matrix may be given in advance in association with the camera 3000.
- the direction of the matrix can be determined in advance.
- the index value calculation unit 2020 calculates the speed at which the matrix advances from the degree of change in the matrix length.
- the speed of the matrix can be calculated by tracking a specific object in the matrix.
- the index value calculation unit 2020 monitors a person or a baggage that is captured in a captured image obtained by capturing a platform of a station or the like. Then, the index value calculation unit 2020 determines the degree of change in the number of people and luggage, the degree of change in the length of the person and luggage matrix (the degree to which people and luggage are left unloaded), and the degree of change in the state of the monitoring target. Calculate as
- FIG. 7 is a diagram illustrating a state in which people are left behind.
- FIG. 7A shows a captured image 10-1 obtained by imaging the state immediately after the train door is opened
- FIG. 7B shows a captured image 10-2 obtained by imaging the state immediately before the train door is closed. Is shown. Comparing the two, in front of the door on the front side, many people are left behind without getting on the train, while there are no people left behind at the back door. In this way, when the degree of unloading varies greatly depending on the boarding position, there may be a possibility that some trouble has occurred in the platform or the train.
- the presentation unit 2040 presents a display based on the degree of unloading on the presentation target image, so that a monitoring person who views the presentation target image can immediately grasp the degree of unloading of people, luggage, and the like.
- the index value calculation unit 2020 not only detects the state of the monitoring target directly obtained by analyzing the captured image, such as the position of the monitoring target described above, but also the state of the monitoring target based on an index obtained by applying the state to a model or the like. May be determined.
- the index value calculation unit 2020 treats the degree of dissatisfaction of the monitoring target as the state of the monitoring target.
- the monitoring target is a crowd.
- the index value calculation unit 2020 calculates the degree of crowd dissatisfaction from the crowd congestion level and flow information. For example, crowds with high congestion and slow flow generally tend to be more dissatisfied.
- the dissatisfaction level of the crowd is modeled using a function ⁇ ⁇ F (u, v) using the congestion degree u and the flow speed ⁇ v.
- g (v) may be modeled by a function that increases as v increases to some extent.
- the speed of progress of each matrix is compared, and when the speed of progress of a certain matrix is smaller than the speed of progress of other matrices, the degree of dissatisfaction may be higher than the degree of dissatisfaction determined by the value of v. That is, if ⁇ v is the difference in speed from the adjacent column (the value of the adjacent column minus the speed of the own column), g2 (v, ⁇ v), which is a monotonic non-decreasing function with respect to ⁇ v May be modeled using instead of (g (v) ⁇ ⁇ ⁇ .
- This method can be applied to other than crowds.
- the crowd may be modeled to increase dissatisfaction. That is, if the slope of v is ⁇ v, the influence of the flow speed may be modeled by g2 (v, ⁇ v).
- the degree of dissatisfaction may be calculated according to the position in the column (how far away from the head) and the expected time to reach the head of the column. This is because it is considered that the closer to the head, the closer to the end of the actions arranged in a row, the higher the degree that dissatisfaction can be tolerated.
- these functions may change depending on other external factors.
- Other external factors include temperature, humidity, weather, and brightness.
- the degree of dissatisfaction is likely to increase when the temperature is too high or too low compared to when the temperature is appropriate. Therefore, a model in which the degree of dissatisfaction is low when the temperature is appropriate and the degree of dissatisfaction is high when the temperature deviates from the appropriate temperature may be used.
- the degree of dissatisfaction is likely to increase more than in the case of fine weather. Therefore, a model in which dissatisfaction is higher when it is raining than when it is sunny.
- winning or losing of the game can be an external factor. For example, if the people who make up the crowd are losing the game or supporting the losing team, they are modeled to be more dissatisfied.
- the index value calculation unit 2020 determines how much damage is caused to a monitoring target when an incident occurs around the monitoring target (for example, when a suspicious object explodes or a person with a weapon appears). That is, the degree of danger around the monitoring target is calculated as the risk level of the monitoring target.
- Such a risk depends on the structural characteristics of the building and the state of the crowd, and it is possible to generate a model for calculating the risk by simulating the behavior of the crowd for various crowd states in advance. it can.
- the index value calculation unit 2020 calculates the risk level of the place by applying the feature amount of the state (density or flow) of the crowd at a place actually captured in the captured image to the model.
- the place where the crowd exists can be specified by the camera 3000 that captures the captured image showing the crowd. For example, when the field of view of the camera 3000 is fixed or the range in which the field of view changes is narrow, the location where the monitoring target appears in a captured image is uniquely identified by the ID of the camera 3000 that captured the captured image it can.
- the position where the monitoring target shown in a certain captured image exists is specified based on, for example, the ID of the camera 3000 and the orientation of the camera 3000 at the time of imaging. it can.
- the index value calculation unit 2020 may calculate the degree of risk in consideration of the characteristics of the crowd to be monitored.
- the index value calculation unit 2020 uses a model that calculates a high risk level for a crowd that takes a long time to move (for example, a crowd of elderly people or a crowd of walkers).
- the index value calculation unit 2020 may use a model that calculates the degree of risk in consideration of external factors such as weather.
- the index value calculation unit 2020 uses a model in which the degree of danger increases when the weather is bad and the ambient light is weak or the floor is wet due to rain.
- a crowd attribute can be acquired separately, such as an elderly person, a child, or a walking restrictor, the risk level may be calculated in consideration of the attribute information.
- the image processing apparatus 2000 may set the monitoring target state to a degree that the monitoring target is not monitored (hereinafter referred to as monitoring thinness).
- monitoring thinness a degree that the monitoring target is not monitored.
- a security guard at the scene may have to be in charge of a wide area alone, or may have to respond to visitors while guarding. Therefore, the degree to which the monitoring target is monitored by the guards varies.
- the index value calculation unit 2020 treats the monitoring level of the monitoring target as the status of the monitoring target.
- the monitoring thinness can be calculated by the distance between the monitoring target and a guard in the vicinity of the monitoring target. Specifically, the longer the distance between the monitoring target and the guard, the greater the monitoring thinness.
- surveillance thinness can be modeled by a monotonic non-decreasing function f (d) that increases as the distance d from the guard increases.
- the monitoring thinness may be modeled in consideration of the direction of the guard.
- the monitoring thinness is the distance d and the angle of deviation between the direction in which the guard is facing and the direction in which the monitoring target is located (a vector representing the direction from the position of the guard to the monitoring target and the guard).
- This is modeled by a function f (d, ⁇ ) that is determined by the absolute value ⁇ of the angle representing the direction of the member.
- f (d, ⁇ ) is a monotonic non-decreasing function with respect to ⁇ .
- the degree of the security guards focusing on security may be used for calculating the monitoring thinness.
- the degree of security focus is determined by, for example, the state or posture of the security guard. For example, when a guard who should look around and look up is facing down or up, it can be said that the security focus of the guard is low. Moreover, even if the posture of the security guard is facing the front, if the security guard is performing an action other than security, it can be said that the security focus of the security guard is low.
- the operations other than security include, for example, an operation of responding to a customer, an operation of contacting with a mobile phone, or an operation of installing a pole.
- the index value calculation unit 2020 grasps the state and posture of the guard.
- the index value calculation unit 2020 analyzes the state and posture of the guard shown in the captured image.
- the index value calculation unit 2020 may grasp the posture of the guard by acquiring the posture information of the portable terminal from the portable terminal held by the guard.
- the attitude information of the mobile terminal is information regarding acceleration in each of the three-dimensional directions measured by an acceleration sensor included in the mobile terminal, for example.
- the index value calculation unit 2020 calculates a security focus indicating a value of 0 or more and 1 or less, for example, according to the state of the guard described above. Then, the index value calculation unit 2020 calculates the monitoring thinness using a model such as f (d, ⁇ ) ⁇ described above, and further multiplies the value by the guarding strength of the guard, thereby obtaining the final monitoring thinness. Calculate the degree.
- the index value calculation unit 2020 may calculate the monitoring thinness in consideration of the above-described risk. Specifically, since it can be said that the higher the degree of risk, the more the monitoring target is to be monitored, even if the monitoring thinness calculated by the above-described method is the same, the monitoring target with the higher risk level is the final. Increase the monitoring thinness calculated in (1). For example, the index value calculation unit 2020 calculates a risk level and a monitoring thinness level for a certain monitoring target by the above-described method, and sets a value obtained by multiplying them as a monitoring thinness level to be finally calculated.
- the index value calculation unit 2020 may calculate the monitoring thinness for a certain monitoring target using the monitoring thinness calculated for each guard. For example, the index value calculation unit 2020 calculates the monitoring thinness for a certain monitoring target as a statistical value (minimum value, maximum value, average value, etc.) of the monitoring thinness for the monitoring target calculated for each guard.
- the index value calculation unit 2020 may use the above-described guarding degree as the value of monitoring thinness.
- FIG. 8 is a block diagram illustrating an image processing apparatus 2000 according to the second embodiment.
- arrows indicate the flow of information.
- each block represents a functional unit configuration, not a hardware unit configuration.
- the image processing apparatus 2000 includes a display color determination unit 2060.
- the display color determination unit 2060 determines a display color corresponding to the index value calculated for the monitoring target.
- the presentation unit 2040 then changes the color of the monitoring target or the surrounding color of the monitoring target to the display color determined for the monitoring target in the presentation target image.
- the display color determination unit 2060 determines the display color of the monitoring target by changing the color intensity of the monitoring target according to the magnitude of the index value of the monitoring target. For example, the display color determination unit 2060 increases the color of the monitoring target as the index value increases. Conversely, the display color determination unit 2060 may make the color to be monitored darker as the index value is smaller.
- the display color determination unit 2060 represents the monitoring target with one type of color, and determines the display color of the monitoring target by determining the color intensity based on the size of the index value. .
- the display color determination unit 2060 sets the display color of the monitoring target to black having a darkness corresponding to the index value of the monitoring target.
- FIG. 9 is a diagram illustrating a color map in which black is darkened based on the size of the index value. The color map in FIG. 9 indicates that the larger the size of the point (the more toward the right side), the darker the black.
- the display color determination unit 2060 may represent the display color to be monitored using any one of RGB colors, and may determine the color intensity according to the size of the index value. . For example, the display color determination unit 2060 sets the display color of the monitoring target to red, and darkens the display color red as the monitoring target index value increases.
- the display color determination unit 2060 uses a specific color map, determines a color corresponding to the index value of the monitoring target from the color map, and sets the color as the display color of the monitoring target.
- a color map to be used is a rainbow color map used for a heat map or the like.
- a typical rainbow color map is composed of gradations of red, orange, yellow, green, blue, indigo, and purple, as shown in FIG. In FIG. 10, the index values are red, orange, yellow, green, blue, indigo, and purple in descending order.
- the color map used by the display color determination unit 2060 is not limited to the color map shown in FIG.
- the display color determination unit 2060 can use an arbitrary color map.
- the color map used by the display color determination unit 2060 is stored in a storage unit provided inside or outside the image processing apparatus 2000.
- the presentation unit 2040 may change only some colors of the monitoring target, not the color of the entire monitoring target. For example, when the monitoring target is a person, the presentation unit 2040 changes only the face color of the monitoring target.
- FIG. 11 is a diagram conceptually illustrating a state in which the color of the monitoring target and the surrounding color are changed to a color corresponding to the index value indicating the degree of change in the position of the monitoring target.
- the captured images 10-1 and 10-2 shown in FIG. 11 are images obtained by capturing the same passage at different times.
- a captured image 10-1 illustrated in FIG. 11A is an image captured before the captured image 10-2 illustrated in FIG.
- the captured images 10-1 and 10-2 are compared, the position of the person 20 is not substantially changed, and the positions of the other persons are largely changed.
- the person who stays is a person who should be carefully monitored.
- the display color determination unit 2060 determines the display color so that the monitoring target (person) with the smaller index value becomes darker. Then, the presentation unit 2040 changes the monitoring target and its surroundings to the determined display color in the captured image 10-2. As a result, the color of the person 20 and the surrounding area is dark, and the other persons and the surrounding area are light colors. In FIG. 11, as in the case of FIG. 9, the larger the dot size, the darker the color. Moreover, the arrow in FIG.11 (b) is drawn in order to demonstrate that the person is moving, and it is not necessary to draw an arrow on an actual captured image.
- the display color of the captured image is determined based on the degree of change in the state of the monitoring target, and a display using the display color is presented on the presentation target image. Therefore, according to the image processing apparatus 2000 of the present embodiment, it is possible to intuitively grasp the degree of change in the state of the monitoring target as compared with the method of displaying the index value as it is on the presentation target image. Therefore, it becomes easier for a monitor who views the presentation target image to grasp the current status of the monitoring target.
- the configuration of the image processing apparatus 2000 according to the third embodiment has the same configuration as that of the image processing apparatus 2000 according to the first or second embodiment.
- the presentation unit 2040 of the third embodiment presents a display that emphasizes the monitoring target on the presentation target image based on the index value of the monitoring target.
- the presentation unit 2040 presents a display that emphasizes a monitoring target with a larger index value or a display that emphasizes a monitoring target with a smaller index value on the presentation target image.
- the presentation unit 2040 displays a frame line having a thickness corresponding to the size of the index value around the monitoring target.
- the presentation unit 2040 calculates the thickness b of the frame line using the following equation (2).
- b0 is the initial thickness value.
- I is an index value calculated by the index value calculation unit 2020.
- ⁇ is a proportionality constant.
- the shape of the frame line presented by the presentation unit 2040 is arbitrary.
- the presentation unit 2040 thickens the frame line as the monitoring target index value increases.
- b0 is the lower limit value of thickness
- ⁇ is a positive real number.
- the presentation unit 2040 thickens the frame line as the monitoring target index value is smaller.
- b0 is the upper limit value of thickness
- ⁇ is a negative real number.
- the presentation unit 2040 may change the thickness of the outline of the monitoring target by the same method as the method of displaying a frame line around the monitoring target. Specifically, the presentation unit 2040 draws a contour to be monitored.
- the presentation unit 2040 may perform highlighting by blinking a frame line at a frequency according to the index value to be monitored. For example, when emphasizing a monitoring target with a larger index value, the presentation unit 2040 increases the number of blinks per unit time (shortens the blinking interval) as the frame line is presented for a monitoring target with a larger index value. Similarly, when emphasizing a monitoring target with a smaller index value, the presentation unit 2040 increases the number of blinks per unit time (shortens the blinking interval) as the frame line is presented for a monitoring target with a smaller index value.
- FIG. 12 is a diagram conceptually illustrating a state in which highlighting is performed by presenting a frame line around the monitoring target.
- the captured images 10-1 and 10-2 shown in FIG. 12 are images obtained by capturing a matrix of people at the same place at different times.
- the captured image 10-1 is an image captured before the captured image 10-2.
- the length of the upper matrix 30-1 does not change, and the length of the lower matrix 30-2 changes greatly.
- the presentation unit 2040 displays a frame line around the monitoring target so that the monitoring target (person) with the smaller index value has a thicker frame line.
- a thick frame line is displayed around the matrix 30-1
- a thin frame line is displayed around the matrix 30-2.
- the image processing apparatus 2000 uses the display color determination unit 2060 described in the second embodiment, and changes the color to be monitored or the surrounding color of the monitoring target to the display color determined for the monitoring target.
- a display that emphasizes the monitoring target may be presented.
- the index value calculation unit 2020 emphasizes the monitoring target by increasing the darkness of the display color of the monitoring target.
- the display color determination unit 2060 configures the display color of the monitoring target using a color map configured with colors that are more conspicuous as the color corresponding to the index value of the monitoring target to be emphasized.
- the display color determination unit 2060 has a color that is more conspicuous (such as red) as the color corresponding to the larger index value, and the color that is less conspicuous as the color corresponding to the smaller index value (gray). Etc.) is used.
- the presentation unit 2040 may make the thickness of the frame line constant regardless of the index value, or may vary it according to the index value.
- the method of determining the thickness of the frame line according to the index value is as described above.
- FIG. 13 is a diagram conceptually illustrating a state in which highlighting is performed by presenting a frame line having a color and thickness corresponding to an index value around a monitoring target.
- Captured images 10-1 and 10-2 shown in FIG. 13 are images obtained by capturing a crowd at the same place at different times. As in the case of FIGS. 11 and 12, the captured image 10-1 is an image captured before the captured image 10-2. When the captured images 10-1 and 10-2 are compared, the number of people included in the upper right crowd 40-1 increases, and the number of people included in the lower left crowd 40-2 decreases.
- the display color determining unit 2060 determines the display color so that the crowd with a larger increase in the number of people becomes darker.
- the presentation unit 2040 determines the thickness of the frame line so that the thickness of the frame line increases as the number of people increases. As a result, the presentation unit 2040 presents a thick and thick frame line around the crowd 40-1 where the increase in the number of people is large, and presents a thin thin frame line around the crowd 40-2 where the increase in the number of people is small. .
- a display that emphasizes the monitoring target to the extent based on the index value of the monitoring target is presented on the presentation target image. Therefore, a monitoring person who views the presentation target image can immediately grasp the degree of change of each monitoring target, and can quickly grasp how much each monitoring target should be monitored carefully.
- FIG. 14 is a block diagram illustrating an image processing apparatus 2000 according to the fourth embodiment.
- arrows indicate the flow of information.
- each block represents a functional unit configuration, not a hardware unit configuration.
- the image processing apparatus 2000 according to the fourth embodiment presents a display on the first image based on how far the degree of change in the state of the monitoring target deviates from the reference degree of change.
- the image processing apparatus 2000 according to the fourth embodiment includes a divergence degree calculation unit 2080.
- the divergence degree calculation unit 2080 calculates the divergence degree between the index value calculated by the index value calculation unit 2020 and the reference change degree. Then, the presentation unit 2040 according to the fourth embodiment presents a display that emphasizes the monitoring target having a larger degree of deviation in the presentation target image.
- the divergence degree calculation unit 2080 acquires a reference change degree from a storage unit inside or outside the image processing apparatus 2000.
- the reference degree of change may differ depending on what is treated as the state of the monitoring target.
- storage part memorize
- the divergence calculation unit 2080 calculates the divergence.
- the divergence degree calculation unit 2080 calculates the divergence degree k using the following equation (3).
- I is an index value calculated for the monitoring target, and I base is a standard change degree.
- the method of calculating the degree of divergence is not limited to the following method.
- the image processing apparatus 2000 according to the fourth embodiment changes the color to be monitored based on the degree of deviation.
- the image processing apparatus 2000 according to the fourth embodiment includes a display color determination unit 2060.
- the display color determination unit 2060 determines the color to be monitored and its surroundings in the same manner as the display color determination unit 2060 described in the second embodiment. For example, the display color determination unit 2060 determines the color density of the monitoring target based on the degree of deviation calculated for the monitoring target. In this case, the display color determination unit 2060 minimizes the density when the divergence degree is 0, and darkens the color to be monitored as the divergence degree increases.
- the index value can be a negative value
- the divergence degree is represented by the absolute value of the divergence between the index value and the reference value.
- the degree of divergence is represented by the absolute value of the value calculated by Expression (3).
- the display color determination unit 2060 sets the color density of the monitoring target when the divergence degree is 0 as the reference density, and the color of the monitoring target increases as the divergence degree increases in the positive direction (greater than the reference value). As the degree of divergence increases in the negative direction (becomes smaller than the reference value), the color to be monitored is lightened.
- FIG. 15 is a diagram conceptually illustrating a method for determining the darkness of the display color according to the degree of divergence when the reference darkness is determined. For example, the display color determination unit 2060 sets the color intensity corresponding to the reference change degree to the original color intensity of the monitoring target. That is, when the divergence degree is 0, the darkness of the color to be monitored is not changed.
- the display color determination unit 2060 makes the color to be monitored darker than the original color and the index value is smaller than the reference change degree. In this case (when the degree of divergence is negative), the color to be monitored is made lighter than the original color.
- the method of determining the display color according to the degree of divergence when using one of the RGB color densities described in the second embodiment or when using a specific color map is also used for the degree of divergence described above. Accordingly, the method is the same as the method of changing the color intensity of the monitoring target.
- the presentation unit 2040 may present a display that emphasizes the monitoring target based on the degree of deviation calculated for the monitoring target by the same method as that described in the third embodiment.
- the presentation unit 2040 performs highlighting using frame lines and colors, as in the third embodiment.
- the presentation unit 2040 determines the thickness b ′ of the monitoring target frame line according to Equation (4).
- k represents the above-mentioned degree of divergence. For example, if ⁇ is a positive real number, the greater the degree of divergence, the thicker the border.
- the presentation unit 2040 changes the thickness of the outline of the monitoring target according to the degree of divergence or blinks the frame line at a frequency according to the degree of divergence. You may go.
- the display color determination unit 2060 of the fourth embodiment may present a display that emphasizes the monitoring target by changing the display color of the monitoring target.
- the display color determination unit 2060 is configured to increase the depth of the display color as the degree of divergence increases, or to make the color corresponding to the degree of divergence more conspicuous when it is desired to emphasize a monitoring target with a larger degree of divergence.
- the display color is determined using the color map.
- the display color determination unit 2060 is configured to increase the darkness of the display color as the degree of divergence decreases, or to make the color corresponding to the smaller degree of divergence stand out when it is desired to emphasize a monitoring target with a lower degree of divergence.
- the display color is determined using the color map.
- the display for emphasizing the monitoring target is presented on the presentation target image based on how far the degree of change in the state of the monitoring target deviates from the reference degree of change.
- FIG. 5 The configuration of the image processing apparatus 2000 according to the fourth embodiment is illustrated in FIG.
- the index value calculation unit 2020 of the fourth embodiment is based on the calculated degree of change in the state of the monitoring target, and the degree of change in the state of the monitoring target after each captured image used for the calculation is captured. The predicted value of is calculated. Then, the index value calculation unit 2020 uses the predicted value calculated for the monitoring target as the index value of the monitoring target.
- the index value calculation unit 2020 calculates a predicted value of the degree of change in the state of the monitoring target after a predetermined time after the time point t using a plurality of captured images taken in a past predetermined time period from the time point t. . Then, a display based on the predicted value is presented on the presentation target image presented on the display screen at time t.
- the index value calculation unit 2020 generates a model for predicting the state of the monitoring target using the plurality of acquired captured images. Note that the method for generating the prediction model from the sampled values is a known method, and the details are omitted. Then, the index value calculation unit 2020 uses the model that predicts the state of the monitoring target generated from the plurality of acquired captured images, and the state of the monitoring target after the captured image used for generating the model is captured. The predicted value of the degree of change is calculated.
- the index value calculation unit 2020 calculates the degree of change in the state of the monitoring target from a certain time point t1 to a future time point t2 by the following equation (5).
- a represents the predicted value of the degree of change in the state of the monitored object.
- the following formula (5) is merely an example, and the method for calculating the predicted value is not limited to the method using formula (5).
- t1t may be at a certain time in the future, at the present time, or at a certain time in the past. If t1 is the current time or a past time, the value of f (t) may be calculated based on the actually measured value instead of calculating using the prediction model.
- the index value calculation unit 2020 may calculate a predicted value of the degree of change in the state of the monitoring target using a prediction model prepared in advance. In this case, the index value calculation unit 2020 uses the state of each monitoring target in the acquired captured image as an input to the prediction model. This prediction model is stored inside or outside the index value calculation unit 2020.
- the index value calculation unit 2020 presents a display based on the predicted value on a captured image captured by a certain camera 3000-1, the captured image captured by another camera 3000 around the camera 3000-1.
- An image may be used.
- the index value calculation unit 2020 analyzes the captured image captured by the camera 3000-2 adjacent to the camera 3000-1.
- the presentation unit 2040 displays the image based on the predicted value of the degree of change of the state calculated for the crowd in the region where the crowd is predicted to move in the captured image captured by the camera 3000-1.
- the presentation unit 2040 changes the color of an area of the captured image captured by the camera 3000-1 and predicted that a crowd flows in, or changes the area to the border line. The process to enclose with.
- the display based on the predicted value of the degree of change in the state of the monitoring target is presented on the presentation target image. For this reason, the monitor or the like can immediately grasp a monitoring target that should be closely watched for future actions.
- a captured image captured by a certain camera 3000 (hereinafter referred to as camera 3000-1) has a period displayed on the display screen 4000 and a period not displayed. For example, this corresponds to a case where captured images captured by a plurality of cameras 3000 are displayed on a single display screen 4000 in a time-sharing manner.
- the index value calculation unit 2020 of Embodiment 6 does not display the captured image captured by the camera 3000-1 on the display screen 4000 for a certain period but displays it on the display screen 4000 after that period. Based on the degree of change in the state of the monitoring target before and after, the index value of the monitoring target is calculated.
- FIG. 16 is a diagram conceptually illustrating how the display screen 4000 displays captured images captured by the plurality of cameras 3000 in a time-sharing manner.
- the captured image captured by the camera 3000-1 is displayed in the periods p1 and p3, and the captured image captured by another camera 3000-2 is displayed in the period p2.
- the index value calculation unit 2020 calculates the index value based on the degree of change between the state of the monitoring target before the period p2 and the state of the monitoring target after the time p2.
- a period in which the captured image captured by the camera 3000-1 is not displayed on the display screen 4000 (such as p2 in FIG. 16) is referred to as a non-display period.
- the index value calculation unit 2020 uses a predetermined number of captured images presented on the display screen 4000 before the non-display period and a predetermined time (predetermined number) of captured images presented on the display screen 4000 after the non-display period. Thus, an index value used for presentation after the non-display period has elapsed is calculated.
- FIG. 17 is a diagram for explaining a method by which the index value calculation unit 2020 according to the sixth embodiment calculates an index value.
- the periods p1, p2, and p3 are the same as those in FIG.
- the index value calculation unit 2020 includes a captured image displayed on the display screen 4000 in a period p4 that is a part of the period p1 and a captured image displayed on the display screen 4000 in p5 that is a part of the period p3. Use to calculate the index value of the monitoring target. Then, the presentation unit 2040 presents a display based on the calculated index value on the presentation target image displayed on the display screen 4000 at the time point t. Note that the length of the period p4 and the length of the period p5 may be the same or different.
- the presentation unit 2040 is configured so that a monitoring person or the like can sufficiently grasp the degree of change in the state of the monitoring target before and after the period p2, between a period p4 and p5 for a predetermined period (for example, 10 seconds) from the point
- the display based on the degree of change in the state of the monitoring target is continuously presented on the captured image.
- FIG. 18 is a flowchart for explaining the flow of processing executed by the image processing apparatus 2000 according to the sixth embodiment.
- the display screen 4000 displays a captured image captured by the camera 3000-1.
- the display target of the display screen 4000 is switched from the camera 3000-1 to the camera 3000-2.
- the display screen 4000 displays a captured image captured by the camera 3000-2.
- the display target of the display screen 4000 is switched from the camera 3000-2 to the camera 3000-1.
- step S210 the index value calculation unit 2020 calculates an index value indicating the degree of change between the monitoring target state displayed in S202 and the monitoring target state to be displayed.
- step S212 the index value calculation unit 2020 presents a display based on the calculated index value on the captured image captured by the camera 3000-1. The captured image presented with this display is displayed on the display screen 4000.
- the index value calculation unit 2020 determines that “the captured image captured by the camera 3000-1 is displayed on the display screen when the period during which the captured image captured by the camera 3000-1 is not displayed on the display screen 4000 is shorter than a predetermined time. It may continue to be displayed at 4000 ”. For example, if the display target camera is simply switched to another camera for a short time of about 1 second, it can be considered that there is no problem even if it is considered that the monitor has continued to watch the video of the same camera.
- the monitor or the like switches the channel of the display screen 4000 from the video of the camera 3000-1 to the video of the camera 3000-2, for example, and after a while, the channel of the display screen 4000 is changed to the video of the camera 3000-1 again. At the time of switching, it is possible to immediately grasp how much the state of each monitoring target has changed compared to the time when the image of the camera 3000-1 was last viewed.
- FIG. 7 The image processing apparatus 2000 according to the seventh embodiment is represented in FIG. 1 like the image processing apparatus 2000 according to the first embodiment.
- the index value calculation unit 2020 of the seventh embodiment is configured so that the first partial area is in the line-of-sight direction of the user (such as a monitor) with respect to the monitoring target displayed in a certain partial area (hereinafter referred to as the first partial area) of the display screen 4000.
- An index value indicating the degree of change in the state of the monitoring target before and after a period that is not supported is calculated.
- the presentation part 2040 of Embodiment 7 presents the display based on the calculated index value on the area displayed in the first partial area in the captured image displayed after the period.
- FIG. 19 is a diagram illustrating the relationship between the user's line-of-sight direction and the partial area.
- the line-of-sight direction 50-1 is the line-of-sight direction corresponding to the partial area 60-1
- the line-of-sight direction 50-2 is the line-of-sight direction corresponding to the partial area 60-2.
- the line-of-sight direction corresponding to the partial area is represented by one arrow, but actually, the line-of-sight direction corresponding to the partial area has a certain width.
- the line-of-sight direction 50-1 may be a line-of-sight direction in which the partial area 60-1 enters the user's field of view so that the user can watch the monitoring target included in the partial area 60-1.
- the basic principle of the process performed by the index value calculation unit 2020 of the seventh embodiment is the same as the principle of the process performed by the index value calculation unit 2020 of the sixth embodiment.
- the index value calculation unit 2020 displays the “period in which the partial region is included in the region corresponding to the user's line-of-sight direction” as “the image captured by the camera 3000-1 is displayed on the display screen 4000 in the sixth embodiment. It is treated in the same way as “Displayed period”. Further, the index value calculation unit 2020 sets the “period in which the partial region is not included in the user's line-of-sight direction” in the same manner as the “period in which the image captured by the camera 3000-1 is not displayed on the display screen 4000”. To deal with.
- the index value calculation unit 2020 acquires the user's line-of-sight direction.
- the line-of-sight direction is represented by a combination of “an angle in the horizontal direction and an angle in the vertical direction”, for example.
- the reference of the angle in the horizontal direction and the angle in the vertical direction is arbitrary.
- the user's line-of-sight direction is calculated by capturing the user's face and eyes with a camera or the like and analyzing the captured image.
- a camera that captures the user's face and eyes is installed in the vicinity of the display screen 4000, for example. Since the technique for capturing the user's face and eyes and detecting the line-of-sight direction is a known technique, a detailed description thereof will be omitted.
- a processing unit that detects the user's line-of-sight direction (hereinafter referred to as a line-of-sight direction detection unit) may be provided inside the image processing apparatus 2000 or may be provided outside.
- the index value calculation unit 2020 handles the display screen 4000 by dividing it into a predetermined number of partial areas in advance. Then, the index value calculation unit 2020 acquires the gaze direction of the monitoring person from the gaze direction detection unit, and determines which partial region the gaze direction corresponds to. Then, when the determined partial area is different from the previously calculated partial area, it is understood that the partial area corresponding to the user's line-of-sight direction has changed.
- FIG. 20 is a diagram illustrating information associating a partial area corresponding to the observer's line-of-sight direction with information when the observer's line-of-sight direction has changed in a table format.
- This table is referred to as a line-of-sight information table 100.
- the line-of-sight information table 100 has two columns of a time point 102 and a partial area ID 104. Each record of the line-of-sight information table 100 indicates the partial area ID included in the user's line-of-sight direction from the time indicated by the time 102 as a partial area ID 104.
- the region corresponding to the observer's line-of-sight direction is partial region 1 at time points t1 and t4. Therefore, the index value calculation unit 2020 monitors the state of the monitoring target in the period between the time points t1 and t2 (the period in which the partial area 1 corresponds to the user's line-of-sight direction), and after the time point t4 (again, the partial area 1 An index value indicating the degree of change with the state of the monitoring target after the point of time corresponding to the user's line-of-sight direction is calculated as the monitoring target index value.
- the index value calculation unit 2020 may consider that “the user continued to look at the same partial area” when the period in which the user's line-of-sight direction was changed to another partial area was shorter than a predetermined time. For example, it is considered that there is no problem even if it is considered that “the supervisor has continued to look at the partial area” if the supervisor has only taken a short time of about 1 second from a certain partial area.
- the index value calculation unit 2020 may use the orientation of the user's face instead of the user's gaze direction.
- the method of acquiring the direction of the user's face or using the direction of the user's face is the same as the method of detecting the user's line-of-sight direction or using the user's line-of-sight direction.
- the display indicating the degree of change in the state of each monitored object since the last time the area was viewed Is presented on the display screen 4000. Therefore, even if the monitor or the like cannot monitor all the areas of the display screen 4000 at a time, the monitor can immediately grasp the degree of change in the state of the monitoring target reflected in each area.
- An image processing apparatus 2000 of Modification 7-1 shown below may be realized with the same configuration as that of the image processing apparatus 2000 of the seventh embodiment.
- the display screen 4000 includes a plurality of small screens 4100. Each small screen 4100 displays captured images captured by different cameras 3000.
- the index value calculation unit 2020 of the modified example 7-1 includes a period in which the small screen 4100-1 is not included in the region corresponding to the user's line-of-sight direction for the monitoring target displayed on the small screen 4100-1. An index value indicating the degree of change in the state of the monitoring target before and after is calculated. Then, the presentation unit 2040 of Modification 7-1 presents a display based on the calculated index value on the captured image displayed on the small screen 4100-1 after the above period.
- the small screen 4100 can be handled in the same manner as the partial area in the seventh embodiment. Therefore, the basic principle of the process performed by the index value calculation unit 2020 of Modification 7-1 is the same as the principle of the process performed by the index value calculation unit 2020 of the seventh embodiment. Specifically, the index value calculation unit 2020 displays the “period in which the small screen 4100-1 is included in the user's line-of-sight direction” and the “period in which the partial area is included in the user's line-of-sight direction” in the seventh embodiment. ”.
- the index value calculation unit 2020 handles the “period in which the small screen 4100-1 is not seen by the monitoring staff” in the same manner as the “period in which the partial area is not included in the user's line-of-sight direction” in the seventh embodiment.
- Index value calculation means for calculating an index value indicating the degree of change in the state of the monitoring target shown in the captured image, using a plurality of captured images captured at different times by the camera; Presenting means for presenting a display based on the index value on a first captured image captured by the camera; An image processing apparatus.
- a first display color determining means for determining a display color corresponding to the index value for the monitoring target; The presenting means changes the color of the monitoring target or the surrounding color of the monitoring target to the display color determined for the monitoring target in the first captured image.
- the presenting means presents a display that emphasizes a monitoring target with a larger index value, or presents a display that emphasizes a monitoring target with a smaller index value. Or 2.
- An image processing apparatus according to 1. 4).
- a deviation degree calculating means for calculating a deviation degree between the index value and a reference change degree; In the first captured image, the presenting unit presents a display that emphasizes a monitoring target with a larger degree of deviation, or presents a display that emphasizes a monitoring target with a smaller degree of deviation.
- a second display color determining means for determining a display color corresponding to the degree of divergence calculated for the monitoring target with respect to the monitoring target;
- the presenting means changes the color of the monitoring target or the surrounding color of the monitoring target to the display color determined for the monitoring target in the first captured image.
- An image processing apparatus uses the calculated degree of change of the state of the monitoring target, and predicts the degree of change of the state of the monitoring target after each captured image used for the calculation is captured. And its predicted value is used as the index value.
- the image processing apparatus according to any one of the above. 7).
- the index value calculation unit displays the state of the monitoring target displayed before that period and the display after that period. Calculating the index value indicating the degree of change between the monitored state and The presenting means uses a captured image displayed after the period as the first captured image.
- the index value calculation means Calculating an index value indicating a degree of change between the state of the monitoring target displayed in the first partial area before and the state of the monitoring target displayed in the first partial area after that period.
- the presenting means uses the captured image displayed after the period as the first captured image, and the first partial region is displayed on the region displayed in the first partial region of the first captured image. 1. Present a display based on the index value calculated for 1. To 7.
- the image processing apparatus according to any one of the above. 9.
- the index value calculation means calculates an index value indicating a degree of change in the position of the monitoring target. To 8. The image processing apparatus according to any one of the above. 10. The index value calculation means calculates an index value indicating the degree of change in the frequency at which the monitoring target appears in the image. To 9. The image processing apparatus according to any one of the above. 11. The index value calculation means calculates an index value indicating a degree of change in the degree of congestion of a plurality of objects included in the monitoring target. To 10. The image processing apparatus according to any one of the above. 12 The monitoring target includes a matrix of objects, The index value calculation means calculates an index value indicating the degree of change in the length or speed of the matrix. To 11. The image processing apparatus according to any one of the above. 13.
- the index value calculation means calculates an index value indicating a degree of change in the number of objects included in the monitoring target.
- the monitoring target includes a person, The index value calculation means calculates an index value indicating a degree of change in the degree of dissatisfaction of the monitoring target as the index value of the monitoring target.
- Thru 13 The image processing apparatus according to any one of the above.
- the monitoring target includes a person or a place, The index value calculation means calculates an index value indicating a degree of change in the degree of risk of the monitoring target as the index value of the monitoring target.
- the image processing apparatus according to any one of the above. 16.
- the monitoring target includes a person or a place
- the index value calculation means calculates an index value indicating the degree of change in the degree of monitoring of the monitoring target as the index value of the monitoring target.
- the image processing apparatus according to any one of the above. 17.
- a monitoring system having the image processing device according to any one of The camera generates a plurality of captured images by capturing images at different times
- the said display screen is a monitoring system which displays the said 1st captured image by which the display based on the said index value was shown by the said presentation means. 18.
- An image processing method executed by a computer An index value calculating step for calculating an index value indicating a degree of change in a state of a monitoring target reflected in the captured image using a plurality of captured images captured at different times by the camera; A presenting step of presenting a display based on the index value on a first captured image captured by the camera; An image processing method. 19. A first display color determining step for determining a display color corresponding to the index value for the monitoring target; The presenting step changes the color of the monitoring target or the surrounding color of the monitoring target to the display color determined for the monitoring target in the first captured image. An image processing method described in 1. 20.
- the presenting step presents a display that emphasizes a monitoring target with a larger index value, or presents a display that emphasizes a monitoring target with a smaller index value. Or 19.
- the presenting step presents a display that emphasizes a monitoring target with a larger degree of deviation in the first captured image, or presents a display that emphasizes a monitoring target with a smaller degree of deviation.
- the presenting step changes the color of the monitoring target or the surrounding color of the monitoring target to the display color determined for the monitoring target in the first captured image.
- the index value calculating step uses the calculated degree of change in the state of the monitoring target, and predicts the degree of change in the state of the monitoring target after each captured image used for the calculation is captured. And the predicted value is used as the index value.
- Thru 22. The image processing method according to any one of the above. 24.
- the index value calculation step when a captured image is not displayed for a certain period on a display screen that displays a captured image captured by the camera, the state of the monitoring target displayed before that period and the display after that period Calculating the index value indicating the degree of change between the monitored state and
- the presenting step uses a captured image displayed after the period as the first captured image.
- the index value calculation step if the first partial area of the display screen that displays the captured image is not included in the screen area corresponding to the line of sight or the face direction of the user who views the display screen for a certain period, Calculating an index value indicating a degree of change between the state of the monitoring target displayed in the first partial area before and the state of the monitoring target displayed in the first partial area after that period.
- the presenting step uses the captured image displayed after the period as the first captured image, and the first partial region on the region displayed in the first partial region of the first captured image. 18. Present a display based on the index value calculated for To 24.
- the image processing method according to any one of the above. 26.
- the index value calculating step calculates an index value indicating a degree of change in the position of the monitoring target.
- the index value calculating step calculates an index value indicating the degree of change in the frequency at which the monitoring target is reflected in the image.
- the index value calculating step calculates an index value indicating a degree of change in the degree of congestion of a plurality of objects included in the monitoring target.
- Thru 27 The image processing method according to any one of the above. 29.
- the monitoring target includes a matrix of objects
- the index value calculating step calculates an index value indicating a degree of change in the length or speed of the matrix.
- the index value calculating step calculates an index value indicating a degree of change in the number of objects included in the monitoring target. Thru 29.
- the monitoring target includes a person,
- the index value calculating step calculates an index value indicating a degree of change in the degree of dissatisfaction of the monitoring target as the index value of the monitoring target.
- the monitoring target includes a person or a place,
- the index value calculation step calculates an index value indicating a degree of change in the degree of risk of the monitoring target as the index value of the monitoring target. Thru 31.
- the image processing method according to any one of the above. 33.
- the monitoring target includes a person or a place,
- the index value calculating step calculates an index value indicating a degree of change in the degree of monitoring of the monitoring target as the index value of the monitoring target.
- the image processing method according to any one of the above. 34.
- a program for operating as the image processing apparatus according to any one of the above.
- An image processing method executed by a computer A calculation step of calculating a degree of change in the state of the monitoring target reflected in the captured image using a plurality of captured images captured at different times by the camera; On the captured image captured by the camera, a presentation step of changing the color of the area representing the monitoring target to a color based on the degree of change; An image processing method. 37. 35. A program that operates as the image processing apparatus described in 1. 38. Calculation means for calculating the degree of change in the state of the monitoring target reflected in the captured image using a plurality of captured images captured at different times by the camera; Presenting means for emphasizing the monitoring target based on the degree of change on a captured image captured by the camera; An image processing apparatus. 39.
- An image processing method executed by a computer A calculation step of calculating a degree of change in the state of the monitoring target reflected in the captured image using a plurality of captured images captured at different times by the camera; On the captured image captured by the camera, a presentation step for emphasizing the monitoring target based on the degree of change; An image processing method. 40. Computer. A program that operates as the image processing apparatus described in 1.
Abstract
Description
前記画像処理装置は、前述した本願発明が提供する画像処理装置である。また、前記表示画面は、前記提示手段によって前記指標値に基づく表示が提示された前記第1撮像画像を表示する。
図1は、実施形態1に係る画像処理装置2000を示すブロック図である。図1において、矢印は情報の流れを表している。さらに、図1において、各ブロックは、ハードウエア単位の構成ではなく、機能単位の構成を表している。
画像処理装置2000の各機能構成部は、各機能構成部を実現するハードウエア構成要素(例:ハードワイヤードされた電子回路など)で実現されてもよいし、ハードウエア構成要素とソフトウエア構成要素との組み合わせ(例:電子回路とそれを制御するプログラムの組み合わせなど)で実現されてもよい。
図5は、実施形態1の画像処理装置2000が実行する処理の流れを例示するフローチャートである。ステップS102において、指標値算出部2020は、撮像画像を取得する。ステップS104において、提示部2040は、撮像画像に写っている監視対象の状態の変化度合いを示す指標値を算出する。ステップS106において、提示部2040は、カメラ3000によって撮像された撮像画像(提示対象画像)上に、指標値に基づく表示を提示する。
監視員等は、監視カメラに写っている監視対象の状態がどの程度変化しているかを把握したい場合、監視カメラの映像をしばらく見続けなければならない。短い時間、例えば1秒程度映像を見ただけでは、その時の監視対象の状態は把握できても、監視対象の状態がどの程度変化しているかを把握することは難しい。
指標値算出部2020が撮像画像を取得する方法は任意である。例えば指標値算出部2020は、カメラ3000から撮像画像を取得する。また、指標値算出部2020は、カメラ3000の外部にある記憶装置に記憶されている撮像画像を取得してもよい。この場合、カメラ3000は、撮像した画像をこの記憶装置に記憶するように構成されている。この記憶装置は、画像処理装置2000の内部に設けられてもよいし、画像処理装置2000の外部に設けられてもよい。
画像処理装置2000が扱う監視対象は様々である。例えば画像処理装置2000は、人や物などのオブジェクト、又はこれらオブジェクトの集合(群衆など)などを監視対象として扱う。なお、物を表すオブジェクトには、場所が含まれてもよい。つまり、画像処理装置2000は、撮像画像に写っている場所(領域)を監視対象として扱ってもよい。
画像処理装置2000は、撮像画像から抽出される全てのオブジェクトを監視対象としてもよいし、特定のオブジェクトのみを監視対象としてもよい。例えば画像処理装置2000は、人又は人の集合(群衆)のみを監視対象として扱う。また、画像処理装置2000は、特定の人や群衆のみを監視対象としてもよい。この場合、画像処理装置2000は、監視対象を示す情報(例えばブラックリスト)を取得し、その情報に基づいて監視対象を決定する。監視対象を示す情報は、例えば監視対象とする各オブジェクトの特徴量を示す。また監視対象を示す情報は、「帽子をかぶっている」や「サングラスをかけている」など、監視対象とする人の特徴を示す情報であってもよい。ここで、画像に含まれるオブジェクトから、特定の特徴を持つオブジェクトを特定する技術は既知の技術であるため、詳細な方法については省略する。
前述したように、提示部2040は、カメラ3000によって撮像された撮像画像(提示対象画像)上に、指標値に基づく表示を提示する。指標値に基づく表示を提示対象画像上に提示する処理は、例えば、監視対象について算出した指標値を、提示対象画像におけるその監視対象の周辺などに提示する処理である。「指標値に基づく表示を提示対象画像上に提示する処理」のその他の例は、後述の各実施形態等で説明する。
前述したように、指標値算出部2020は、撮像画像に写っている監視対象の状態の変化度合いを示す指標値を算出する。ここで、画像処理装置2000が扱う「監視対象の状態」は様々であり、指標値の算出方法は、何を監視対象として扱うかに依存する。そこで以下、画像処理装置2000が扱う監視対象の状態と、その監視対象の状態の変化度合いを示す指標値の算出方法とを例示する。
例えば指標値算出部2020は、監視対象の状態として、監視対象の位置を扱う。例えば、人が通過する通路において長時間立ち止まっている人物がいたら、その人物を注意して監視すべきであると考えられる。そこで、指標値算出部2020は、監視対象の位置の変化度合いを、監視対象の状態の変化度合いとして扱う。監視対象の位置の変化度合いは、監視対象の滞留度合いとも言い換えることができる。監視対象の滞留度合いに基づいて指標値を算出し、その指標値に基づく表示を提示対象撮像画像上に提示することで、監視員等は、各監視対象の滞留度合いをすぐに把握することができる。
また例えば、指標値算出部2020は、ある監視対象が撮像画像に写る頻度を、監視対象の状態として扱う。つまり、指標値算出部2020は、ある監視対象が撮像画像に写る頻度(撮像画像に写る時間の長さ)の変化度合いを、監視対象の状態の変化度合いとして扱う。例えばある監視対象が、最初の30分には撮像画像で一度も検知されず、次の30分には撮像画像で1回検知され、次の30分には撮像画像で5回検知されているとする。この場合、その監視対象が撮像画像に写る頻度は上昇している。そのため、監視対象の状態の変化度合いは、高い度合いを示す。
例えば指標値算出部2020は、監視対象の密集度合いを、監視対象の状態として扱う。例えば人を監視対象として扱う場合、監視対象の密集度合いは人が密集している度合いであり、混雑度合いとも言い換えられる。例えば、狭い通路に人が過度に密集すると、群衆雪崩などが発生する危険がある。このような場合、警備員が適切な誘導を行う等の措置が必要となるため、監視カメラの映像を見る監視員はこのような状況を即座に把握できることが好ましい。画像処理装置2000が監視対象の密集度合いの変化度合いに基づく表示を提示対象画像上に提示することで、提示対象画像を見た監視員は、時間が経ってもなかなか混雑が解消されない監視対象をすぐに見分けることができる。
例えば指標値算出部2020は、監視対象の行列の長さ又は進む速さを、監視対象の状態として扱う。例えば複数のレジがある店舗などにおいて、各レジにできる待ち行列の内、あるレジにできている待ち行列の長さだけが長時間変化しない(行列が進む速さが遅い)とする。この場合、そのレジで何かしらトラブルが起きていたりすることが考えられる。
例えば指標値算出部2020は、駅のプラットフォームなどを撮像した撮像画像に写っている人や荷物などを監視対象とする。そして、指標値算出部2020は、人や荷物等の数の変化度合いや人や荷物の行列の長さの変化度合い(人や荷物が積み残された度合い)を、監視対象の状態の変化度合いとして算出する。
指標値算出部2020は、前述した監視対象の位置など、撮像画像を解析して直接求まる監視対象の状態だけでなく、その状態をモデル等に適用して得られる指標に基づいて監視対象の状態を定めてもよい。
例えば指標値算出部2020は、監視対象について、その監視対象の周辺で何か事件が起こったとき(例えば不審物が爆発した場合や凶器を持った人物が出現した場合など)にどれだけ大きなダメージが生じ得るか、すなわちその監視対象の周辺の危険性の度合いを、その監視対象の危険度として算出する。
画像処理装置2000は、監視対象が監視されていない度合い(以下、監視手薄度)を、監視対象の状態としてもよい。ここで、カメラ3000が設置されている施設において、現場にいる警備員が監視を行っているとする。現場にいる警備員は、一人で広範囲を担当しなければならなかったり、警備中に来場者の応対をしなければならなかったりすることがある。そのため、監視対象が警備員に監視されている度合いには、ばらつきがある。
図8は、実施形態2に係る画像処理装置2000を示すブロック図である。図8において、矢印は情報の流れを表している。さらに、図8において、各ブロックは、ハードウエア単位の構成ではなく、機能単位の構成を表している。
図11は、監視対象及びその周辺の色を、監視対象の位置の変化度合いを示す指標値に応じた色に変更する様子を概念的に例示する図である。図11が示す撮像画像10-1と10-2は、それぞれ異なる時点で同じ通路を撮像した画像である。図11(a)に示す撮像画像10-1は、図11(b)に示す撮像画像10-2によりも前に撮像された画像である。撮像画像10-1と10-2とを比較すると、人20の位置はほぼ変化しておらず、それ以外の人の位置は大きく変化している。ここで、滞留している人は注意して監視すべき人であると考えられる。
本実施形態の画像処理装置2000によれば、監視対象の状態の変化度合いに基づいて撮像画像の表示色が決定され、その表示色を用いた表示が提示対象画像上に提示される。そのため本実施形態の画像処理装置2000によれば、提示対象画像上に指標値をそのまま表示する方法と比較し、監視対象の状態の変化度合いを直感的に把握することができる。よって、提示対象画像を見る監視員等にとって、監視対象の現状がより把握しやすくなる。
実施形態3の画像処理装置2000の構成は、実施形態1又は2の画像処理装置2000と同様の構成を有する。
例えば提示部2040は、監視対象の周辺に、指標値の大きさに応じた太さの枠線を表示する。この場合、例えば提示部2040は、以下の式(2)を用いて枠線の太さ b を算出する。b0 は太さの初期値である。I は、指標値算出部2020が算出した指標値である。αは比例定数である。なお、提示部2040が提示する枠線の形状は任意である。
図12は、監視対象の周辺に枠線を提示することで強調表示を行う様子を概念的に例示する図である。図12が示す撮像画像10-1と10-2は、それぞれ異なる時点で同じ場所にある人の行列を撮像した画像である。図11の場合と同様、撮像画像10-1は撮像画像10-2によりも前に撮像された画像である。撮像画像10-1と10-2とを比較すると、上の行列30-1の長さは変化しておらず、下の行列30-2の長さは大きく変化している。ここで、行列の長さは時間が経つにつれ短くなることが好ましく、長さの変化度合いが小さい行列を注視すべきであると考えられる。
また、実施形態3の画像処理装置2000は、実施形態2で説明した表示色決定部2060を用い、監視対象の色又は監視対象の周辺の色をその監視対象について決定した表示色に変更することで、その監視対象を強調する表示を提示してもよい。例えば指標値算出部2020は、監視対象の表示色の濃さを濃くすることで、その監視対象を強調する。また、表示色決定部2060は、強調すべき監視対象の指標値に対応する色ほど目立つ色で構成されたカラーマップを用いて、監視対象の表示色を構成する。例えば、指標値が大きい監視対象ほど強調する場合、表示色決定部2060は、大きい指標値に対応する色ほど目立つ色(赤色など)であり、小さい指標値に対応する色ほど目立たない色(灰色など)であるカラーマップを用いる。
図13は、監視対象の周辺に指標値に応じた色及び太さの枠線を提示することで強調表示を行う様子を概念的に例示する図である。図13が示す撮像画像10-1と10-2は、それぞれ異なる時点で同じ場所にある群衆を撮像した画像である。図11や図12の場合と同様、撮像画像10-1は撮像画像10-2によりも前に撮像された画像である。撮像画像10-1と10-2とを比較すると、右上の群衆40-1に含まれる人の数は増加し、左下の群衆40-2に含まれる人の数は減少している。
本実施形態の画像処理装置2000によれば、監視対象の指標値に基づいた程度でその監視対象を強調する表示が、提示対象画像上に提示される。そのため、提示対象画像を見る監視員等は、各監視対象の変化度合いをすぐに把握できると共に、各監視対象をどの程度注意して監視すべきかをすぐに把握することができる。
図14は、実施形態4に係る画像処理装置2000を示すブロック図である。図14において、矢印は情報の流れを表している。さらに、図14において、各ブロックは、ハードウエア単位の構成ではなく、機能単位の構成を表している。
乖離度算出部2080が乖離度を算出する方法は様々である。例えば乖離度算出部2080は、下記の式(3)を用いて乖離度 k を算出する。I は監視対象について算出した指標値であり、Ibase は基準となる変化度合いである。ただし、乖離度の算出方法は下記の方法に限定されない。
例えば実施形態4の画像処理装置2000は、乖離度に基づいて監視対象の色を変更する。この場合、実施形態4の画像処理装置2000は、表示色決定部2060を有する。
提示部2040は、実施形態3で説明した方法と同様の方法により、監視対象について算出した乖離度の大きさに基づいて、その監視対象を強調する表示を提示してもよい。
例えば提示部2040は、実施形態3と同様に、枠線や色を用いて強調表示を行う。この場合、例えば提示部2040は、数式(4)にしたがって監視対象の枠線の太さ b' を決定する。ここで、k は前述の乖離度を表す。例えばαを正の実数とすると、乖離度が大きいほど枠線の太さが太くなる。
実施形態4の表示色決定部2060は、実施形態3の表示色決定部2060と同様に、監視対象の表示色を変更することで、監視対象を強調する表示を提示してもよい。具体的には、表示色決定部2060は、乖離度が大きい監視対象ほど強調したい場合、乖離度が大きいほど表示色の濃さを濃くしたり、大きい乖離度に対応する色ほど目立つ色で構成されたカラーマップを用いて表示色を決定する。同様に、表示色決定部2060は、乖離度が小さい監視対象ほど強調したい場合、乖離度が小さいほど表示色の濃さを濃くしたり、小さい乖離度に対応する色ほど目立つ色で構成されたカラーマップを用いて表示色を決定する。
本実施形態によれば、監視対象の状態の変化度合いが、基準となる変化度合いからどの程度乖離しているかに基づいて、監視対象を強調する表示が提示対象画像上に提示される。基準となる変化度合いを導入することにより、監視対象を強調すべき度合いをより正確に求めることができる。よって、監視員等は、各監視対象について、注意して監視すべき度合いをより正確に把握することができる。
実施形態4の画像処理装置2000の構成は、実施形態1と同様に図1で表される。
本実施形態によれば、監視対象の状態の変化度合いの予測値に基づく表示が、提示対象画像上に提示される。そのため、監視員等は、今後の行動を注視した方がよい監視対象を即座に把握することができる。
実施形態6において、あるカメラ3000(以下、カメラ3000-1)によって撮像された撮像画像は、表示画面4000によって表示される期間と表示されない期間がある。例えば1つの表示画面4000に、複数のカメラ3000によって撮像された撮像画像を時分割で表示させる場合などが該当する。
本実施形態によれば、カメラ3000-1によって撮像された撮像画像が、ある期間表示画面4000に表示されず、その期間後に表示画面4000に表示される場合、その期間の前後において監視対象の状態が変化した度合いを示す指標値が算出される。こうすることで、監視員等は、例えば表示画面4000のチャンネルをカメラ3000-1の映像からカメラ3000-2の映像に切り替えてしばらく後、再度表示画面4000のチャンネルをカメラ3000-1の映像に切り替えた際、前回カメラ3000-1の映像を見た時と比べて各監視対象の状態がどの程度変化したかを即座に把握することができる。よって、特定のカメラ3000が撮像した撮像画像のみを監視し続けることが難しい場合でも、あるカメラ3000が撮像した画像を見た時に、そのカメラ3000に写っている監視対象の状態の変化度合いを即座に把握することができる。
実施形態7の画像処理装置2000は、実施形態1の画像処理装置2000と同様に、図1で表される。
指標値算出部2020は、ユーザの視線方向を取得する。視線方向は、例えば「水平方向の角度、垂直方向の角度」の組み合わせで表される。ここで、水平方向の角度や垂直方向の角度の基準(0度とする方向)は任意である。
例えば、指標値算出部2020は、表示画面4000を予め所定数の部分領域に分割して扱う。そして、指標値算出部2020は、視線方向検出部から監視員の視線方向を取得し、その視線方向がどの部分領域に対応するか割り出す。そして、割り出した部分領域が、前回割り出した部分領域と異なる場合、ユーザの視線方向に対応する部分領域が変化したことを把握する。
本実施形態によれば、ある領域を監視していない期間があった場合、次にその領域を見た際に、前回その領域を見た時からの各監視対象の状態の変化度合いを示す表示が、表示画面4000上に提示される。そのため、監視員等は、表示画面4000の全ての領域を一度に監視できない場合であっても、各領域に写っている監視対象の状態の変化度合いをすぐに把握することができる。
実施形態7の画像処理装置2000と同様の構成で、以下に示す変形例7-1の画像処理装置2000を実現してもよい。当該変形例7-1の画像処理装置2000において、表示画面4000は、複数の小画面4100を有する。そして、各小画面4100には、それぞれ異なるカメラ3000によって撮像された撮像画像が表示される。
1. カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを示す指標値を算出する指標値算出手段と、
前記カメラによって撮像された第1撮像画像上に、前記指標値に基づく表示を提示する提示手段と、
を有する画像処理装置。
2. 前記監視対象について、前記指標値に応じた表示色を決定する第1表示色決定手段を有し、
前記提示手段は、前記第1撮像画像において、前記監視対象の色又はその監視対象の周辺の色を、その監視対象について決定した前記表示色に変更する1.に記載の画像処理装置。
3. 前記提示手段は、前記指標値が大きい監視対象ほど強調する表示を提示するか、又は前記指標値が小さい監視対象ほど強調する表示を提示する1.又は2.に記載の画像処理装置。
4. 前記指標値と基準となる変化度合いとの乖離度を算出する乖離度算出手段を有し、
前記提示手段は、前記第1撮像画像において、前記乖離度が大きい監視対象ほど強調する表示を提示するか、又は前記乖離度が小さい監視対象ほど強調する表示を提示する1.に記載の画像処理装置。
5. 前記監視対象に対し、その監視対象について算出した前記乖離度に応じた表示色を決定する第2表示色決定手段を有し、
前記提示手段は、前記第1撮像画像において、前記監視対象の色又はその監視対象の周辺の色を、その監視対象に対して決定した前記表示色に変更する4.に記載の画像処理装置。
6. 前記指標値算出手段は、算出された前記監視対象の状態の変化度合いを用いて、その算出に用いられた各撮像画像が撮像された時点以降における、その監視対象の状態の変化度合いの予測値を算出し、その予測値を前記指標値とする1.乃至5.いずれか一つに記載の画像処理装置。
7. 前記指標値算出手段は、前記カメラによって撮像された撮像画像を表示する表示画面において、ある期間撮像画像が表示されない場合、その期間の前に表示される監視対象の状態と、その期間の後に表示される監視対象の状態との間の変化度合いを示す前記指標値を算出し、
前記提示手段は、前記期間の後に表示される撮像画像を前記第1撮像画像として用いる1.乃至6.いずれか一つに記載の画像処理装置。
8. 前記指標値算出手段は、前記撮像画像を表示する表示画面の第1部分領域が、ある期間その表示画面を見るユーザの視線方向又は顔方向に対応する画面領域に含まれていない場合、その期間の前に前記第1部分領域に表示される前記監視対象の状態と、その期間の後に前記第1部分領域に表示される前記監視対象の状態との間の変化度合いを示す指標値を算出し、
前記提示手段は、前記期間の後に表示される前記撮像画像を前記第1撮像画像として用い、その第1撮像画像のうちの前記第1部分領域に表示される領域上に、前記第1部分領域について算出した前記指標値に基づく表示を提示する1.乃至7.いずれか一つに記載の画像処理装置。
9. 前記指標値算出手段は、前記監視対象の位置の変化度合いを示す指標値を算出する1.乃至8.いずれか一つに記載の画像処理装置。
10. 前記指標値算出手段は、前記監視対象が前記画像に写っている頻度の変化度合いを示す指標値を算出する1.乃至9.いずれか一つに記載の画像処理装置。
11. 前記指標値算出手段は、前記監視対象に含まれる複数のオブジェクトの密集度合いの変化度合いを示す指標値を算出する1.乃至10.いずれか一つに記載の画像処理装置。
12. 前記監視対象はオブジェクトの行列を含み、
前記指標値算出手段は、前記行列の長さ又は速さの変化度合いを示す指標値を算出する1.乃至11.いずれか一つに記載の画像処理装置。
13. 前記指標値算出手段は、前記監視対象に含まれるオブジェクトの数の変化度合いを示す指標値を算出する1.乃至12.いずれか一つに記載の画像処理装置。
14. 前記監視対象は人を含み、
前記指標値算出手段は、前記監視対象の前記指標値として、その監視対象が持つ不満度の変化度合いを示す指標値を算出する1.乃至13.いずれか一つに記載の画像処理装置。
15. 前記監視対象は人又は場所を含み、
前記指標値算出手段は、前記監視対象の前記指標値として、その監視対象の危険度の変化度合いを示す指標値を算出する1.乃至14.いずれか一つに記載の画像処理装置。
16. 前記監視対象は人又は場所を含み、
前記指標値算出手段は、前記監視対象の前記指標値として、その監視対象が監視されている度合いの変化度合いを示す指標値を算出する1.乃至15.いずれか一つに記載の画像処理装置。
17. カメラ、表示画面、及び1.乃至16.いずれか一つに記載の画像処理装置を有する監視システムであって、
前記カメラは、それぞれ異なる時点で撮像を行うことで、複数の撮像画像を生成し、
前記表示画面は、前記提示手段によって前記指標値に基づく表示が提示された前記第1撮像画像を表示する監視システム。
18. コンピュータによって実行される画像処理方法であって、
カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを示す指標値を算出する指標値算出ステップと、
前記カメラによって撮像された第1撮像画像上に、前記指標値に基づく表示を提示する提示ステップと、
を有する画像処理方法。
19. 前記監視対象について、前記指標値に応じた表示色を決定する第1表示色決定ステップを有し、
前記提示ステップは、前記第1撮像画像において、前記監視対象の色又はその監視対象の周辺の色を、その監視対象について決定した前記表示色に変更する18.に記載の画像処理方法。
20. 前記提示ステップは、前記指標値が大きい監視対象ほど強調する表示を提示するか、又は前記指標値が小さい監視対象ほど強調する表示を提示する18.又は19.に記載の画像処理方法。
21. 前記指標値と基準となる変化度合いとの乖離度を算出する乖離度算出ステップを有し、
前記提示ステップは、前記第1撮像画像において、前記乖離度が大きい監視対象ほど強調する表示を提示するか、又は前記乖離度が小さい監視対象ほど強調する表示を提示する18.に記載の画像処理方法。
22. 前記監視対象に対し、その監視対象について算出した前記乖離度に応じた表示色を決定する第2表示色決定ステップを有し、
前記提示ステップは、前記第1撮像画像において、前記監視対象の色又はその監視対象の周辺の色を、その監視対象に対して決定した前記表示色に変更する21.に記載の画像処理方法。
23. 前記指標値算出ステップは、算出された前記監視対象の状態の変化度合いを用いて、その算出に用いられた各撮像画像が撮像された時点以降における、その監視対象の状態の変化度合いの予測値を算出し、その予測値を前記指標値とする18.乃至22.いずれか一つに記載の画像処理方法。
24. 前記指標値算出ステップは、前記カメラによって撮像された撮像画像を表示する表示画面において、ある期間撮像画像が表示されない場合、その期間の前に表示される監視対象の状態と、その期間の後に表示される監視対象の状態との間の変化度合いを示す前記指標値を算出し、
前記提示ステップは、前記期間の後に表示される撮像画像を前記第1撮像画像として用いる18.乃至23.いずれか一つに記載の画像処理方法。
25. 前記指標値算出ステップは、前記撮像画像を表示する表示画面の第1部分領域が、ある期間その表示画面を見るユーザの視線方向又は顔方向に対応する画面領域に含まれていない場合、その期間の前に前記第1部分領域に表示される前記監視対象の状態と、その期間の後に前記第1部分領域に表示される前記監視対象の状態との間の変化度合いを示す指標値を算出し、
前記提示ステップは、前記期間の後に表示される前記撮像画像を前記第1撮像画像として用い、その第1撮像画像のうちの前記第1部分領域に表示される領域上に、前記第1部分領域について算出した前記指標値に基づく表示を提示する18.乃至24.いずれか一つに記載の画像処理方法。
26. 前記指標値算出ステップは、前記監視対象の位置の変化度合いを示す指標値を算出する18.乃至25.いずれか一つに記載の画像処理方法。
27. 前記指標値算出ステップは、前記監視対象が前記画像に写っている頻度の変化度合いを示す指標値を算出する18.乃至26.いずれか一つに記載の画像処理方法。
28. 前記指標値算出ステップは、前記監視対象に含まれる複数のオブジェクトの密集度合いの変化度合いを示す指標値を算出する18.乃至27.いずれか一つに記載の画像処理方法。
29. 前記監視対象はオブジェクトの行列を含み、
前記指標値算出ステップは、前記行列の長さ又は速さの変化度合いを示す指標値を算出する18.乃至28.いずれか一つに記載の画像処理方法。
30. 前記指標値算出ステップは、前記監視対象に含まれるオブジェクトの数の変化度合いを示す指標値を算出する18.乃至29.いずれか一つに記載の画像処理方法。
31. 前記監視対象は人を含み、
前記指標値算出ステップは、前記監視対象の前記指標値として、その監視対象が持つ不満度の変化度合いを示す指標値を算出する18.乃至30.いずれか一つに記載の画像処理方法。
32. 前記監視対象は人又は場所を含み、
前記指標値算出ステップは、前記監視対象の前記指標値として、その監視対象の危険度の変化度合いを示す指標値を算出する18.乃至31.いずれか一つに記載の画像処理方法。
33. 前記監視対象は人又は場所を含み、
前記指標値算出ステップは、前記監視対象の前記指標値として、その監視対象が監視されている度合いの変化度合いを示す指標値を算出する18.乃至32.いずれか一つに記載の画像処理方法。
34. コンピュータを、1.乃至16.いずれか一つに記載の画像処理装置として動作させるプログラム。
35. カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを算出する算出手段と、
前記カメラによって撮像された撮像画像上において、前記監視対象を表す領域の色を、前記変化度合いに基づく色に変更する提示手段と、
を有する画像処理装置。
36. コンピュータによって実行される画像処理方法であって、
カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを算出する算出ステップと、
前記カメラによって撮像された撮像画像上において、前記監視対象を表す領域の色を、前記変化度合いに基づく色に変更する提示ステップと、
を有する画像処理方法。
37. コンピュータを35.に記載の画像処理装置として動作させるプログラム。
38. カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを算出する算出手段と、
前記カメラによって撮像された撮像画像上において、前記変化度合いに基づいて前記監視対象を強調する提示手段と、
を有する画像処理装置。
39. コンピュータによって実行される画像処理方法であって、
カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを算出する算出ステップと、
前記カメラによって撮像された撮像画像上において、前記変化度合いに基づいて前記監視対象を強調する提示ステップと、
を有する画像処理方法。
40. コンピュータを39.に記載の画像処理装置として動作させるプログラム。
Claims (21)
- カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを示す指標値を算出する指標値算出手段と、
前記カメラによって撮像された第1撮像画像上に、前記指標値に基づく表示を提示する提示手段と、
を有する画像処理装置。 - 前記監視対象について、前記指標値に応じた表示色を決定する第1表示色決定手段を有し、
前記提示手段は、前記第1撮像画像において、前記監視対象の色又はその監視対象の周辺の色を、その監視対象について決定した前記表示色に変更する請求項1に記載の画像処理装置。 - 前記提示手段は、前記指標値が大きい監視対象ほど強調する表示を提示するか、又は前記指標値が小さい監視対象ほど強調する表示を提示する請求項1又は2に記載の画像処理装置。
- 前記指標値と基準となる変化度合いとの乖離度を算出する乖離度算出手段を有し、
前記提示手段は、前記第1撮像画像において、前記乖離度が大きい監視対象ほど強調する表示を提示するか、又は前記乖離度が小さい監視対象ほど強調する表示を提示する請求項1に記載の画像処理装置。 - 前記監視対象に対し、その監視対象について算出した前記乖離度に応じた表示色を決定する第2表示色決定手段を有し、
前記提示手段は、前記第1撮像画像において、前記監視対象の色又はその監視対象の周辺の色を、その監視対象に対して決定した前記表示色に変更する請求項4に記載の画像処理装置。 - 前記指標値算出手段は、算出された前記監視対象の状態の変化度合いを用いて、その算出に用いられた各撮像画像が撮像された時点以降における、その監視対象の状態の変化度合いの予測値を算出し、その予測値を前記指標値とする請求項1乃至5いずれか一項に記載の画像処理装置。
- 前記指標値算出手段は、前記カメラによって撮像された撮像画像を表示する表示画面において、ある期間撮像画像が表示されない場合、その期間の前に表示される監視対象の状態と、その期間の後に表示される監視対象の状態との間の変化度合いを示す前記指標値を算出し、
前記提示手段は、前記期間の後に表示される撮像画像を前記第1撮像画像として用いる請求項1乃至6いずれか一項に記載の画像処理装置。 - 前記指標値算出手段は、前記撮像画像を表示する表示画面の第1部分領域が、ある期間その表示画面を見るユーザの視線方向又は顔方向に対応する画面領域に含まれていない場合、その期間の前に前記第1部分領域に表示される前記監視対象の状態と、その期間の後に前記第1部分領域に表示される前記監視対象の状態との間の変化度合いを示す指標値を算出し、
前記提示手段は、前記期間の後に表示される前記撮像画像を前記第1撮像画像として用い、その第1撮像画像のうちの前記第1部分領域に表示される領域上に、前記第1部分領域について算出した前記指標値に基づく表示を提示する請求項1乃至7いずれか一項に記載の画像処理装置。 - 前記指標値算出手段は、前記監視対象の位置の変化度合いを示す指標値を算出する請求項1乃至8いずれか一項に記載の画像処理装置。
- 前記指標値算出手段は、前記監視対象が前記画像に写っている頻度の変化度合いを示す指標値を算出する請求項1乃至9いずれか一項に記載の画像処理装置。
- 前記指標値算出手段は、前記監視対象に含まれる複数のオブジェクトの密集度合いの変化度合いを示す指標値を算出する請求項1乃至10いずれか一項に記載の画像処理装置。
- 前記監視対象はオブジェクトの行列を含み、
前記指標値算出手段は、前記行列の長さ又は速さの変化度合いを示す指標値を算出する請求項1乃至11いずれか一項に記載の画像処理装置。 - 前記指標値算出手段は、前記監視対象に含まれるオブジェクトの数の変化度合いを示す指標値を算出する請求項1乃至12いずれか一項に記載の画像処理装置。
- 前記監視対象は人を含み、
前記指標値算出手段は、前記監視対象の前記指標値として、その監視対象が持つ不満度の変化度合いを示す指標値を算出する請求項1乃至13いずれか一項に記載の画像処理装置。 - 前記監視対象は人又は場所を含み、
前記指標値算出手段は、前記監視対象の前記指標値として、その監視対象の危険度の変化度合いを示す指標値を算出する請求項1乃至14いずれか一項に記載の画像処理装置。 - 前記監視対象は人又は場所を含み、
前記指標値算出手段は、前記監視対象の前記指標値として、その監視対象が監視されている度合いの変化度合いを示す指標値を算出する請求項1乃至15いずれか一項に記載の画像処理装置。 - カメラ、表示画面、及び請求項1乃至16いずれか一項に記載の画像処理装置を有する監視システムであって、
前記カメラは、それぞれ異なる時点で撮像を行うことで、複数の撮像画像を生成し、
前記表示画面は、前記提示手段によって前記指標値に基づく表示が提示された前記第1撮像画像を表示する監視システム。 - コンピュータによって実行される画像処理方法であって、
カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを示す指標値を算出する指標値算出ステップと、
前記カメラによって撮像された第1撮像画像上に、前記指標値に基づく表示を提示する提示ステップと、
を有する画像処理方法。 - コンピュータを、請求項1乃至16いずれか一項に記載の画像処理装置として動作させるプログラム。
- カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを算出する算出手段と、
前記カメラによって撮像された撮像画像上において、前記監視対象を表す領域の色を、前記変化度合いに基づく色に変更する提示手段と、
を有する画像処理装置。 - カメラによってそれぞれ異なる時点で撮像された複数の撮像画像を用いて、前記撮像画像に写っている監視対象の状態の変化度合いを算出する算出手段と、
前記カメラによって撮像された撮像画像上において、前記変化度合いに基づいて前記監視対象を強調する提示手段と、
を有する画像処理装置。
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2016531203A JP6597609B2 (ja) | 2014-06-30 | 2015-06-01 | 画像処理装置、監視システム、画像処理方法、及びプログラム |
CN201580035751.XA CN107113399A (zh) | 2014-06-30 | 2015-06-01 | 图像处理装置、监视系统、图像处理方法和程序 |
US15/323,294 US10269126B2 (en) | 2014-06-30 | 2015-06-01 | Image processing apparatus, monitoring system, image processing method, and program |
US16/293,359 US20190206067A1 (en) | 2014-06-30 | 2019-03-05 | Image processing apparatus, monitoring system, image processing method,and program |
US16/293,381 US10909697B2 (en) | 2014-06-30 | 2019-03-05 | Image processing apparatus, monitoring system, image processing method,and program |
US17/121,433 US11403771B2 (en) | 2014-06-30 | 2020-12-14 | Image processing apparatus, monitoring system, image processing method, and program |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014134786 | 2014-06-30 | ||
JP2014-134786 | 2014-06-30 |
Related Child Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/323,294 A-371-Of-International US10269126B2 (en) | 2014-06-30 | 2015-06-01 | Image processing apparatus, monitoring system, image processing method, and program |
US16/293,381 Continuation US10909697B2 (en) | 2014-06-30 | 2019-03-05 | Image processing apparatus, monitoring system, image processing method,and program |
US16/293,359 Continuation US20190206067A1 (en) | 2014-06-30 | 2019-03-05 | Image processing apparatus, monitoring system, image processing method,and program |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016002408A1 true WO2016002408A1 (ja) | 2016-01-07 |
Family
ID=55018960
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/065725 WO2016002408A1 (ja) | 2014-06-30 | 2015-06-01 | 画像処理装置、監視システム、画像処理方法、及びプログラム |
Country Status (5)
Country | Link |
---|---|
US (4) | US10269126B2 (ja) |
JP (3) | JP6597609B2 (ja) |
CN (1) | CN107113399A (ja) |
AR (1) | AR101008A1 (ja) |
WO (1) | WO2016002408A1 (ja) |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPWO2017122258A1 (ja) * | 2016-01-12 | 2018-11-22 | 株式会社日立国際電気 | 混雑状況監視システム及び混雑状況監視方法 |
JP2019029747A (ja) * | 2017-07-27 | 2019-02-21 | セコム株式会社 | 画像監視システム |
JP2019061640A (ja) * | 2017-09-27 | 2019-04-18 | ソフトバンク株式会社 | 監視システム |
JP2020095357A (ja) * | 2018-12-10 | 2020-06-18 | トヨタ自動車株式会社 | 行動監視装置、行動監視システム、及び行動監視プログラム |
JP2020184252A (ja) * | 2019-05-09 | 2020-11-12 | パナソニックIpマネジメント株式会社 | ストレス推定システム |
WO2020241034A1 (ja) * | 2019-05-30 | 2020-12-03 | アースアイズ株式会社 | 監視システム及び監視方法 |
JP2020195110A (ja) * | 2019-05-30 | 2020-12-03 | アースアイズ株式会社 | 共用スペース用の監視システム及び監視方法 |
WO2021181790A1 (ja) * | 2020-03-12 | 2021-09-16 | オムロン株式会社 | 画像処理装置、画像処理方法およびプログラム |
JP2022082627A (ja) * | 2020-07-16 | 2022-06-02 | 日本電気株式会社 | 情報処理方法、情報処理システム及び情報処理装置 |
WO2023212151A1 (en) * | 2022-04-27 | 2023-11-02 | Armored Things, Inc. | Simulation-driven localization and tracking of people |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017081717A1 (ja) * | 2015-11-09 | 2017-05-18 | 三菱電機株式会社 | 画像生成システム及び画像生成方法 |
JP6918523B2 (ja) * | 2017-03-06 | 2021-08-11 | キヤノン株式会社 | 情報処理システム、情報処理装置、情報処理方法ならびに情報処理方法をコンピュータに実行させるプログラム |
CN108596045B (zh) * | 2018-04-02 | 2021-08-20 | 四川大学 | 一种基于空中监控平台的群体异常行为检测方法 |
US11704543B2 (en) * | 2018-06-12 | 2023-07-18 | Samsung Electronics Co., Ltd. | Neural network hardware acceleration with stochastic adaptive resource allocation |
US11164335B2 (en) * | 2018-11-06 | 2021-11-02 | International Business Machines Corporation | Passenger travel route inferencing in a subway system |
JPWO2022004302A1 (ja) * | 2020-06-30 | 2022-01-06 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003259337A (ja) * | 2002-02-26 | 2003-09-12 | Toshiba Lighting & Technology Corp | 監視カメラシステム |
JP2008519567A (ja) * | 2004-11-02 | 2008-06-05 | センサーマティック・エレクトロニクス・コーポレーション | 行列監視のためのシステム及び方法 |
JP2011130203A (ja) * | 2009-12-17 | 2011-06-30 | Canon Inc | 映像情報処理方法及びその装置 |
JP2012022370A (ja) * | 2010-07-12 | 2012-02-02 | Hitachi Kokusai Electric Inc | 監視システムおよび監視方法 |
Family Cites Families (29)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5275184A (en) * | 1990-10-19 | 1994-01-04 | Dainippon Screen Mfg. Co., Ltd. | Apparatus and system for treating surface of a wafer by dipping the same in a treatment solution and a gate device for chemical agent used in the apparatus and the system |
JPH0981874A (ja) * | 1995-09-18 | 1997-03-28 | Isamu Kato | 紛失警報装置 |
US20020168084A1 (en) | 2001-05-14 | 2002-11-14 | Koninklijke Philips Electronics N.V. | Method and apparatus for assisting visitors in navigating retail and exhibition-like events using image-based crowd analysis |
US20020180866A1 (en) * | 2001-05-29 | 2002-12-05 | Monroe David A. | Modular sensor array |
JP4480299B2 (ja) * | 2001-06-21 | 2010-06-16 | 富士通マイクロエレクトロニクス株式会社 | 移動体を含む画像の処理方法及び装置 |
JP2007209008A (ja) * | 2003-10-21 | 2007-08-16 | Matsushita Electric Ind Co Ltd | 監視装置 |
JP2005328236A (ja) | 2004-05-13 | 2005-11-24 | Nippon Telegr & Teleph Corp <Ntt> | 映像監視方法、映像監視装置、および映像監視プログラム |
US8284254B2 (en) | 2005-08-11 | 2012-10-09 | Sightlogix, Inc. | Methods and apparatus for a wide area coordinated surveillance system |
US7378969B2 (en) * | 2005-10-25 | 2008-05-27 | Sap Ag | Systems and methods for visualizing auto-id data |
KR100882011B1 (ko) * | 2007-07-29 | 2009-02-04 | 주식회사 나노포토닉스 | 회전 대칭형의 광각 렌즈를 이용하여 전방위 영상을 얻는 방법 및 장치 |
US8195598B2 (en) | 2007-11-16 | 2012-06-05 | Agilence, Inc. | Method of and system for hierarchical human/crowd behavior detection |
JP2009148498A (ja) * | 2007-12-21 | 2009-07-09 | Panasonic Corp | 超音波診断装置および超音波計測値表示方法 |
KR20100107049A (ko) * | 2008-01-16 | 2010-10-04 | 코닌클리즈케 필립스 일렉트로닉스 엔.브이. | 존재 검출에 기초하여 조명 분위기를 자동 조정하기 위한 시스템 및 방법 |
EP2093698A1 (en) * | 2008-02-19 | 2009-08-26 | British Telecommunications Public Limited Company | Crowd congestion analysis |
JP5025607B2 (ja) | 2008-09-17 | 2012-09-12 | セコム株式会社 | 異常行動検知装置 |
CN101751553B (zh) * | 2008-12-03 | 2012-04-25 | 中国科学院自动化研究所 | 一种大规模人群密度分析和预测方法 |
JP5003731B2 (ja) | 2009-07-07 | 2012-08-15 | 日本電気株式会社 | 巡回警備支援システム、方法及びプログラム |
TWI397865B (zh) | 2009-08-12 | 2013-06-01 | Utechzone Co Ltd | Security personnel to monitor the degree of focus monitoring system |
JP4547040B1 (ja) | 2009-10-27 | 2010-09-22 | パナソニック株式会社 | 表示画像切替装置及び表示画像切替方法 |
US20130268230A1 (en) | 2010-12-16 | 2013-10-10 | Bae Systems Plc | Processing distributions |
US9098604B2 (en) * | 2011-12-22 | 2015-08-04 | General Electric Company | System and method for monitoring clinician responsiveness to alarms |
US9530221B2 (en) * | 2012-01-06 | 2016-12-27 | Pelco, Inc. | Context aware moving object detection |
JP5971045B2 (ja) * | 2012-09-12 | 2016-08-17 | サクサ株式会社 | 監視システム、撮影装置及びデータ管理装置 |
EP2918071B1 (en) * | 2012-11-06 | 2017-05-03 | Alcatel Lucent | System and method for processing visual information for event detection |
US9536308B2 (en) * | 2012-12-10 | 2017-01-03 | Verint Systems Ltd. | Irregular event detection in push notifications |
CN103246869B (zh) * | 2013-04-19 | 2016-07-06 | 福建亿榕信息技术有限公司 | 基于人脸识别和行为语音识别的犯罪监控方法 |
JP6532106B2 (ja) | 2013-04-26 | 2019-06-19 | 日本電気株式会社 | 監視装置、監視方法および監視用プログラム |
JP5438861B1 (ja) * | 2013-07-11 | 2014-03-12 | パナソニック株式会社 | 追跡支援装置、追跡支援システムおよび追跡支援方法 |
WO2015155833A1 (ja) * | 2014-04-08 | 2015-10-15 | 三菱電機株式会社 | 衝突防止装置 |
-
2015
- 2015-06-01 JP JP2016531203A patent/JP6597609B2/ja active Active
- 2015-06-01 WO PCT/JP2015/065725 patent/WO2016002408A1/ja active Application Filing
- 2015-06-01 CN CN201580035751.XA patent/CN107113399A/zh active Pending
- 2015-06-01 US US15/323,294 patent/US10269126B2/en active Active
- 2015-06-26 AR ARP150102057A patent/AR101008A1/es unknown
-
2019
- 2019-03-05 US US16/293,359 patent/US20190206067A1/en not_active Abandoned
- 2019-03-05 US US16/293,381 patent/US10909697B2/en active Active
- 2019-10-02 JP JP2019182148A patent/JP6801760B2/ja active Active
-
2020
- 2020-11-13 JP JP2020189231A patent/JP7092177B2/ja active Active
- 2020-12-14 US US17/121,433 patent/US11403771B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003259337A (ja) * | 2002-02-26 | 2003-09-12 | Toshiba Lighting & Technology Corp | 監視カメラシステム |
JP2008519567A (ja) * | 2004-11-02 | 2008-06-05 | センサーマティック・エレクトロニクス・コーポレーション | 行列監視のためのシステム及び方法 |
JP2011130203A (ja) * | 2009-12-17 | 2011-06-30 | Canon Inc | 映像情報処理方法及びその装置 |
JP2012022370A (ja) * | 2010-07-12 | 2012-02-02 | Hitachi Kokusai Electric Inc | 監視システムおよび監視方法 |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10346688B2 (en) | 2016-01-12 | 2019-07-09 | Hitachi Kokusai Electric Inc. | Congestion-state-monitoring system |
JPWO2017122258A1 (ja) * | 2016-01-12 | 2018-11-22 | 株式会社日立国際電気 | 混雑状況監視システム及び混雑状況監視方法 |
JP7099809B2 (ja) | 2017-07-27 | 2022-07-12 | セコム株式会社 | 画像監視システム |
JP2019029747A (ja) * | 2017-07-27 | 2019-02-21 | セコム株式会社 | 画像監視システム |
JP2019061640A (ja) * | 2017-09-27 | 2019-04-18 | ソフトバンク株式会社 | 監視システム |
JP2020095357A (ja) * | 2018-12-10 | 2020-06-18 | トヨタ自動車株式会社 | 行動監視装置、行動監視システム、及び行動監視プログラム |
JP7115277B2 (ja) | 2018-12-10 | 2022-08-09 | トヨタ自動車株式会社 | 行動監視装置、行動監視システム、及び行動監視プログラム |
JP2020184252A (ja) * | 2019-05-09 | 2020-11-12 | パナソニックIpマネジメント株式会社 | ストレス推定システム |
WO2020241034A1 (ja) * | 2019-05-30 | 2020-12-03 | アースアイズ株式会社 | 監視システム及び監視方法 |
JP2020195110A (ja) * | 2019-05-30 | 2020-12-03 | アースアイズ株式会社 | 共用スペース用の監視システム及び監視方法 |
WO2021181790A1 (ja) * | 2020-03-12 | 2021-09-16 | オムロン株式会社 | 画像処理装置、画像処理方法およびプログラム |
JP2022082627A (ja) * | 2020-07-16 | 2022-06-02 | 日本電気株式会社 | 情報処理方法、情報処理システム及び情報処理装置 |
WO2023212151A1 (en) * | 2022-04-27 | 2023-11-02 | Armored Things, Inc. | Simulation-driven localization and tracking of people |
Also Published As
Publication number | Publication date |
---|---|
CN107113399A (zh) | 2017-08-29 |
US20190197708A1 (en) | 2019-06-27 |
US20190206067A1 (en) | 2019-07-04 |
US10909697B2 (en) | 2021-02-02 |
JP6597609B2 (ja) | 2019-10-30 |
JP2021044821A (ja) | 2021-03-18 |
US20210104053A1 (en) | 2021-04-08 |
JP6801760B2 (ja) | 2020-12-16 |
US11403771B2 (en) | 2022-08-02 |
AR101008A1 (es) | 2016-11-16 |
JP7092177B2 (ja) | 2022-06-28 |
US20170148178A1 (en) | 2017-05-25 |
JP2020078058A (ja) | 2020-05-21 |
JPWO2016002408A1 (ja) | 2017-05-25 |
US10269126B2 (en) | 2019-04-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP7092177B2 (ja) | 画像処理装置、画像処理方法、及びプログラム | |
US11373408B2 (en) | Image processing apparatus, monitoring system, image processing method, and program | |
JP5597781B1 (ja) | 滞留状況分析装置、滞留状況分析システムおよび滞留状況分析方法 | |
US9875408B2 (en) | Setting apparatus, output method, and non-transitory computer-readable storage medium | |
AU2021201574A1 (en) | Security system and method | |
US20150363638A1 (en) | Information processing system, information processing method, and program | |
US20200363903A1 (en) | Engagement analytic system and display system responsive to interaction and/or position of users | |
CN109410890B (zh) | 显示控制方法、装置和电子设备 | |
JP2017046196A (ja) | 画像情報生成装置、画像情報生成方法、画像処理システム及びプログラム | |
CN112070052A (zh) | 一种间距监测方法、装置、系统及存储介质 | |
US9436996B2 (en) | Recording medium storing image processing program and image processing apparatus | |
US20120038602A1 (en) | Advertisement display system and method | |
JP6516702B2 (ja) | 人数集計システム、人数集計方法、及び人数集計結果の閲覧方法 | |
US9767564B2 (en) | Monitoring of object impressions and viewing patterns | |
JP2019029747A (ja) | 画像監視システム | |
CN110622183B (zh) | 用于行为控制的信息提示装置、信息提示系统、以及信息提示方法 | |
JP2021027399A (ja) | 情報処理装置、情報処理方法及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15814643 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2016531203 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15323294 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15814643 Country of ref document: EP Kind code of ref document: A1 |