JP2022110291A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

To determine whether a physical object is present in a monitored area.SOLUTION: An information processing apparatus according to one aspect of the present disclosure includes means of acquiring a space image relating to a target space, means of extracting an object image included in the space image from the space image, means of identifying a position of a physical object corresponding to the object image in a reference coordinate system of a space model relating to the target space by aligning the space image with the space mode, and means of determining whether the identified position of the physical object is included in a target area defined in the reference coordinate system.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to an information processing device, an information processing method, and a program.

Conventionally, surveillance cameras are sometimes used to monitor human intrusion into restricted areas.

In Patent Document 1, a change detection area in an image is determined based on temporal changes, and it is determined whether or not the geometric feature value of the change detection area corresponds to an intruder. If so, it describes locating the intruder in the image.

Japanese Patent Application Laid-Open No. 2006-252248

According to Patent Document 1, the position of the intruder in the image is specified. However, the relationship between the position in the image and the position in real space is not clear. Therefore, when the visual field of the camera is wider than the restricted area, even if the position of the intruder in the image is specified, it cannot be determined immediately whether the intruder exists in the restricted area.

An object of the present disclosure is to determine whether a physical object exists in a monitored area.

An information processing apparatus according to an aspect of the present disclosure includes means for obtaining a spatial image regarding a target space, means for extracting an object image included in the spatial image from the spatial image, and position of the spatial image and a spatial model regarding the target space. Alignment includes means for identifying a position of a physical object corresponding to the object image in a reference coordinate system of the spatial model and whether the identified physical object position falls within a region of interest defined by the reference coordinate system. means for determining whether or not

1 is a block diagram showing the configuration of an information processing system according to an embodiment; FIG. 2 is a block diagram showing configurations of a client device and a server according to this embodiment; FIG. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an explanatory diagram of the outline of this embodiment; It is a figure which shows the data structure of the target area|region database of this embodiment. It is a figure which shows the whole flow of information processing of this embodiment. It is a figure which shows the example of a screen displayed in the information processing of this embodiment. FIG. 10 is a diagram showing a data structure of a target area database of modification 1; FIG. 10 is a diagram showing an overall flow of information processing according to modification 1;

An embodiment of the present invention will be described in detail below with reference to the drawings. In the drawings for describing the embodiments, in principle, the same constituent elements are denoted by the same reference numerals, and repeated description thereof will be omitted.

(1) Configuration of information processing system The configuration of the information processing system of this embodiment will be described. FIG. 1 is a block diagram showing the configuration of the information processing system of this embodiment.

As shown in FIG. 1 , the information processing system 1 includes a client device 10 and a server 30 .
The client device 10 and server 30 are connected via a network (for example, the Internet or an intranet) NW.

The client device 10 is an example of an information processing device. The client device 10 is, for example, a smart phone, a tablet terminal, or a personal computer.

The server 30 is an example of an information processing device. Server 30 is, for example, a web server. The server 30 refers to an image (spatial image) of a target space (referred to as a “target space”) and a spatial model of the target space, and monitors an area to be monitored for the existence of a physical object in the target space. (referred to as a "monitoring area") determines whether a physical object exists. The server 30 notifies the user via the client device 10 according to the determination result.

Spatial images are generated by sensors (not shown) placed in the target space. Here, an image means an optical sensing result. In the following description, images are assumed to be two-dimensional images generated by an image sensor. However, the image may also be a point cloud (which may include color point clouds) generated by a depth camera.

A sensor is installed in the target space. A sensor performs sensing and generates an image. The sensor transmits the image to server 30 or an image server (not shown). A sensor is any type of optical sensor device. A sensor may be any of the following:
・Camera ・Depth camera ・LIDAR (Light Detection And Ranging)
・ToF (Time of Flight) camera ・Laser scanner

(1-1) Configuration of Client Apparatus The configuration of the client apparatus according to this embodiment will be described. FIG. 2 is a block diagram showing configurations of a client device and a server according to this embodiment.

As shown in FIG. 1, the client device 10 includes a storage device 11, a processor 12, an input/output interface 13, and a communication interface .

Storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of ROM (Read Only Memory), RAM (Random Access Memory), and storage (eg, flash memory or hard disk).

Programs include, for example, the following programs.
・OS (Operating System) program ・Application (for example, web browser) program that executes information processing

The data includes, for example, the following data.
・Data and databases referenced in information processing ・Data obtained by executing information processing (that is, execution results of information processing)

The processor 12 is configured to implement the functions of the client device 10 by activating programs stored in the storage device 11 and processing data. Processor 12 is an example of a computer. The programs and data stored in the storage device 11 may be provided via a network, or may be provided by being recorded on a computer-readable recording medium. At least part of the functions of the client device 10 may be realized by one or more pieces of dedicated hardware.

The input/output interface 13 acquires a signal (for example, a user's instruction) from an input device connected to the client device 10 and outputs a signal (for example, an image signal) to an output device connected to the client device 10. configured as
Input devices are, for example, keyboards, pointing devices, touch panels, or combinations thereof.
An output device is, for example, a display.

Communication interface 14 is configured to control communications between client device 10 and server 30 .

(1-2) Server Configuration The configuration of the server 30 will be described with reference to FIG.

As shown in FIG. 2, the server 30 includes a storage device 31, a processor 32, an input/output interface 33, and a communication interface .

Storage device 31 is configured to store programs and data. Storage device 31 is, for example, a combination of ROM, RAM, and storage (eg, flash memory or hard disk).

Programs include, for example, the following programs.
・OS program ・Application program that executes information processing

The data includes, for example, the following data.
・Data and databases referenced in information processing ・Execution results of information processing

The processor 32 is configured to implement the functions of the server 30 by activating programs stored in the storage device 31 . Processor 32 is an example of a computer. The programs and data stored by the storage device 31 may be provided via a network, or may be provided by being recorded on a computer-readable recording medium. At least part of the functions of the server 30 may be realized by one or more pieces of dedicated hardware.

The input/output interface 33 acquires a signal (for example, a user's instruction) from an input device connected to the server 30 and outputs a signal (for example, an image signal) to an output device connected to the server 30. Configured.
Input devices are, for example, keyboards, pointing devices, touch panels, or combinations thereof.
An output device is, for example, a display.

Communication interface 34 is configured to control communications between server 30 and client device 10 .

(2) Outline of Embodiment An outline of the present embodiment will be described. FIG. 3 is an explanatory diagram of the outline of this embodiment.

As shown in FIG. 3, the sensor 50 installed in the target space SP captures, for example, the target space SP and generates a spatial image SI1 regarding the target space SP. The spatial image SI1 includes an object image OBI1 corresponding to the physical object OBJ1 that existed in the target space SP at the time of shooting. Here, the physics object is a person, an object, a non-human creature, or a combination thereof.

The storage device 31 stores a space model SM regarding the target space SP. Spatial models include, for example, at least one of the following:
・3D model (BIM (Building Information Modeling) data as an example)
・ 4D model related to 3D model that changes along time series (as an example, BIM data with time information added)

The server 30 acquires the spatial image SI1 and extracts the object image OBI1 from the spatial image SI1. The server 30 aligns the spatial image SI1 and the spatial model SM. As an example, the server 30 converts the position of each information element (pixel or point) included in the object image OBI1 to a position in the reference coordinate system. Thereby, the server 30 identifies the position of the physical object in a coordinate system (referred to as a "reference coordinate system") corresponding to the space model SM.

In the target space SP, there is an area (“monitoring area”) to monitor whether or not a physical object exists. The spatial model SM can be aligned with any component in the object space SP (i.e., matching the reference coordinate system of the spatial model SM with the coordinate system of the object space SP (“global coordinate system”). ), any region in the object space SP can be transformed into a region expressed in the reference coordinate system. A region obtained by expressing the monitoring region in the reference coordinate system is called a “target region”. A setting person can, for example, intuitively set a desired location in the reference coordinate system as a target area while viewing a three-dimensional view of the space model SM. By setting the target space, the corresponding location in the target space SP is specified as the monitoring area. Moreover, even when the field of view of the sensor 50 (for example, the position, orientation, focal length, or combination thereof) of the sensor 50 is changed, the target region TR1 and the monitoring region remain unchanged. In the example of FIG. 3, the target area TR1 is set in the space model SM. The server 30 can determine whether the physical object OBJ1 exists in the monitoring area in the target space SP by determining whether the position of the physical object OBJ1 in the reference coordinate system is included in the target area TR1.

When the server 30 determines that the position of the physical object OBJ1 in the reference coordinate system is included in the target region TR1, the server 30 notifies the user US. Specifically, the server 30 transmits notification data (for example, image, text, voice, or a combination thereof) for notifying the user US that the physical object OBJ1 exists in the monitoring area corresponding to the target area TR1. Send to the client device 10 . The client device 10 notifies the user US according to the notification data received from the server 30 . As an example, the client device 10 causes the display to display the output image OI1.

Thus, the server 30 can determine from the spatial image whether or not a physical object exists in the monitoring area in the target space SP. In other words, according to this server 30, for example, it is possible to detect that a person is in a place where entry is prohibited or in a dangerous place, or that equipment is left in an unfixed position. .

(3) Database The database of this embodiment will be described. The following databases are stored in the storage device 31.

(3-1) Target Area Database The target area database of this embodiment will be described. FIG. 4 is a diagram showing the data structure of the target area database of this embodiment.

The target area database stores target area information. The target area information is information about the target area. By referring to the target area information, the position of the target area in the reference coordinate system can be specified.

As shown in FIG. 4, the target area database includes an "area ID" field and an "area definition" field. Each field is associated with each other.

An area ID is stored in the "area ID" field. The area ID is information that identifies the target area.

The "area definition" field stores area definition information. The area definition information is information relating to the definition of the target area. As an example, the area definition information includes area shape information and coordinate information.

The area shape information is information about the shape of the target area. The target area may be a three-dimensional shape (for example, a horizontal shape of the target area (width and depth as an example) and a vertical shape of the target area (height as an example)), or two-dimensional. It may be a shape (for example, a horizontal plane shape of the region of interest).

The coordinate information is information about the coordinates of a predetermined point in the target area in the reference coordinate system. As an example, the coordinate information includes information on the coordinates of each vertex forming the target area in the reference coordinate system.

(4) Information processing Information processing according to the present embodiment will be described. FIG. 5 is a diagram showing the overall flow of information processing according to this embodiment. FIG. 6 is a diagram showing an example of a screen displayed during information processing according to this embodiment.

The processing in FIG. 5 starts when the start condition is satisfied. Entry conditions can include, for example, at least one of the following:
- The user has input an instruction to start information processing to the client device 10, and the server 30 has received a request to start information processing from the client device 10. -A predetermined date and time has arrived.

As shown in FIG. 5, the server 30 acquires a spatial image (S110).
Specifically, processor 32 acquires a spatial image from a sensor or image server (not shown).

After step S110, the server 30 extracts an object image (S120).
Specifically, processor 32 extracts an object image from the spatial image acquired in step S110. As an example, the processor 32 identifies an object image corresponding to a physical object that existed in the target space when the spatial image was captured by performing image recognition processing on the spatial image. For example, the physical object is a physical object to be monitored (a person, for example).

After step S120, the server 30 performs physical object location determination (S130).
Specifically, the processor 32 aligns the spatial image acquired in step S110 with the spatial model of the target space. As an example, the processor 32 transforms the position of each information element of the object image included in the spatial image to a position in the reference coordinate system. Processor 32 thereby locates the physical object in the reference coordinate system corresponding to the spatial model.

In the first example of specifying the position of the physical object (S130), the processor 32 refers to the shooting conditions of the spatial image and the reference coordinate system, and reproduces the shooting direction of the spatial image in the reference coordinate system. Align with the spatial model. Here, the imaging condition is the position, orientation, focal length, or a combination thereof of the camera that captured the spatial image. The imaging conditions may be transmitted by the camera together with the spatial image, or may be stored in the storage device 31 as known information. For example, the processor 32 identifies the position and orientation in the reference coordinate system corresponding to the position and orientation in the physical space of the camera that captured the spatial image.
The processor 32 refers to the specified position and orientation in the reference coordinate system and the focal length of the camera, and sets the virtual camera in the reference coordinate system.
The processor 32 compares the model image of the spatial model photographed by the set camera with the spatial image. As an example, the processor 32 determines the position of an arbitrary pixel of the spatial image in the reference coordinate system by comparing the feature quantity (eg edge) extracted from the model image and the feature quantity extracted from the spatial image.

In a second example of locating the physical object (S130), the processor 32 compares a model image of the spatial model taken from a particular viewpoint with the spatial image, and aligns the spatial image and the spatial model. For example, the processor 32 repeats selecting a viewpoint and comparing a model image obtained by photographing the spatial model with a virtual camera from the selected viewpoint and the spatial image while changing the viewpoint. Thereby, the processor 32 searches for a model image that best matches the spatial image (for example, the feature amount matches best).
Processor 32 determines the position in the reference coordinate system of any pixel (“target pixel”) of the model image as the position in the reference coordinate system of the pixel in the spatial image corresponding to the target pixel.

After step S130, the server 30 executes object position determination (S140).
Specifically, the processor 32 determines whether the position of the physical object identified in step S130 is included in the target area defined by the reference coordinate system.

When it is determined in step S140 that the position of the physical object is included in the target area, the server 30 executes notification of the existence of the physical object (S150).
Specifically, processor 32 notifies the user that a physical object exists in the monitored area. As an example, processor 32 sends notification data (eg, image, text, sound, or a combination thereof) to client device 10 to notify the user that a physical object is present in the monitored area.

After step S150, the client device 10 performs notification (S160).
Specifically, the processor 12 notifies the user according to the notification data received from the server 30 . As an example, the processor 12 displays the screen P10 of FIG. 6 on the display.

Screen P10 includes a notification object N10. The notification object N10 is an object related to a message corresponding to notification data (for example, a message indicating that a physical object exists in the monitoring area).

When it is determined in step S140 that the position of the physical object is not included in the target area, the server 30 executes spatial image acquisition (S110) to acquire an unprocessed spatial image (that is, monitor the physical object). continue).

(5) Summary As described above, the server 30 of this embodiment extracts the object image from the spatial image, and aligns the spatial image with the spatial model to determine the position of the physical object in the reference coordinate system. It is determined whether or not the specified position is included in the target area. Therefore, this server can determine from the spatial image whether or not a physical object exists in the monitored area in the target space. This makes it possible, for example, to detect that a person is in an off-limits or dangerous place, or that equipment is left out of place.

The server 30 may notify when the position of the physical object is included in the target area. This allows the user to recognize that a physical object exists in the monitored area.

(6) Modification A modification of the present embodiment will be described.

(6-1) Modification 1
Modification 1 will be described. Modification 1 is an example in which, when a physical object corresponding to an object image corresponds to an exceptional object, no notification is made even if the physical object exists within the monitoring area.

(6-1-1) Database The database of modification 1 will be described. The following databases are stored in the storage device 31.

(6-1-1-1) Target Area Database The target area database of Modification 1 will be described. FIG. 7 is a diagram showing the data structure of the target area database of Modification 1. As shown in FIG.

The target area database stores target area information. The target area information is information about the target area. By referring to the target area information, it is possible to identify the position of the target area in the reference coordinate system and the exception object associated with the target area. Even if the server 30 of Modification 1 determines that the physical object exists within the target area, if it determines that the physical object corresponds to the exception object, it determines that the physical object exists within the target area. is not notified to the user.

As shown in FIG. 7, the subject area database includes a "area ID" field, a "area definition" field, and an "exception object ID" field. Each field is associated with each other.

The "region ID" field and "region definition" field are as described in the embodiment.

The "exception object ID" field stores an exception object ID. The exception object ID is information that identifies a physical object that satisfies a predetermined exception condition (hereinafter referred to as "exception object"). Exceptional conditions include, for example, at least one of the following:
- It is a physical object authorized to enter and exit the monitoring area corresponding to the target area.
- It is a physical object that is permitted to be brought into the monitoring area corresponding to the target area.

(6-1-2) Information Processing The information processing of Modification 1 will be described. FIG. 8 is a diagram showing the overall flow of information processing according to Modification 1. As shown in FIG.

The processing in FIG. 8 is performed when the user inputs an information processing start instruction to the client device 10 and the server 30 receives an information processing start request from the client device 10 or when another start condition is satisfied. start accordingly.

As shown in FIG. 8, the server 30 acquires a spatial image (S110) and extracts an object image (S120), as in FIG.

After step S120, the server 30 executes exception object determination (S230).
Specifically, the processor 32 determines whether or not the physical object corresponding to the object image extracted in step S120 corresponds to the exception object associated with the target area. If a plurality of object images are extracted in step S120, the processor 32 determines whether or not all physical objects correspond to exception objects (that is, whether or not at least one physical object corresponds to an exception object). do.

In a first example of exceptional object determination (S230), the processor 32 determines whether or not a physical object corresponds to an exceptional object by performing image recognition processing on an object image.

In a second example of exceptional object determination (S230), the processor 32 determines whether the physical object corresponds to an exceptional object based on the identification information of the tag carried by the physical object. Processor 32 may obtain the identification information from the tag or may obtain the identification information from a reader (not shown) configured to read the identification information of the tag.

When it is determined in step S230 that the physical object does not correspond to the exception object, the server 30 specifies the position of the physical object (S130) and determines the object position (S140), as in FIG.

When it is determined in step S140 that the position of the physical object is included in the target area, the server 30 performs notification of the existence of the physical object (S150), as in FIG.

After step S150, the client device 10 performs notification (S160), as in FIG.

If it is determined in step S230 that the physical object corresponds to the exception object, the server 30 executes spatial image acquisition (S110) to acquire an unprocessed spatial image (that is, continue monitoring the physical object).

When it is determined in step S140 that the position of the physical object is not included in the target area, the server 30 executes spatial image acquisition (S110) to acquire an unprocessed spatial image (that is, monitor the physical object). continue).

(6-1-3) Summary As described above, the server 30 of Modification 1 extracts the object image from the spatial image and determines whether or not the physical object corresponds to the exceptional object. The server 30 identifies the position of the physical object in the reference coordinate system by aligning the spatial image and the spatial model, and determines whether or not the identified position is included in the target area. Therefore, this server 30 can determine from the space image whether or not a physical object exists in the monitored area in the target space and whether or not the physical object corresponds to an exception object. As a result, it is possible to detect, for example, that an unauthorized person is present in an off-limits or dangerous area, or that an unauthorized article is brought into the target area.

The server 30 may give notification when the physical object does not correspond to the exception object and the position of the physical object is included in the target area. This allows the user to recognize that physical objects other than exception objects exist in the monitored area.

(6-2) Modification 2
Modification 2 will be described. Modification 2 is an example in which a spatial part image corresponding to the spatial part is extracted from the spatial image, and the position of the physical object in the reference coordinate system is specified based on the positional relationship between the spatial part and the physical object.

The server 30 performs the locating (S130) of the physical object of FIG. 5 or 8 as described below.

Specifically, the processor 32 extracts spatial part images corresponding to spatial parts included in the spatial model from the spatial image. Spatial parts are components of a space (eg, floors or walls).
The processor 32 aligns the spatial image and the spatial model as described in the present embodiment or modification 1. FIG. Thereby, the processor 32 identifies the position of each information element of the spatial parts image in the reference coordinate system.
Processor 32 identifies the position of the physical object in the reference coordinate system by referring to the positional relationship between the spatial part and the physical object and the position of each information element of the spatial part image in the reference coordinate system.

As an example, the processor 32 determines that the vertically downward edge of the physical object (for example, a human foot) constitutes at least a part of the vertically downward surface of the space (that is, the vertically downward edge of the physical object Physics objects are located under the assumption that they are in contact with a spatial part (e.g., the floor). In this example, the processor 32 calculates the reference coordinate system of the downward vertical end of the physical object based on the position of the part of the space part that is in contact with the downward vertical end of the physical object in the reference coordinate system. determine the position in Here, the positional relationship between the spatial parts and the physical object is not limited to contact. For example, the processor 32 moves the vertically upward end of the physical object (eg, a human head) vertically upward by a predetermined distance (eg, the height of the physical object) from the spatial part located on the vertically downward plane of the space. ) may locate physics objects under the assumption that they are at a distance.

As described above, the server 30 of Modification 2 extracts the spatial parts image corresponding to the spatial part from the spatial image, and calculates the position of the physical object in the reference coordinate system based on the positional relationship between the spatial part and the physical object. Identify. This makes it possible to specify the position of the physical object in the reference coordinate system with higher accuracy.

(6-3) Modification 3
Modification 3 will be described. Modification 3 is based on the size (eg, the number of pixels) of a partial image corresponding to at least a portion of the physical object (“target portion”) in the object image and the size (eg, actual size) of the target portion. It is an example of identifying the position of a physical object in a coordinate system.

The server 30 performs the locating (S130) of the physical object of FIG. 5 or 8 as described below.

Specifically, the processor 32 refers to the shooting conditions of the spatial image (for example, the position and orientation of the sensor in the reference coordinate system and the focal length of the sensor), and extracts the target part (for example, the human body) from the spatial image from the sensor. Determine the orientation in the reference coordinate system of the vector (the "target vector") directed toward the head). The processor 32 refers to the direction of the target vector, the dimension of the partial image corresponding to the target region in the spatial image, and the dimension of the target region to identify the magnitude of the target vector. Processor 32 refers to the position of the sensor and the direction and magnitude of the target vector to identify the position of the target site in the reference coordinate system.

As described above, the server 30 of Modification 3 calculates the position of the physical object in the reference coordinate system based on the dimensions of the partial image corresponding to the target portion of the physical object in the object image and the dimensions of the target portion. Identify. This makes it possible to specify the position of the physical object in the reference coordinate system with higher accuracy.

(7) Other Modifications The storage device 11 may be connected to the client device 10 via the network NW. Storage device 31 may be connected to server 30 via network NW.

Each step of the information processing described above can be executed by either the client device 10 or the server 30 .

The server 30 may identify the target area according to a predetermined algorithm.
As an example, the server 30 may refer to the spatial model and the shooting time of the spatial image, and identify the target area corresponding to the shooting time. Here, the spatial model may be a four-dimensional model that variably defines the target area with respect to time. Thereby, the target area can be set on a time basis. For example, it is possible to include the surroundings of a certain space part in the target area during the construction period for the space part, and exclude the surroundings of the space part from the target area after the construction period ends. Similarly, exception object IDs may be defined variably with respect to time.

The server 30 may specify the target area according to the user's instruction. As a first example, the user sets an area in the reference coordinate system as the target area while checking the space model displayed on the display of the client device 10 . As a second example, the user sets an area in the reference coordinate system as the target area while checking the point cloud data displayed on the display of the client device 10 . The point cloud data may be, for example, spatial image (point cloud) data generated by a depth camera, or by performing SfM (Structure from Motion) processing on a plurality of spatial image (two-dimensional image) data. may be created. For example, the server 30 may generate target area information according to a user's instruction and store it in the target area database (FIG. 4 or FIG. 7). This makes it possible to flexibly and accurately set the target area in accordance with the contents of the physical object monitoring task imposed on the user.

In the embodiment, an example is given in which the user is notified that the physical object exists in the monitored area when it is determined that the position of the physical object is included in the target area. However, when the server 30 determines that the position of the physical object is not included in the target area, the server 30 may notify the user that the physical object does not exist in the monitored area.

In Modified Example 1, when it is determined that the physical object does not correspond to the exception object and the position of the physical object is included in the target area, the user is notified that the physical object exists in the monitoring area. . However, if the server 30 determines that the physical object does not correspond to the exception object and the position of the physical object is not included in the target area, the server 30 notifies the user that the physical object does not exist in the monitored area. good.
Alternatively, the server 30 may notify the user that the physical object does not exist in the monitoring area when determining that the physical object corresponds to the exception object and the position of the physical object is not included in the target area. .
Alternatively, the server 30 may notify the user that the physical object exists in the monitoring area when determining that the physical object corresponds to the exception object and the position of the physical object is included in the target area.

In the embodiment and the modified example, the example of ending the information processing after notifying the existence of the physical object has been shown. However, after notifying the physical object, the server 30 may perform spatial image acquisition (S110) to acquire the unprocessed spatial image (ie, continue monitoring the physical object).

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the above embodiments. Also, the above embodiments can be modified and modified in various ways without departing from the gist of the present invention. Also, the above embodiments and modifications can be combined.

(8) Supplementary notes The matters explained in the embodiment and the modified example are additionally noted below.

(Appendix 1)
means for acquiring a spatial image of the target space (S110);
means for extracting an object image included in the spatial image from the spatial image (S120);
means for identifying the position of the physical object corresponding to the object image in the reference coordinate system of the spatial model by aligning the spatial image with the spatial model for the target space (S130);
An information processing apparatus (30) comprising: means (S140) for determining whether or not the specified position of the physical object is included in a target area defined by a reference coordinate system.

(Appendix 2)
Further comprising means for notifying (S150) when the position of the specified physical object is included in the target area,
The information processing device according to appendix 1.

(Appendix 3)
further comprising means (S230) for determining whether the physical object corresponds to an exception object that satisfies a predetermined exception condition;
The information processing apparatus according to appendix 1 or appendix 2.

(Appendix 4)
Further comprising means (S150) for notifying when the physical object does not correspond to the exception object and the position of the specified physical object is included in the target area,
The information processing device according to appendix 3.

(Appendix 5)
A means of determining the position of a physical object is
A spatial part image corresponding to the spatial part included in the spatial model is extracted from the spatial image,
Identifying the position of the physical object in the reference coordinate system by referring to the positional relationship between the spatial part and the physical object and the position of the spatial part in the reference coordinate system;
The information processing apparatus according to any one of appendices 1 to 4.

(Appendix 6)
The space part constitutes at least a part of the vertically downward surface of the target space,
The information processing device according to appendix 5.

(Appendix 7)
The means for specifying the position of the physical object further refers to the dimensions of a partial image corresponding to the target portion, which is at least a part of the physical object, in the object image and the dimensions of the target portion to determine the position of the physical object in the reference coordinate system. locate,
The information processing apparatus according to any one of appendices 1 to 6.

(Appendix 8)
further comprising means for referring to the spatial model and the shooting time of the spatial image, and specifying a target area corresponding to the shooting time;
The spatial model changes along the time series and is a model for the target space,
The information processing apparatus according to any one of appendices 1 to 7.

(Appendix 9)
8. The information processing apparatus according to any one of appendices 1 to 7, further comprising means for specifying a target area according to a user's instruction.

(Appendix 10)
a computer (30)
obtaining a spatial image of the target space (S110);
extracting an object image included in the spatial image from the spatial image (S120);
locating a physical object corresponding to the object image in a reference coordinate system of the spatial model by aligning the spatial image with the spatial model for the target space (S130);
Determining whether or not the position of the specified physical object is included in a target area defined by a reference coordinate system (S140).

(Appendix 11)
A program for causing a computer (30) to implement each means according to any one of Appendices 1 to 9.

1: Information processing system 10 : Client device 11 : Storage device 12 : Processor 13 : Input/output interface 14 : Communication interface 30 : Server 31 : Storage device 32 : Processor 33 : Input/output interface 34 : Communication interface 50 : Sensor

Claims (11)

means for obtaining a spatial image of a target space;
means for extracting an object image included in the spatial image from the spatial image;
means for locating a physical object corresponding to the object image in a reference coordinate system of the spatial model by aligning the spatial image with a spatial model for the target space;
and means for determining whether or not the specified position of the physical object is included in a target area defined by the reference coordinate system. further comprising means for notifying when the position of the identified physical object is included in the target area;
The information processing device according to claim 1 . further comprising means for determining whether the physical object corresponds to an exception object that satisfies a predetermined exception condition;
The information processing apparatus according to claim 1 or 2. further comprising means for notifying when the physical object does not correspond to the exception object and the position of the identified physical object is included in the target area;
The information processing apparatus according to claim 3. The means for locating the physical object comprises:
extracting a spatial part image corresponding to the spatial part included in the spatial model from the spatial image;
identifying the position of the physical object in the reference coordinate system by referring to the positional relationship between the spatial part and the physical object and the position of the spatial part in the reference coordinate system;
The information processing apparatus according to any one of claims 1 to 4. The spatial parts constitute at least part of a vertically downward surface of the target space,
The information processing device according to claim 5 . The means for specifying the position of the physical object further refers to the size of a partial image corresponding to a target portion that is at least a part of the physical object in the object image and the size of the target portion, and further refers to the reference coordinates. identifying the position of the physics object in a system;
The information processing apparatus according to any one of claims 1 to 6. further comprising means for referring to the spatial model and the shooting time of the spatial image and specifying a target area corresponding to the shooting time;
The spatial model changes along a time series and is a model for the target space,
The information processing apparatus according to any one of claims 1 to 7. 8. The information processing apparatus according to any one of claims 1 to 7, further comprising means for specifying said target area according to a user's instruction. the computer
obtaining a spatial image of the target space;
extracting an object image included in the spatial image from the spatial image;
locating a physical object corresponding to the object image in a reference coordinate system of the spatial model by aligning the spatial image with a spatial model for the target space;
and determining whether or not the identified position of the physical object is included in a target area defined by the reference coordinate system. A program for causing a computer to implement each means according to any one of claims 1 to 9.

Images