JP6386263B2 - Image forming apparatus, image forming system, and image forming program - Google Patents

Description

  The present invention relates to a technical field such as an image forming apparatus that forms an image obtained by superimposing CG (Computer Graphics) representing a structure on an image of a site where the structure is installed.

  In recent years, a movable platform fence (hereinafter referred to as “home door”) has been developed in order to improve the safety of users on a platform (hereinafter referred to as “home”) at a railway station. In planning the installation of a home door, studies are being conducted by visually recognizing the home door virtually installed on the home using a head-mounted display with mixed reality. However, a system that realizes mixed reality using a head-mounted display is expensive, and in order to check the visibility for different stations, it is necessary to create new virtual spaces. However, since it requires a lot of labor and time, it may not be used easily.

  On the other hand, some proposals have been made on a method of superimposing a three-dimensional VR (Virtual Reality) model on an image taken by a camera like mixed reality. A technique for superimposing a three-dimensional VR model of a structure on an image of a surveillance camera is disclosed.

JP 2014-13527 A

  The technique described in Patent Document 1 superimposes a three-dimensional VR model of a structure on an image using three-dimensional absolute coordinates, and can superimpose a three-dimensional VR model on an accurate position in the image. Yes. On the other hand, it is necessary to obtain the three-dimensional absolute coordinates in advance by surveying or the like, and it may be difficult to apply to a place where it is difficult to survey the position where the three-dimensional VR model is superimposed.

The present invention has been made in view of such problems and the like, and it is not necessary to survey the three-dimensional absolute coordinate of the position where the three-dimensional VR model is superimposed, and the structure 3 is positioned at an accurate position in the image. It is an object of the present invention to provide an image forming apparatus or the like that can form a superimposed image on which a three-dimensional VR model is superimposed.

In order to solve the above-mentioned problem, the invention according to claim 1 is characterized in that a storage means for storing a three-dimensional VR (Virtual Reality) model representing a structure, a site where the structure is installed, and a marker are within an imaging range. Image acquisition means for acquiring a captured image, forming means for forming a superimposed image in which the three-dimensional VR model is superimposed on the captured image with reference to a marker reflected in the captured image, and the marker in real space as a reference Real information acquisition means for acquiring actual information indicating the direction and actual distance from the predetermined position, and based on the direction and actual distance indicated by the actual information, the in-image distance in the captured image corresponding to the actual distance. An image distance calculation means for calculating, wherein the forming means is based on the direction indicated by the real information and the image distance based on the predetermined position in the captured image. And forming a superimposed image obtained by superimposing.

According to the first aspect of the present invention, the three-dimensional VR model representing the structure is superimposed based on the direction and the actual distance from the predetermined position with reference to the marker appearing in the captured image. It is possible to form a superimposed image in which a three-dimensional VR model of a structure is superimposed at an accurate position in the image without requiring surveying of the three-dimensional absolute coordinates of the position to be superimposed. That is, only the direction and the actual distance from the predetermined position are measured, and the position for superimposing the 3D VR model in the captured image can be determined without requiring labor for surveying the 3D absolute coordinates. . In addition, since the direction and the actual distance can be easily measured as compared with the case where the three-dimensional absolute coordinates are measured, it is also easy to consider a plurality of positions with respect to the position where the structure is installed.

  The structure includes station equipment such as a platform door, for example. In addition, not only station equipment, but also facilities, facilities, buildings, etc. such as soundproof walls installed in places other than the station are included.

  Note that the captured image may be either a moving image or a still image. Similarly, the formed superimposed image may be an image in which a 3D VR model is superimposed on a moving image, or may be an image in which a 3D VR model is superimposed on a still image. When a 3D VR model is superimposed on a moving image, a moving image captured by an existing surveillance camera or the like can be used as a captured image. Since surveillance cameras are often installed near the site where railway-related facilities are installed, there is an advantage that it is not necessary to provide a camera for forming a new superimposed image. Of course, the captured image is not limited to the image captured by the existing surveillance camera, and may be an image captured by the newly installed camera. That is, the captured image of the present invention is not limited to an existing camera, but includes an image captured by a newly installed camera.

According to a second aspect of the invention, an image forming apparatus according to claim 1, further comprising a calculating means for calculating the three-dimensional position and orientation from the shape of the marker in the captured image by the image acquisition unit has acquired The forming unit forms a superimposed image in which the three-dimensional VR model is superimposed on the captured image based on the calculation result calculated by the calculating unit.

According to the second aspect of the present invention, it is possible to superimpose the 3D VR model at an appropriate position in the captured image based on the 3D position and orientation.

According to a third aspect of the invention, an image forming apparatus according to claim 1 or 2, and acquires the operation content of the superimposed positions or modifying modifies the superposition angle operation of the 3D VR Model in the superimposed image The information processing apparatus further includes a correction operation content acquisition unit, wherein the forming unit updates a superimposition position or a superposition angle of the three-dimensional VR model in the superimposed image according to the operation content acquired by the correction operation content acquisition unit. .

According to the invention described in claim 3 , the superimposition position or superimposition angle of the three-dimensional VR model in the superimposed image can be corrected by the correction operation.

Invention of Claim 4 is an image forming apparatus as described in any one of Claim 1 thru | or 3 , Comprising: The said memory | storage means memorize | stores the three-dimensional VR model showing the said multiple types of structure, It further comprises selection operation content acquisition means for acquiring the operation content of the selection operation for selecting a 3D VR model representing any kind of the structure, and the forming means is acquired by the selection operation content acquisition means A superimposed image is formed by superimposing a three-dimensional VR model representing the type of structure specified by the operation content. Regarding the three-dimensional VR model representing a plurality of types of structures, it can be considered that the models having different shapes at the time of display specified by the construction stage / home door parameter described later are also of different types.

According to the fourth aspect of the present invention, it is possible to form a superimposed image in which a three-dimensional VR model representing a plurality of types of structures is superimposed. It becomes easy to consider.

A fifth aspect of the present invention is the image forming apparatus according to any one of the first to fourth aspects, wherein the structure is a home door, the site is a home, and the three-dimensional VR model. Is constructed according to the conditions regarding the home.

According to the fifth aspect of the present invention, it is possible to make progress on the installation of the home door to the platform. In addition, by constructing a three-dimensional VR model according to the conditions related to the platform such as the curve radius, gradient, and height of the platform, it is possible to immediately superimpose it on the platform of a station with different conditions regarding the platform.

A sixth aspect of the present invention includes the image forming apparatus according to any one of the first to fifth aspects, and a camera that captures the captured image.

According to a seventh aspect of the present invention, there is provided a computer included in an image forming apparatus including a storage unit that stores a three-dimensional VR (Virtual Reality) model representing a structure. An image acquisition means for acquiring a captured image included in the image forming apparatus, a forming means for forming a superimposed image in which the three-dimensional VR model is superimposed on the captured image with reference to a marker reflected in the captured image, and the marker in real space as a reference Based on the actual information acquisition means for acquiring the actual information indicating the direction and the actual distance from the predetermined position, the intra-image distance in the captured image corresponding to the actual distance is calculated based on the direction and the actual distance indicated by the actual information. An image forming program that functions as an in-image distance calculating unit, wherein the computer that functions as the forming unit uses the predetermined position in the captured image as a reference As an image forming program, a superimposed image is formed by superimposing the three-dimensional VR model on the basis of the direction indicated by the actual information and the distance in the image.

According to the present invention, since the superposing 3D VR model representing the structure based on the direction and the actual distance from a predetermined position relative to the marker caught on the photographed image, third position for superimposing a three-dimensional VR model It is possible to form a superimposed image in which a three-dimensional VR model of a structure is superimposed at an accurate position in the image without requiring measurement of the dimensional absolute coordinates .

1 is a block diagram illustrating an example of a configuration of an image forming system S. FIG. 1 is a block diagram illustrating a configuration of an image forming apparatus 1. FIG. It is a figure which shows the example of the three-dimensional VR model of multiple types of home doors. It is a figure which shows the example of the imaging | photography image which image | photographed the installation site of a home door, and a marker. It is a figure which shows a mode that the 3D VR model of the home door was superimposed on the picked-up image. 4 is a flowchart illustrating an example of a superimposed image forming process performed by a control unit 11.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiment described below is an embodiment when the present invention is applied to an image forming system. Moreover, although this embodiment demonstrates the case where a structure is a platform door, a structure is not restricted to a platform door, This invention is applicable also to another structure.

[1. Overview of image forming system configuration and functions]
First, the configuration and outline functions of an image forming system S according to an embodiment of the present disclosure will be described with reference to FIG.

  FIG. 1 is a diagram illustrating an example of a schematic configuration of an image forming system S according to the present embodiment. The image forming system S is configured by incorporating the image forming apparatus 1 into a conventional surveillance camera system. A conventional surveillance camera system includes a surveillance camera system control device 200, a surveillance camera 300 (an example of a “camera”) that photographs a surveillance location such as a home, and a surveillance monitor 400 that displays an image captured by the surveillance camera 300. It is configured to include. The monitoring camera system control device 200, the monitoring camera 300, and the monitoring monitor 400 are connected via a network NW. The network NW is constructed by, for example, a LAN (local area network), the Internet, a dedicated communication line (for example, CATV (Community Antenna Television) line), a mobile communication network (including a base station, etc.), a gateway, and the like. . An image captured by the monitoring camera 300 is displayed on the monitoring monitor 400 under the control of the monitoring camera system control apparatus 200. The monitoring monitor 400 is installed at a station staff's standby place, a monitoring center, or the like, and the station staff or the monitoring staff monitors the home via the monitoring monitor 400.

  The image forming apparatus 1 is connected to the monitoring camera system control apparatus 200, captures an image (sometimes referred to as “captured image”) taken by the monitoring camera 300 via a converter (not shown), and three-dimensionally displays the home door. A process of superimposing the VR model is performed, and the process is again returned to the monitoring camera system control apparatus 200 via a converter (not shown). The monitoring camera system control device 200 displays a superimposed image on which the three-dimensional VR model is superimposed on the monitoring monitor 400. As a result, an image on which the three-dimensional VR model of the home door is superimposed is displayed on the monitoring monitor 400. According to the image forming system S, the monitor monitor 400 or the image forming apparatus 1 can easily provide an image on which a three-dimensional VR model of a home door is superimposed simply by incorporating the image forming apparatus 1 into a conventional monitoring camera system. This can be confirmed on the display unit.

[2. Configuration of Image Forming Apparatus 1]
FIG. 2 is a block diagram illustrating a schematic configuration example of the image forming apparatus 1 according to the present embodiment. As illustrated in FIG. 2, the image forming apparatus 1 includes a control unit 11, a storage unit 12, an interface unit 13, a display unit 14, and an operation unit 15.

  The storage unit 12 includes, for example, a hard disk drive (HDD) and stores various programs such as an operating system and application programs. In particular, the storage unit 12 of the present embodiment stores an application program (hereinafter referred to as “image forming program”) for forming a superimposed image. Note that the various programs may be acquired from another server device or the like via a network, or may be recorded on a recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc) to be a drive device. You may make it read via.

  The storage unit 12 (an example of “storage means”) stores a three-dimensional VR model of the home door. The 3D VR model is created based on the design drawing of the home door. As shown in FIG. 3, the storage unit 12 stores a plurality of types of three-dimensional VR models 700 of home doors. These three-dimensional VR models 700 are stored for each home door parameter. Home door parameters include radius of curvature (if the edge of the home is curved, it is necessary to install the home door along the curve), distance (length), height, depth, home slope, home There is a height. Since the three-dimensional VR model is constructed according to the conditions relating to the platform in this way, it can be immediately superimposed on the platforms of stations with different conditions relating to the platform.

  For each home door whose shape is determined by the type of home door and the home door parameter, the storage unit 12 stores a three-dimensional VR model 700 for each member of the home door. For each member, construction stage information, display type information, construction stage start / end date information, and order information are associated. The construction stage information is information indicating at which construction stage the member is used. The display type information is information indicating whether or not to display the 3D VR model 700 of the member, and is information that can be appropriately changed by the user. The construction stage start date / end date information is information indicating the start date / end date of the construction stage in which the member is used. The order information is information indicating the order of construction in the construction stage where the member is used. Based on this information, the 3D VR model 700 in the middle of construction is superimposed on the captured image before the platform door is installed, or the 3D VR model 700 at the completion of construction of the platform door is superimposed. You can make it.

  The interface unit 13 transmits, to the monitoring camera system control apparatus 200, a process for receiving a captured image captured by the monitoring camera 300 from the monitoring camera system control apparatus 200, and a superimposed image on which the three-dimensional VR model 700 is superimposed. The process when you do.

  The display unit 14 includes, for example, a liquid crystal display, and displays an operation screen when the user operates the image forming system S or the image forming apparatus 1, an image captured by the monitoring camera 300, and a three-dimensional VR model. A superimposed image or the like is displayed.

  The operation unit 15 includes, for example, a keyboard and a mouse, receives a user operation, and transmits an operation signal indicating the operation content to the control unit 11.

  The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. And the control part 11 as a computer performs the process for forming a superimposed image by reading and executing the image formation program memorize | stored in the memory | storage part 12. FIG. At this time, the control unit 11 functions as an image acquisition unit, a forming unit, an actual information acquisition unit, an in-image distance calculation unit, a calculation unit, a correction operation content acquisition unit, and a selection operation content acquisition unit.

[3. Superposition image formation method]
Next, a method for forming a superimposed image will be described with reference to FIGS. However, detailed description of conventionally known portions will be omitted, and the description will be simplified.

[3.1. Calculation of position / posture of surveillance camera 300]
When the control unit 11 superimposes the three-dimensional VR model 700 on the image captured by the monitoring camera 300, when the image captured by the monitoring camera 300 is acquired from the monitoring camera system control apparatus 200, the capturing of the monitoring camera 300 is performed. 3 based on the measured position and orientation and the unique parameters of the monitoring camera 300 such as lens distortion and focal length (obtained in advance by camera calibration). A dimensional VR model 700 is superimposed. A captured image from the monitoring camera 300 can be used for measuring the position and orientation of the monitoring camera 300. As shown in FIG. 4, the alignment method using the photographed image is obtained by photographing the marker 500 provided on the home with the monitoring camera 300 and matching the position of the marker 500 on the photographed image with the three-dimensional position. And the position and orientation of the monitoring camera 300 are calculated. The marker 500 has a square shape in which a unique two-dimensional pattern is drawn.

  The control unit 11 extracts a square area from the captured image, and identifies whether the marker 500 is the internal two-dimensional pattern. In addition, the control unit 11 determines the position of the monitoring camera 300 based on the correspondence between the position of each vertex of the marker 500 on the image captured by the monitoring camera 300 and the position of each vertex of the marker 500 in the marker coordinate system. And calculate the posture.

[3.2. Identification of home door installation position]
Next, when superimposing the 3D VR model 700 on the captured image, the control unit 11 needs to specify at which position of the captured image the 3D VR model 700 is to be superimposed. Therefore, as shown in FIG. 4, the user provisionally determines the installation position 502 where the home door is installed in the real space, and measures the shortest distance from the reference point 501 that is one vertex of the marker 500 to the installation position 502. As a method for measuring the distance, instead of the distance from the reference point 501 to the installation position 502, a predetermined position (for example, the other vertex of the marker 500, the center of the marker 500, or their center) The distance from the point to the installation position 502 may be measured from a point separated by a predetermined distance in a predetermined direction. This is based on the intention that the image forming apparatus 1 only needs to be able to specify the installation position 502 with reference to the marker 500. The user also specifies the direction from the reference point 501 to the installation position 502 when the distance from the reference point 501 to the installation position 502 is measured. This is because the image forming apparatus 1 cannot determine in which direction the installation position 502 is seen from the reference point 501 only by the distance from the reference point 501 to the installation position 502, and therefore the installation position based on the distance and the direction. Based on the intention to identify 502.

  The user inputs real information indicating the distance and direction from the reference point 501 to the installation position 502 in the real space to the image forming apparatus 1. The control unit 11 specifies on which position of the captured image the 3D VR model 700 of the platform door is to be superimposed based on the input actual information. Specifically, the control unit 11 determines the distance from the reference point 501 to the installation position 502 in the captured image based on the size of the marker 500 on the image captured by the monitoring camera 300 and the actual size of the marker 500 ( (Referred to as “in-image distance”). Further, the control unit 11 specifies a coordinate system in the captured image (referred to as “intra-image coordinate system”) from the shape of the marker 500 in the captured image, and moves in the direction from the in-image coordinate system and the reference point 501 to the installation position 502. Based on this, a direction from the reference point 501 to the installation position 502 (referred to as “in-image direction”) in the captured image is calculated. And the control part 11 pinpoints the position which superimposes the three-dimensional VR model 700 based on the calculated image inner distance and image inner direction.

[3.3. Selection of 3D VR model]
The storage unit 12 stores a plurality of types of three-dimensional VR models 700 of home doors according to the construction stage and the home door parameters. The user designates the type of the home door, the construction stage, and the home door parameters via the operation unit 15. At this time, the control unit 11 causes the display unit 14 to display a list of home door type names, construction stages, and selectable home door parameters, from which the user can select the home door type, construction stage, and home door parameters. It may be specified. At that time, if the 3D VR model 700 of the home door at the final construction stage (when completed) is displayed together on the display unit 14, the user who cannot imagine the type of home door from the type name of the home door is displayed. Useful. The three-dimensional VR model 700 corresponding to the type of the home door, the construction stage, and the home door parameter specified by the user is acquired from the storage unit 12 by the control unit 11 and superimposed on the image taken by the monitoring camera 300. The designation of the 3D VR model 700 by the user (that is, designation of the type of the home door, the construction stage, and the home door parameter) can be performed again after being superimposed on the captured image. In this case, the control unit 11 The superimposed three-dimensional VR model 700 is changed to a newly designated three-dimensional VR model 700. Thereby, when examining installation of a platform door, it becomes easy to consider a plurality of types of platform doors.

[3.4. Superimposition of 3D VR model]
The control unit 11 acquires the image captured by the monitoring camera 300 based on the user's designation based on the position and orientation of the monitoring camera 300 and the distance and direction from the reference point 501 to the installation position 502. The three-dimensional VR model 700 is superimposed, a superimposed image is formed and displayed on the display unit 14. FIG. 5 shows a state in which a three-dimensional VR model 700 representing a platform door is superimposed on both sides of an island platform.

[3.5. Position and angle adjustment of 3D VR model]
In the superimposed image displayed on the display unit 14, when the three-dimensional VR model 700 representing the home door is not displayed along the installation position 502 (the edge of the home) assumed by the user, There is a case where it is desired to shift and display the position of the portion to get on and off. In preparation for such a case, the image forming apparatus 1 has a function of adjusting the position and angle of the superimposed three-dimensional VR model 700. Specifically, a configuration in which the user can perform an operation of adjusting the position or angle of the three-dimensional VR model 700 via the operation unit 15 while viewing the superimposed image displayed on the display unit 14 is provided. When the control unit 11 receives the operation, the control unit 11 updates the position or the superposition angle of the three-dimensional VR model 700 according to the operation content. Thereby, the user can correct the superimposition position or superimposition angle of the three-dimensional VR model in the superimposed image by a correction operation.

[3.6. Mask processing]
When the 3D VR model 700 is superimposed on the image captured by the monitoring camera 300, an object (a pillar or a structure) that should be in front of the home door is added to the 3D VR model 700 as shown in an area 601 in FIG. May be hidden. Therefore, the image forming apparatus 1 has a mask processing function that displays an object that should be in front without superimposing the three-dimensional VR model 700 on the region designated by the user. By this mask processing, an object (post or structure) that should be in front of the home door is not hidden in the three-dimensional VR model 700 as shown in an area 602 in FIG. Can do.

[3.7. Change marker position]
After the platform door is superimposed based on the position where the marker was first installed, if the platform door is placed in another shape or position (such as the inside of the platform) in the course of study, the marker Appearance problems may occur, such as a home door covering or above. In that case, the position of the marker may be moved to make the structure easy to see. Thus, when the position of the marker is changed, the user again inputs real information indicating the distance and direction from the reference point 501 to the installation position 502 in the real space to the image forming apparatus 1. Then, the control unit 11 specifies again at which position of the captured image the 3D VR model 700 of the home door is to be superimposed based on the input actual information. As described above, since the position of the marker can be freely moved, it is possible to repeatedly change the shape and position of the platform door during the examination process.

[4. Superimposed image formation processing]
Next, the superimposed image forming process by the control unit 11 will be described with reference to FIG. FIG. 6 is a flowchart illustrating a superimposed image forming process performed by the control unit 11 of the image forming apparatus 1. Note that the superimposed image forming process starts when the user performs an operation for starting the formation of a superimposed image. Before the superimposed image forming process is executed, the user designates the type of the home door, the home door parameter, and the construction stage, so that the three-dimensional VR model at any construction stage of the home door having any shape can be obtained. It is assumed that the image forming apparatus 1 accepts these operations by designating whether to superimpose and inputting actual information indicating the shortest distance and direction from the reference point 501 of the marker 500 to the installation position 502. .

  First, the control part 11 acquires the picked-up image which the monitoring camera 300 image | photographed from the monitoring camera system control apparatus 200 (step S1).

  Next, the control unit 11 calculates the position and orientation of the monitoring camera 300 based on the captured image acquired in the process of step S1 (step S2).

  Next, the control unit 11 acquires the shortest distance and direction from the reference point 501 of the marker 500 to the installation position 502 input by the user (step S3).

  Next, the control unit 11 performs a step based on the position / orientation of the monitoring camera 300 calculated in the process of step S2 and the shortest distance and direction from the reference point 501 to the installation position 502 acquired in the process of step S3. The position where the three-dimensional VR model 700 is superimposed in the captured image acquired in the process of S1 is specified (step S4). At this time, the control unit 11 calculates a distance (referred to as “in-image distance”) from the reference point 501 to the installation position 502 in the captured image based on the size of the marker 500 in the captured image and the actual size of the marker 500. To do. Note that the actual size of the marker 500 is stored in the storage unit 12 in advance. In addition, the control unit 11 specifies the in-image coordinate system from the shape of the marker 500 in the captured image, and calculates the in-image direction based on the in-image coordinate system and the direction from the reference point 501 to the installation position 502. And the control part 11 pinpoints the position which superimposes the three-dimensional VR model 700 based on the calculated image inner distance and image inner direction.

  Next, the control unit 11 acquires the three-dimensional VR model 700 designated by the user from the storage unit 12 (step S5).

  Next, the control unit 11 superimposes the three-dimensional VR model 700 acquired by the process of step S5 on the superposition position specified by the process of step S4 (step S6).

  Next, the control unit 11 causes the display unit 14 to display a superimposed image on which the three-dimensional VR model 700 is superimposed in the process of step S6 (step S7).

  Next, the control unit 11 determines whether or not a position / angle correction operation for correcting the position / angle by the user has been detected (step S8). The position / angle correction operation is an operation for the user to correct the position / angle of the 3D VR model 700 in the superimposed image via the operation unit 15 (for example, the 3D VR model 700 to correct the position). And dragging the end of the three-dimensional VR model 700 in the intended direction in order to correct the angle. At this time, when it is determined that the position / angle correction operation is not detected (step S8: NO), the control unit 11 proceeds to the process of step S10. On the other hand, if it is determined that the position / angle correction operation has been detected (step S8: YES), the control unit 11 determines the position / angle of the 3D VR model 700 in the captured image according to the position / angle correction operation. Change (step S9).

  Next, the control part 11 determines whether the mask operation for performing the mask process by a user was detected (step S10). The mask operation is, for example, an operation for masking a part of a portion of the superimposed image on which the three-dimensional VR model 700 is superimposed via the operation unit 15, and an operation for designating a portion to be masked. including. At this time, when it is determined that the mask operation has not been detected (step S10: NO), the control unit 11 proceeds to the process of step S12. On the other hand, when it is determined that the mask operation has been detected (step S10: YES), the control unit 11 performs mask processing on the portion specified as the target portion of the mask processing (step S11). The mask process in the present embodiment is a process for hiding the specified portion of the three-dimensional VR model 700. That is, for the specified portion, the 3D VR model 700 is not displayed, but what is originally displayed in the captured image (what should be in front of the home door) is displayed.

  Next, the control unit 11 determines whether or not a three-dimensional VR model change operation by the user has been detected (step S12). The 3D VR model change operation is an operation for changing the superimposed 3D VR model, and specifically, an operation in which the user newly designates the type of the home door, the home door parameter, and the construction stage. . When it is determined that the three-dimensional VR model change operation has been detected (step S12: YES), the control unit 11 proceeds to the process of step S5 and obtains the three-dimensional VR model specified by the user from the storage unit 12 again. To do. As a result, the subsequent steps S6 and S7 are executed, and a superimposed image on which the newly designated three-dimensional VR model is superimposed is displayed on the display unit 14. On the other hand, when it determines with the control part 11 not detecting 3D VR model change operation (step S12: NO), it transfers to the process of step S13.

  Next, the control unit 11 determines whether or not a marker position changing operation by the user has been detected (step S13). The marker position changing operation is an operation performed when the position of the marker 500 is moved, and is an operation for re-inputting information indicating the shortest distance and direction from the reference point 501 of the marker 500 to the installation position 502. At this time, if the control unit 11 does not determine that the marker position change operation has been detected (step S13: NO), the control unit 11 proceeds to the process of step S14. On the other hand, when it is determined that the marker position change operation has been detected (step S13: YES), the control unit 11 proceeds to the process of step S3, and the installation position 502 from the reference point 501 of the marker 500 input by the user. Get the shortest distance and direction.

  Next, the control unit 11 determines whether or not an end operation by the user has been detected (step S14). The end operation is an operation for ending the superimposed image forming process. At this time, if the control unit 11 does not determine that the end operation has been detected (step S14: NO), the control unit 11 proceeds to the process of step S1. On the other hand, when it is determined that the end operation has been detected (step S14: YES), the control unit 11 ends the superimposed image forming process.

  If “NO” is determined in the process of step S14, the process returns to the process of step S1 and the same process is repeated. However, when the monitoring camera 300 is a fixed point camera whose position and orientation are not changed. Does not need to perform the process of step S2 again in the repeated process. Further, when the monitoring camera 300 is a fixed point camera and the user does not re-input the distance and direction from the reference point 501 of the marker 500 to the installation position 502, the processing in step S3 and step S4 are performed again in the repeated processing. It is not necessary to perform the process. However, when the user re-enters the distance and direction from the reference point 501 of the marker 500 to the installation position 502, it is necessary to perform the process of step S3 and the process of step S4 again. Furthermore, when the user does not change the designation of the three-dimensional VR model 700, the process of step S5 may not be performed again in the repeated process. In this way, by omitting some of the repeated processing, the processing load on the control unit 11 can be reduced. In particular, when the captured image acquired from the monitoring camera system control apparatus 200 is a frame image that forms a moving image, the moving image can be displayed smoothly.

  In the process of step S7, the superimposed image is displayed on the display unit 14. However, when a predetermined operation by the user is detected, the control unit 11 returns the superimposed image to the monitoring camera system control apparatus 200. The superimposed image is displayed on the monitor 400. Thereby, installation of a platform door can be examined using the superimposed image displayed not only on the display unit 14 but also on the monitor monitor 400.

  As described above, in the image forming apparatus 1, the storage unit 12 (an example of “storage unit”) stores the three-dimensional VR model 700 representing the home door (an example of “structure”), and the control unit 11 (“ An example of “image acquisition means” and “formation means” acquires a photographed image including the site where the home door is installed and the marker 500 within the photographing range, and uses the marker 500 appearing in the photographed image as a reference to the photographed image in three dimensions. A superimposed image in which the VR model 700 is superimposed is formed.

  Therefore, according to the image forming apparatus 1, the 3D VR model 700 is superimposed on the basis of the marker 500 that appears in the captured image. Therefore, it is not necessary to survey the position where the 3D VR model 700 is superimposed. A superimposed image in which the three-dimensional VR model 700 of the structure is superimposed at an accurate position can be formed.

  Further, in the image forming apparatus 1, the control unit 11 (an example of “actual information acquisition unit” and “intra-image distance calculation unit”) has a reference point 501 (an example of “predetermined position with reference to a marker”) in the real space. Real information indicating the direction and the shortest distance (an example of “actual distance”) from the image is acquired, and the in-image distance in the captured image corresponding to the actual distance is calculated based on the direction and distance indicated by the actual information, and the captured image A superimposed image is formed by superimposing the three-dimensional VR model 700 on the basis of the reference point 501 on the basis of the direction indicated by the actual information and the distance in the image. As a result, only the direction from the reference point 501 and the actual distance are measured, and the position for superimposing the 3D VR model 700 on the captured image is determined without requiring the labor of surveying the 3D absolute coordinates. be able to. In addition, since the direction and actual distance can be measured more easily than when measuring three-dimensional absolute coordinates, a plurality of installations can be performed by inputting a plurality of directions and actual distances for the position where the platform door is installed. It becomes easy to consider the position.

  Furthermore, in the image forming apparatus 1, the control unit 11 (an example of “calculation unit”) calculates a three-dimensional position and orientation from the shape of the marker 500 in the captured image, and based on the calculation result, the three-dimensional position and orientation are calculated. A superimposed image in which the VR model 700 is superimposed is formed. As a result, the three-dimensional VR model 700 can be superimposed at an appropriate position in the captured image.

  Furthermore, in the image forming apparatus 1, the control unit 11 (an example of “correction operation content acquisition unit”) displays the operation content of the position / angle correction operation for correcting the superposition position or superposition angle of the three-dimensional VR model 700 in the superposition image. According to the acquired operation content, the superimposition position or superimposition angle of the three-dimensional VR model 700 in the superimposition image is updated. Thereby, the user can correct the superimposition position or superimposition angle of the three-dimensional VR model 700 in the superimposed image by a correction operation.

  Furthermore, in the image forming apparatus 1, the storage unit 12 stores a three-dimensional VR model 700 representing a plurality of types of home doors, and the control unit 11 (an example of “selection operation content acquisition unit”) The operation content of the selection operation (operation to specify the type of the home door, the home door parameter, and the construction stage) for selecting the three-dimensional VR model 700 representing the home door is acquired and specified by the acquired operation content A superimposed image is formed by superimposing a three-dimensional VR model representing a type of home door. Thereby, it is possible to form a superimposed image in which the three-dimensional VR model 700 representing a plurality of types of platform doors is superimposed, and it becomes easy to consider a plurality of types of structures when considering the installation of platform doors. .

S Image forming system 1 Image forming apparatus 11 Control unit 12 Storage unit 13 Interface unit 14 Display unit 15 Operation unit 200 Monitoring camera system control device 300 Monitoring camera 400 Monitoring monitor 500 Marker 501 Reference point 502 Installation position 503 Distance 700 3D VR model NW network

Claims (7)

Storage means for storing a three-dimensional VR (Virtual Reality) model representing a structure;
An image acquisition means for acquiring a photographed image including a site where the structure is installed and a marker within a photographing range;
Forming means for forming a superimposed image in which the three-dimensional VR model is superimposed on the photographed image with reference to a marker appearing in the photographed image;
Real information acquisition means for acquiring real information indicating a direction and an actual distance from a predetermined position based on the marker in the real space ;
Based on a direction and an actual distance indicated by the actual information, an image inner distance calculating unit that calculates an image inner distance in the captured image corresponding to the actual distance,
The forming unit forms a superimposed image in which the three-dimensional VR model is superimposed on the basis of the direction indicated by the actual information and the distance in the image based on the predetermined position in the captured image. Image forming apparatus. The image forming apparatus according to claim 1,
A calculation unit that calculates a three-dimensional position and orientation from the shape of the marker in the captured image acquired by the image acquisition unit;
The image forming apparatus, wherein the forming unit forms a superimposed image in which the three-dimensional VR model is superimposed on the captured image based on a calculation result calculated by the calculating unit. The image forming apparatus according to claim 1, wherein
A correction operation content acquisition means for acquiring operation details of a correction operation for correcting the superposition position or superposition angle of the three-dimensional VR model in the superimposed image;
The image forming apparatus, wherein the forming unit updates a superimposed position or a superimposed angle of the three-dimensional VR model in the superimposed image according to the operation content acquired by the correction operation content acquiring unit . The image forming apparatus according to any one of claims 1 to 3,
The storage means stores a three-dimensional VR model representing a plurality of types of structures.
A selection operation content acquisition means for acquiring the operation content of the selection operation for selecting a 3D VR model representing any type of the structure;
The image forming apparatus, wherein the forming unit forms a superimposed image on which a three-dimensional VR model representing the structure of a type specified by the operation content acquired by the selection operation content acquisition unit is superimposed . An image forming apparatus according to any one of claims 1 to 4,
The structure is a platform door;
The site is a home,
The image forming apparatus, wherein the three-dimensional VR model is constructed according to a condition regarding a home . An image forming system comprising: the image forming apparatus according to claim 1; and a camera that captures the captured image.   A computer included in an image forming apparatus including a storage unit that stores a three-dimensional VR (Virtual Reality) model representing a structure;
  An image acquisition means for acquiring a captured image including a site where the structure is installed and a marker within the imaging range;
  Forming means for forming a superimposed image in which the three-dimensional VR model is superimposed on the photographed image with reference to a marker appearing in the photographed image;
  Real information acquisition means for acquiring real information indicating a direction and an actual distance from a predetermined position based on the marker in the real space;
  An image forming program that functions as an intra-image distance calculation unit that calculates an intra-image distance in the captured image corresponding to the actual distance based on a direction and an actual distance indicated by the actual information,
  The computer functioning as the forming unit forms a superimposed image in which the three-dimensional VR model is superimposed based on the direction indicated by the real information and the distance in the image based on the predetermined position in the captured image. An image forming program.