JP6214981B2 - Architectural image display device, architectural image display method, and computer program - Google Patents

Description

  The present invention relates to a display device that displays an image of a structure such as a building, exterior, or furniture. In particular, an architectural image display device, an architectural image display method, and a computer capable of superimposing and displaying an image based on the three-dimensional information of the structure without causing a sense of incongruity on a captured image actually taken in two dimensions. Regarding the program.

  When selling homes, it is very useful to let customers imagine what kind of houses are actually built. Especially in private houses, it is beneficial to show what is close to reality in terms of appearance. In addition, regarding the appearance and exterior of the house, harmony with the actual surrounding scenery is also emphasized.

  Conventionally, various ideas have been made for housing sales support, such as a technique for superimposing and displaying an appearance model of a house to be completed or a CG image based on three-dimensional information on a live-action image such as a planned construction site. . In particular, Patent Document 1 discloses a method of photographing an appearance model so as to match the direction and position based on information on a photographing direction and a photographing position of a photographed image, and displaying them in a superimposed manner. Patent Document 2 discloses a method of calculating a photographed viewpoint based on edge detection in a real image and synthesizing a CG image without a sense of incongruity.

JP 05-242204 A Japanese Patent Application Laid-Open No. 07-200650

  It is useful to superimpose and display a CG image on various two-dimensional live-action images taken from unfixed distances and angles, but displaying them without feeling a sense of incongruity is not possible. Difficult in terms. It is possible to obtain the photographing direction or the like in advance as in the method disclosed in Patent Document 1, or to calculate the viewpoint as in the method disclosed in Patent Document 2. There are actual photographed images that can be planned locations, which are photographed from various angles or distances including those photographed by the customer, and it is difficult to calculate or determine the photographing direction and the like in advance. In addition, it is assumed that most of the persons who perform the display operation are sales persons related to home sales. Therefore, troublesome operations or time-consuming processes reduce convenience.

  The present invention has been made in view of such circumstances, and an architectural image display device and an architectural image display method capable of easily displaying an image based on three-dimensional information about a house superimposed on a real image without a sense of incongruity. And to provide a computer program.

The architectural image display device according to the present invention creates an appearance image of a building or a structure arranged inside or outside the building based on the data of the three-dimensional shape of the building or the structure, and the building of the building In the architectural image display device that displays the appearance image on the display unit by superimposing the appearance image on the captured image obtained by photographing the planned location or the planned arrangement location of the structure, the display position and the scale of the appearance image of the building or structure are displayed. Based on the settings and the setting of the viewpoint to the building or structure, respectively, and the setting image received by the receiving means, the appearance image of the building or structure is displayed on the building or structure. a generating means for re-created based on data of a three-dimensional shape, the appearance image said producing means, and means for re-displayed superimposed on the shooting image, of the appearance image of the building or structure, Invisibility Means for accepting an operation for designating an arbitrary location, means for creating a foreground image obtained by making the photographed image translucent, means for displaying the created foreground image superimposed on the photographed image and the appearance image, Means for making an area of the foreground image corresponding to a location designated by the operation when the operation is accepted in a state where the foreground image is displayed superimposed on the photographed image and the appearance image; It is characterized by providing.

  The architectural image display device according to the present invention is based on the display position, scale, and viewpoint of the external appearance image of the building or structure, and the image of a person standing in the vicinity of the building or structure or the vehicle arranged in the vicinity. The image processing apparatus further includes means for displaying an image corresponding to the display position, scale, and viewpoint.

  In the architectural image display device according to the present invention, a two-dimensional plane image having a known width dimension and depth dimension is superimposed on the captured image and displayed on the display unit, and 2 is displayed on the display unit. Means for accepting the setting of the display position and scale of the image of the two-dimensional plane and the setting of the viewpoint for the two-dimensional plane, the coordinate axis orthogonal to the two-dimensional plane based on the setting accepted by the means, and the two-dimensional Means for setting a three-dimensional Cartesian coordinate system having two coordinate axes orthogonal to each other in a plane, and the building or structure of the building or structure is placed on the two-dimensional plane in a space based on the set three-dimensional Cartesian coordinate system. An appearance image is displayed.

  In the architectural image display apparatus according to the present invention, when the setting of the display position of the appearance image is received by the reception unit, the movement of the display position of the appearance image is moved in a direction parallel to the two-dimensional plane. Means for limiting is provided.

  In the architectural image display device according to the present invention, when the setting of the viewpoint to the building or structure is received by the receiving unit, the movement of the viewpoint is changed to the movement of the viewpoint around the coordinate axis orthogonal to the two-dimensional plane. It is further characterized by further comprising means for limiting to the above.

  The architectural image display device according to the present invention has a bottom surface on the two-dimensional plane, and has a display unit that displays a rectangular parallelepiped image having a known width dimension, depth dimension, and height dimension, and the display. Means for receiving the setting of the display position and scale of the image of the cuboid displayed on the section, and the setting of the viewpoint for the cuboid, and the image of the cuboid and the width of the cuboid based on the setting received by the means And a means for displaying the dimension, the depth dimension, and the height dimension.

The architectural image display method according to the present invention creates an appearance image of a building or a structure arranged inside or outside the building based on the three-dimensional shape data of the building or the structure, and the building of the building In the architectural image display method for superimposing and displaying the appearance image on a photographed image obtained by photographing the planned location or the planned placement location of the structure, setting the display position and scale of the external image of the building or structure, and The viewpoint setting for the building or structure is received, and the appearance image of the building or structure is recreated based on the three-dimensional shape data of the building or structure based on the received setting. , the appearance image recreated, superimposed on the shooting image is displayed again, among the appearance image of the building or structure, receiving an operation for specifying an arbitrary position to be invisible, the captured image Translucent The foreground image is created, the created foreground image is superimposed and displayed on the captured image and the appearance image, and the foreground image is superimposed and displayed on the captured image and the appearance image. Is received, the foreground image area corresponding to the location designated by the operation is made opaque .

  The architectural image display method according to the present invention creates a foreground image in which the photographed image is translucent, displays the created foreground image superimposed on the photographed image and the appearance image, and displays the building or structure. In the appearance image, designation of an arbitrary portion to be invisible is received, and the foreground image in the portion where the designation is accepted is made opaque.

A computer program according to the present invention creates an external appearance image of a building or a structure arranged inside or outside a building on a computer connected to a display unit based on the three-dimensional shape data of the building or structure. In the computer program for superimposing the appearance image on a photographed image obtained by photographing the planned building location of the building or the planned layout location of the structure and outputting the image to the display unit, the computer or the building or structure Processing for accepting the setting of the display position and scale of the appearance image of the object and the setting of the viewpoint to the building or structure, respectively, and the appearance image of the building or structure based on the accepted setting process of re-created based on the data of the three-dimensional shape of the object or structure, process the appearance images created and re-displayed by being superimposed on the shooting image, the building Alternatively, a process of accepting an operation for designating an arbitrary portion to be invisible among the appearance images of the structure, a process of creating a foreground image in which the photographed image is made translucent, the created foreground image as the photographed image and the appearance When the operation is accepted in a state in which the foreground image is superimposed and displayed on the image, and the foreground image is superimposed and displayed on the captured image and the appearance image, it corresponds to the location specified by the operation A process of making the area of the foreground image opaque is executed.

  The computer program according to the present invention includes a process for creating a foreground image in which the photographed image is made translucent, a process for displaying the created foreground image superimposed on the photographed image and the appearance image, and the building. Alternatively, a process of receiving designation of an arbitrary portion to be invisible in an appearance image of a structure and a process of making the foreground image opaque at the location of accepting the designation are executed.

In the present invention, an external appearance image of a building or a structure including facilities arranged inside and outside the building is created based on the data of the three-dimensional shape of the building or the structure, and the building planned construction site or the structure When the created appearance image is superimposed on the captured image obtained by actually capturing the planned placement location, the display position and scale of the appearance image can be changed, and the viewpoint of the appearance image can be set.
The building includes a house and a store, and the structure includes furniture arranged in the building, facilities such as a system kitchen, or so-called external structures such as an approach formed outside the building, a gate, and the like.

  In the present invention, an appearance image of a building or structure is displayed, and the image of the building or structure person or car is displayed together according to the fact that the display position, scale, and viewpoint can be changed. Thereby, it becomes a standard of the change of a scale and an angle especially.

  In the present invention, when an appearance image of a building or a structure is superimposed on a captured image, it is possible to arbitrarily designate a portion to be invisible in the appearance image. Of the captured images that are superimposed on the exterior image of the building or structure, the object that should be seen in the foreground when the building is built or the structure is placed is displayed in front of the building or structure It becomes possible to make it.

  In the present invention, an appearance image of a building or a structure is displayed on a two-dimensional plane corresponding to the ground or floor. Moreover, when the movement operation with respect to the external appearance image of a building or a structure is received, the movement is restrict | limited to the movement adsorb | sucked to the ground or the floor surface.

  In the present invention, a rectangular parallelepiped image whose display position, size, and orientation can be arbitrarily changed is displayed together with the width dimension, the depth dimension, and the height dimension. It is possible to measure dimensions of various structures arranged in the appearance image.

  According to the present invention, it is possible to change the size, scale, and viewpoint of the appearance image of a building or a structure arranged inside or outside the building on the captured image. Thereby, the external appearance image of a building can be easily superimposed on the captured image.

It is a schematic diagram which shows the external appearance of the building image display apparatus in Embodiment 1. 1 is a block diagram illustrating a configuration of an architectural image display device according to Embodiment 1. FIG. It is a flowchart which shows an example of the process sequence by the control part of a building image display apparatus. It is a flowchart which shows an example of the process sequence by the control part of a building image display apparatus. It is explanatory drawing which shows the example of the initial screen displayed on the display part of a building image display apparatus. It is explanatory drawing which shows the example of the main screen displayed on the display part of a building image display apparatus. It is explanatory drawing which shows the other example of the main screen displayed on the display part of a building image display apparatus. It is explanatory drawing which shows the example of the person image displayed with the external appearance image of a house model. It is explanatory drawing for demonstrating the function of the "eraser tool" with respect to the external appearance image of a house model. It is explanatory drawing for demonstrating the function of the "eraser tool" with respect to the external appearance image of a house model. It is a schematic diagram which shows the example of a screen when the tool icon of "eraser tool" is selected. It is a schematic diagram which shows the example of a screen when the tool icon of "eraser tool" is selected. It is a flowchart which shows an example of the procedure of the reception process of site information. It is explanatory drawing which shows the example of the main screen displayed on the display part of the building image display apparatus which concerns on Embodiment 2. FIG. It is explanatory drawing which shows the example of the setting screen displayed on the display part of a building image display apparatus, when performing space recognition processing. It is a flowchart which shows an example of the process sequence of the space recognition process by a control part. It is a schematic diagram which shows the example of a screen which has arrange | positioned the external appearance image of a house model. It is a flowchart which shows an example of the process sequence of the process which concerns on the movement restriction | limiting by a control part. It is explanatory drawing explaining the setting process of a light source. It is explanatory drawing which shows the structure of a building image display system. It is explanatory drawing which shows the example of a screen in the case of displaying the external appearance image of the furniture installed in a house. It is explanatory drawing which shows the example of a screen at the time of displaying merchandise information. It is explanatory drawing which shows the example of the setting screen displayed on the display part of a building image display apparatus, when performing space recognition processing. It is explanatory drawing which shows the example of the setting screen displayed on the display part of a building image display apparatus, when performing space recognition processing. It is explanatory drawing which shows the example of a screen at the time of a dimension measurement.

  Hereinafter, the present invention will be specifically described with reference to the drawings showing embodiments thereof.

Embodiment 1 FIG.
FIG. 1 is a schematic diagram showing an external appearance of the architectural image display device 1 according to the first embodiment, and FIG. 2 is a block diagram showing a configuration of the architectural image display device 1 according to the first embodiment. The architectural image display device 1 uses a tablet PC (Personal Computer). The architectural image display device 1 includes a control unit 10, a temporary storage unit 11, a storage unit 12, a display unit 13, an operation unit 14, a communication unit 15, and an imaging unit 16.

  The control unit 10 uses a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The control unit 10 causes the tablet PC to operate as the building image display device 1 by reading and executing the control program 1P stored in the storage unit 12. Moreover, the control part 10 implement | achieves a dynamic drawing process using GPU, when displaying the external appearance image based on the three-dimensional shape of a building on the display part 13 so that it may mention later. The temporary storage unit 11 uses a RAM such as a DRAM (Dynamic Random Access Memory). The temporary storage unit 11 temporarily stores information generated by the processing of the control unit 10.

  The storage unit 12 uses a flash memory. In addition to the control program 1P read by the control unit 10, the storage unit 12 is a captured image that the control unit 10 displays on the display unit 13, data of a three-dimensional shape of the building, building information such as a plan view, and a display screen. The image of the material is stored. Note that the storage unit 12 may use a storage device other than the flash memory.

  The display unit 13 uses a liquid crystal display with a built-in touch panel. The control unit 10 causes the display unit 13 to display various operation screens including images such as characters and icons. The display unit 13 may be other than a liquid crystal display or may not be a touch panel built-in type.

  The operation unit 14 uses a touch panel built in the display unit 13 and a button group provided on the housing of the tablet PC. The operation unit 14 notifies the control unit 10 of contact information by the operator and position information of the contact location on the touch panel. Further, the operation unit 14 notifies the control unit 10 of information such as button presses and press times.

  The communication unit 15 realizes wireless LAN connection. The control unit 10 can capture various kinds of information, for example, a live-action image, via the communication unit 15 and store it in the storage unit 12. Further, the control unit 10 can output the created image to the external storage device via the communication unit 15.

  The imaging unit 16 uses a complementary metal oxide semiconductor (CMOS) image sensor. The imaging unit 16 is provided at the corners of the front and back surfaces of the tablet PC with a display screen, and the operator can take a picture through the operation unit 14 while checking on the display screen. The control unit 10 can acquire a captured image captured by the imaging unit 16 and store the captured image in the storage unit 12. The imaging unit 16 may use not only a CMOS image sensor but also a CCD (Charge Coupled Device) image sensor.

  In the architectural image display device 1 configured as described above, the control unit 10 displays the captured image stored in the storage unit 12 or the image captured by the imaging unit 16 on the display unit 13 based on the control program 1P. Let And the control part 10 produces the image of the external appearance of a house model based on the three-dimensional information of the some house model memorize | stored in the memory | storage part 12, and superimposes it on the above-mentioned captured image, and displays it. And the control part 10 receives the change of a display position, the change of a scale, and the change of a viewpoint with respect to the image of the external appearance of the house model displayed on the display part 13 from operation on the touch panel of the operation part 14, and displays according to operation. To change. Thus, the operator can display an appearance image of the house model superimposed on an arbitrary captured image, change the size, position, etc., and adjust it so as to match and become familiar with the captured image. .

  A processing procedure performed by the control unit 10 will be described below with reference to flowcharts and screen examples displayed in each process.

  3 and 4 are flowcharts illustrating an example of a processing procedure performed by the control unit 10 of the architectural image display device 1. When receiving an instruction to execute the control program 1P on the tablet PC, that is, when instructed to start the application, the control unit 10 executes the following processing procedure.

  The control unit 10 displays an initial screen (step S101) and accepts selection of a captured image (step S102). In step S102, the control unit 10 determines whether a captured image photographed by the imaging unit 16 is selected (step S103).

  FIG. 5 is an explanatory diagram illustrating an example of an initial screen displayed on the display unit 13 of the architectural image display device 1. On the initial screen, an icon 131 for selecting a captured image captured by the imaging unit 16 and a storage unit 12 are stored in advance as a captured image (background image) on which an appearance image of the house model is superimposed. An icon 132 for selecting a captured image is displayed.

  When the touch panel on the icon 131 is touched and the control unit 10 determines that the captured image captured by the imaging unit 16 has been selected (S103: YES), the control unit 10 activates the imaging application by the imaging unit 16 (step S104). As a result, a monitor screen of a captured image by the imaging unit 16, a shutter button, a cancel button, and the like are displayed on the display unit 13. The operator operates the button to take a picture. The control unit 10 acquires a captured image captured by the imaging unit 16 (step S105), and stores the acquired captured image in the storage unit 12 (step S106).

  On the other hand, when the touch panel on the icon 132 is touched and the captured image stored in the storage unit 12 is selected (S103: NO), the control unit 10 selects the captured image stored in the storage unit 12 in advance. Is received (step S107). In step S107, the control unit 10 causes the display unit 13 to display a thumbnail list of captured images stored in advance in the storage unit 12, and accepts selection by the operation unit 14 (touch panel). The operator selects a captured image from the thumbnail list. The control unit 10 specifies the selected captured image (step S108).

  The control unit 10 reads the captured image stored in Step 106 or the captured image specified in Step S108 from the storage unit 12 (Step S109). The control unit 10 displays a main screen with the read captured image as a background image (step S110).

  FIG. 6 is an explanatory diagram illustrating an example of a main screen displayed on the display unit 13 of the architectural image display device 1. The thin line shown in FIG. 6 represents the captured image. The main screen includes a housing model selection tab 133, a toolbar 134 including tool icons 41 to 45, a person display button 135, a selection history 136, and an icon 137 for displaying a corresponding plan view. The selection history 136 is an icon for selecting again the house model selected so far. When the house model is not selected, it may be hidden. Further, the history can be deleted by performing a predetermined operation on the selection history 136. Of the tool icons 41 to 45, the tool icon 41 is an icon for re-selecting a captured image (S107), and the tool icon 42 is an icon for starting up an “eraser tool” described later. The tool icon 43 is an icon for hiding various buttons such as the selection tab 133 and the tool bar 134 on the main screen and displaying the captured image and the appearance image of the house model in full screen. The tool icon 44 is an icon for switching the display method of the appearance image of the house model. Each time the tool icon 44 for switching the display method is selected, the control unit 10 sequentially switches whether to display the bottom surface of the house model (the circumscribed rectangle of the frontage and the depth) or not to display the appearance image. When the bottom surface is displayed, it becomes a reference for matching the appearance image with the captured image by “move”, “rotate”, and “enlarge / reduce” operations. However, when displaying as a natural composite image, it is desirable that the bottom surface is not displayed, so switching of the display method is effective. The tool icon 45 is an icon for returning to the initial screen. When the house model for displaying the appearance image is not selected, the toolbar 134, the person display button 135, and the icon 137 may be hidden. The toolbar 134 can be minimized.

  The control unit 10 determines whether or not the selection tab 133 has been pressed on the main screen (step S111). When it is determined that the selection tab 133 has been pressed (S111: YES), the control unit 10 displays a list of housing models (step S112) and accepts selection of a housing model (step S113). The control unit 10 reads out the three-dimensional shape data and the plan view information of the selected house model from the storage unit 12 (step S114), and has an external view with a predetermined size based on the three-dimensional shape data. An image is created (step S115) and displayed superimposed on the captured image of the background image (step S116).

  FIG. 7 is an explanatory diagram illustrating another example of the main screen displayed on the display unit 13 of the architectural image display device 1. In the example illustrated in FIG. 7, the appearance image (thick line) of the selected house model is superimposed on the captured image. In the architectural image display device 1, the operator moves his / her finger on the touch panel on the appearance image, thereby changing the display position and scale of the appearance image, and rotating the appearance image around the vertical or horizontal direction of the screen. Is possible. Note that the display position can be changed, that is, the appearance image “move” operation can be performed by the operator sliding two fingers on the display unit 13 in the same direction. The scale change, that is, the “enlargement / reduction” operation can be performed by the operator opening or closing two fingers on the appearance image. The change of the viewpoint of the appearance image, that is, the “rotation” operation can be performed by the operator sliding one finger on the display unit 13 in the vertical direction or the horizontal direction on the display image.

  The control unit 10 determines whether or not the touch panel on the appearance image has been touched (step S117). When it is determined that the touch panel on the appearance image has been touched (S117: YES), the control unit 10 determines whether the operation is “movement” of the appearance image (step S118). When the controller 10 detects contact at two locations simultaneously in step S118 and detects that the contact location has moved (dragged) in the same direction while maintaining contact, the controller 10 determines that it is “move”. . When it is determined that the movement is “movement” (S118: YES), the control unit 10 identifies a movement destination that is a non-contact portion (step S119), and stores the movement destination position, scale, and viewpoint angle. (Step S120). The control unit 10 may store parameters as the position and viewpoint angles, for example, with the center of the screen as the origin, the normal to the screen as the z axis, the horizontal direction as the x axis, and the vertical direction as the y axis. The control unit 10 redisplays the appearance image at the position of the movement destination (step S121), and returns the process to step S117.

  When it is determined that the operation is not “movement” (S118: NO), the control unit 10 determines whether the operation is “rotation” (step S122). In step S122, the control unit 10 detects contact at one location, and determines that the location is “rotation” when it is detected that the location has moved (dragged) while being touched. When determining that the rotation is “rotation” (S122: YES), the control unit 10 specifies the rotation angle from the current viewpoint (initially the front) according to the moving direction and moving distance of the contact location ( In step S123), the position, scale, and viewpoint angle of the changed appearance image are stored in the temporary storage unit 11 (S120). The control unit 10 redisplays the appearance image at the changed viewpoint angle (S121), and returns the process to step S117.

  When it is determined that the operation is not “rotation” (S122: NO), the control unit 10 determines that the operation is “expansion / reduction”. In step S122, the control unit 10 detects contact at two locations, and when the location moves (drags) in the opposite direction while maintaining contact, distinguishes it from “rotation”. When the control unit 10 determines that it is “enlargement / reduction” (S122: NO), it specifies the scale after the change from the current scale according to the moving distance of the contact location (step S124), and the appearance after the change The image position, scale, and viewpoint angle are stored in the temporary storage unit 11 (S120). The control unit 10 redisplays the appearance image at the scale after the change (S121), and returns the process to step S117. Note that, in the processing of steps S118 to S121, the control unit 10 continuously redisplays the appearance image while the “movement”, “rotation”, and “enlargement / reduction” operations are performed.

  When the touch panel other than the appearance image is touched in step S117 (S117: NO), the control unit 10 determines whether or not the person display button 135 is selected according to the touched location (step S125). . If the control unit 10 determines that the touch panel on the person display button 135 is touched and selected (S125: YES), the controller 10 switches between displaying and hiding the person image (step S126), and returns the process to step S117.

  FIG. 8 is an explanatory diagram illustrating an example of a person image displayed together with the appearance image of the house model. Reference numeral 50 in FIG. 8 denotes a displayed person image. The person image 50 is displayed semi-transparently with a predetermined α value. The person image 50 is displayed at a position, scale, and viewpoint angle corresponding to the position, scale, and viewpoint angle of the displayed appearance image. That is, the coordinates from a predetermined position (distance from the center or the direction and distance on the plane from the position of the doorknob of the entrance) in the three-dimensional shape of the house model, assuming that it exists at a predetermined distance with respect to the house model Information is stored in association with each other. Therefore, the position of the person image 50 is changed with the change of the position of the appearance image of the house model, the position and the scale of the person image 50 are changed with the change of the scale of the appearance image, and the viewpoint of the appearance image is changed. As the angle changes, the viewpoint angle of the person image 50 is also changed. As a result, the “moving”, “rotating”, and “enlarging / reducing” operations are performed so that the size of the human image 50 does not cause a sense of incongruity on the captured image of the background image. The display position, the scale, and the viewpoint are adjusted and become familiar with the captured image of the planned construction site. Even if information such as a shooting direction and a shooting distance of a captured image is unknown, the operator can adjust intuitively so as not to cause a sense of incongruity, and convenience is high. The image displayed together with the appearance image of the house model is not limited to the person image 50, but may be an image of an object whose size is easily recognized with respect to the captured image as the background image. It may be a point image.

  Returning to the flowcharts of FIGS. 3 and 4, the description will be continued. When determining that the person display button 135 is not selected in step S125 (S125: NO), the control unit 10 determines whether or not the icon 42 for activating the “eraser tool” in the tool bar 134 has been selected. Judgment is made (step S127). When it is determined that the “eraser tool” has been selected (S127: YES), the control unit 10 accepts designation of an erasure location for the displayed external image (step S128). The control unit 10 deletes the designated portion in the displayed appearance image (step S129), and returns the process to step S117.

  9 and 10 are explanatory diagrams for explaining the function of the “eraser tool” for the appearance image of the house model. FIG. 9 shows a state in which the function of the “eraser tool” is being used, and FIG. 10 shows a result of processing by the “eraser tool”. In FIG. 9, an appearance image indicated by a thick line is superimposed on a captured image indicated by a thin line. When the tool icon 42 of the “eraser tool” is selected, the control unit 10 performs a process of making the contact location on the appearance image invisible when detecting the contact of the operator's finger at an arbitrary position on the appearance image. Do. The contact range is represented by a very thick line in FIG. As a result, as shown in FIG. 9, a captured image of the background can be seen at the contact location. The captured image of the background may include a neighboring house, an outer wall, and planting that should be seen in front of the house model. In this case, it is uncomfortable to simply superimpose the appearance image of the house model on the captured image. On the other hand, as shown by the broken line in FIG. 10, the “eraser tool” makes it possible to see what is to be seen in front, so that the house model matches the background image and becomes a natural composite image. .

  The size of the range of the erased part corresponding to the contacted part (so-called eraser shape and size) can be specified with an eraser and a reclaim pen, and can be changed in a straight line or an arbitrary curve, and the size in multiple stages. It is.

  Specifically, when the tool icon 42 of the “eraser tool” is selected, the control unit 10 duplicates the background captured image to make it translucent and superimpose it on the appearance image. Here, “translucent” means, for example, that the transparency of the image is made relatively high by changing the α value of each pixel constituting the image, and includes that it is made completely transparent. Thereby, it looks as if a semi-transparent appearance image is superimposed on the captured image. Next, the control unit 10 duplicates the erased portion received by touching the finger from the copied captured image displayed in the foremost (semi-transparent), further superimposes it, and makes it opaque. Here, the opacity refers to, for example, changing the α value of each pixel constituting the image to make the transparency of the image relatively low, and includes making it completely opaque. As a result, the captured image and appearance image of the background image are not changed at all, and the appearance image appears to be invisible only at the erased portion. In addition, regarding the place designated by selecting the reproduction pen, it is possible to return to the state before erasing with the eraser by making the designated part of the most opaque background image transparent again. The designation of the erasure location may be a method such as accepting at all the contact locations of the finger, or accepting a point identifying a graphic such as a straight line, a rectangle, and a circle as the erasure range at the contact location. In addition, other methods may be used to partially invisible the appearance image of the house model.

  Further, when the tool icon 42 of “eraser tool” is selected, the control unit 10 may display a toolbar for controlling the function of the “eraser tool” on the display unit 13. FIGS. 11 and 12 are schematic diagrams showing examples of screens when the tool icon 42 of the “eraser tool” is selected.

  When the tool icon 42 is selected on the main screen displayed on the display unit 13 of the building image display device 1, the control unit 10 activates the “eraser tool” and functions the “eraser tool” instead of the toolbar 134. A toolbar 420 for controlling the image is displayed on the screen. Of the tool icons 421 to 428 included in the tool bar 420, the tool icon 421 is an icon for switching a method for specifying a range to be invisible. Each time the tool icon 421 is selected, the range to be invisible is sequentially switched, for example, to a range of a trajectory drawn by a free curve, a range of a trajectory drawn by a straight line, and a range surrounded by each side of a specified polygon. It is configured. The tool icon 422 is an icon for functioning as the above-mentioned “eraser tool”, and the tool icon 423 is an icon for functioning as a playback pen for re-visualizing (revitalizing) a portion invisible by the “eraser tool”. . Tool icons 424a and 424b are icons for switching the shape of the eraser to a rectangle or a circle. The tool icons 425a and 425b are icons for enlarging or reducing the size of the eraser. Each time the tool icon 425a is selected, the size of the eraser gradually increases to a predetermined size, and the tool icon 425b Each time it is selected, the size of the eraser is configured to gradually decrease to a predetermined size.

  When the playback pen is selected by the tool icon 423, the tool icon 421 functions as an icon for switching a range to be restored by the playback pen, and the tool icons 424a and 424b and the tool icons 425a and 425b are respectively played back. It functions as an icon for switching the shape and size of the pen.

  The tool icon 426 is an icon for canceling the process executed immediately before, and the tool icon 427 is an icon for canceling all processes executed after the “eraser tool” is activated. The tool icon 428 is an icon for returning to the original toolbar 134.

  FIG. 12 shows a screen example when a mode for making the inside of the polygon invisible is selected by the tool icon 421. When the operator selects this mode, the operator designates positions of three or more points on the screen. The control unit 10 of the building image display device 1 displays a polygon by displaying the sides connecting the points designated by the operator on the display unit 13, and within the range surrounded by the sides of the polygon, the house model Processing to make the appearance image of invisible. In addition, after executing the process of making invisible, the change of the position of each vertex of the polygon may be received, and the range to be made invisible may be arbitrarily changed.

  Returning to the flowcharts of FIGS. 3 and 4, the description will be continued. If the control unit 10 determines in step S127 that the tool icon 42 of the eraser tool has not been selected (S127: NO), whether the selection tab 133 or the selection history 136 has been touched and another house model has been selected. It is determined whether or not (step S130). When it is determined that another housing model has been selected (S130: YES), the control unit 10 displays a list of housing models in the storage unit 12 or a list of recently selected housing models and accepts the selection (Step S130). S131). The control unit 10 recreates the appearance image of the selected other house model based on the position, scale, and viewpoint angle stored in step S120 (step S132), redisplays it (S121), and performs processing. It returns to step S117. If the process of step S120 is not executed, the re-creation in step S130 is performed at the initial center position with the front as the viewpoint.

  In step S130, when it is determined that another house model has not been selected (S130: NO), the control unit 10 determines whether or not the plan view icon 137 has been selected (step S133). When the control unit 10 determines that the plan view icon 137 has been selected (S133: YES), the control unit 10 reads a plan view corresponding to the appearance image of the displayed house model from the storage unit 12, and displays the plan view display unit 13. The display is switched between superimposed display and non-display (step S134), and the process returns to step S117.

  In step S134, the control unit 10 switches whether to display a plan view of the house model so as to be further superimposed on the appearance image or to display a thumbnail (icon). When displaying two plan views such as the first floor plan view and the second floor plan view, the control unit 10 displays them side by side on the display unit 13. When the number of plan views is three or more, the control unit 10 displays all by scrolling in the horizontal direction or the vertical direction.

  When determining in step S133 that the icon 137 in the plan view has not been selected (S133: NO), the control unit 10 determines whether or not termination has been selected (step S135). The control unit 10 determines whether or not the end is selected by pressing the power button of the operation unit 14 or the like. When it is determined that the end is not selected (S135: NO), the control unit 10 returns the process to step S117, and when it is determined that the end is selected (S135: YES), the process ends.

  Note that in the above-described processing procedure, processing related to the tool icon 43 for full screen, processing related to the tool icon 44 for switching the display method, and processing related to the icon 45 for returning to the initial screen are omitted.

  Among the processing procedures described above, the control unit 10 may perform the following processing before detecting the pressing of the selection tab 133 in step S111. FIG. 13 is a flowchart illustrating an example of a procedure of site information reception processing.

  The control unit 10 displays the site information input screen on the display unit 13 (step S21), and accepts the frontage and depth (step S22). The control unit 10 stores the received frontage and depth (step S23), creates a rectangular image based on the received frontage and depth in the same format as the three-dimensional shape of the house model (step S24), and the rectangular image Is superimposed on the captured image and displayed on the display unit 13 (step S25). The control unit 10 receives “movement”, “rotation” and “enlargement / reduction” operations (similar to S118 to 124) with respect to the rectangle (step S26), and temporarily stores the display position, scale, and viewpoint angle of the rectangle. 11 (step S27). The control unit 10 extracts a house model that fits in the size of the site specified from the stored frontage and depth from the house model stored in the storage unit 12 (step S28), and in the flowcharts of FIGS. The process returns to step S111.

  By executing the processing procedure as shown in FIG. 13, after that, when the control unit 10 detects the pressing of the selection tab 133 in step S <b> 111 and displays a list of house models, the control unit 10 extracts in step S <b> 28. Display only the house models that have been set. Thereby, since selection items are narrowed down, the convenience for the operator is enhanced.

Embodiment 2. FIG.
In the first embodiment, the operation unit 14 of the architectural image display device 1 accepts a change in display position, a change in scale, and a change in viewpoint, and the appearance of the house model changed to the display position, scale, and viewpoint desired by the user. However, after the captured image displayed as a two-dimensional image is recognized as a three-dimensional space, the image of the appearance of the house model is placed in the recognized three-dimensional space. It is good also as a structure superimposed on a captured image by doing.
In the second embodiment, a configuration will be described in which a captured image is recognized as a three-dimensional space and an image of the appearance of a house model is superimposed. Note that the hardware configuration inside the architectural image display device 1 is exactly the same as that of the first embodiment, and thus the description thereof will be omitted.

  FIG. 14 is an explanatory diagram illustrating an example of a main screen displayed on the display unit 13 of the architectural image display device 1 according to the second embodiment. On the main screen, a picked-up image (background image) selected in the same procedure as in the first embodiment is displayed, and tools corresponding to various functions provided by the architectural image display device 1 according to the second embodiment A toolbar 50 including icons 51 to 57 is displayed. It is assumed that the actual dimensions (frontage dimensions and depth dimensions) of the site on which the appearance image of the house model is to be superimposed are already known.

  The tool icon 51 is an icon for switching the background image without changing the house model being displayed. The tool icon 52 is an icon for executing a space recognition process described later. The tool icon 53 is an icon for setting a light irradiation direction and the like for the appearance image of the superimposed house model. The tool icon 54 is an icon for activating the above-mentioned “eraser tool” and displaying a toolbar 420 as shown in FIGS. 11 and 12. The tool icon 55 is an icon for causing the storage unit 12 to store an image displayed on the display unit 13 (for example, an image in which an appearance image of a house model is superimposed on a captured image). When storing the image in the storage unit 12, it is preferable to hide various toolbars, various buttons, and the like. The tool icon 56 is an icon for switching the display method, and sequentially switches whether to display the bottom of the house model (the circumscribed rectangle of the frontage and the depth) or not to display the appearance image. The tool icon 57 is an icon for returning to the initial screen.

  FIG. 15 is an explanatory diagram illustrating an example of a setting screen displayed on the display unit 13 of the architectural image display device 1 when executing the space recognition process. When the tool icon 52 is selected on the main screen displayed on the display unit 13 of the building image display device 1, the control unit 10 has a tool bar 520 for receiving settings related to the space recognition processing, and three-dimensionally in the screen. A site grid 525 that defines the space is displayed on the screen. The site grid 525 is a two-dimensional plane having known frontage dimensions (width dimensions) and depth dimensions. For example, the center of the site grid 525 is the origin, the frontage direction is the X axis, the depth direction is the Y axis, and the site grid 525 is This is a reference for defining a three-dimensional space in the screen with the normal direction to the Z axis. Note that initial values (for example, a frontage 10 m and a depth 8 m) are set for the frontage dimensions and the depth dimensions.

  A tool icon 521 provided in the toolbar 520 is an icon for accepting changes in the frontage and depth of the site. When the size of the frontage and depth of the site grid 525 displayed on the screen is different from the actual size of the site, the operator can input the correct size of the site on the input screen displayed through the tool icon 521. The site grid 525 displays the input new frontage size and depth size. The slider 522 is an icon for adjusting the degree of distortion of the site grid 525. The picked-up image displayed as the background image has a different angle of view depending on the lens used in the camera at the time of shooting, so that the distortion caused by the angle of view can be adjusted by the slider 522. The tool icon 523 is an icon for returning the frontage dimension and depth dimension of the site grid 525 and the viewing angle to the initial values. The tool icon 524 is an icon for ending the space recognition processing and returning to the main screen shown in FIG.

  Similarly to the appearance image of the house model, the site grid 525 can change its display position, scale, and orientation by accepting an operator's operation on the touch panel of the architectural image display device 1. For example, when the operator slides two fingers in the same direction on the touch panel, the site grid 525 can be moved in that direction. In addition, the site grid 525 can be reduced (or enlarged) by sliding the two fingers in a direction approaching each other (or a direction away from each other) on the touch panel. Furthermore, when the operator slides one finger on the touch panel, the site grid 525 can be rotated around an axis intersecting the direction. By these operations, the operator can change the display position, scale, and orientation of the site grid 525 so as to match the site in the image. When the site grid 525 is adjusted to match the site in the image and the tool icon 524 is selected, the architectural image display device 1 displays the position of the origin determined by the site grid 525, the X axis, the Y axis, and the Z axis. Each axial direction and the set scale are stored, and the captured image is recognized as a three-dimensional space.

  FIG. 16 is a flowchart illustrating an example of a processing procedure of space recognition processing by the control unit 10. The space recognition process performed by the control unit 10 is executed, for example, before detecting the pressing of the selection tab 133 in step S111 in the flowchart shown in FIG.

  When the control unit 10 receives selection of the tool icon 52 on the touch panel, the control unit 10 displays the site grid 525 on the display unit 13 (step S201). Next, the control unit 10 determines whether or not the dimensions (frontage dimensions and depth dimensions) of the site grid 525 are input through the input screen displayed by the tool icon 521 in FIG. 15 (step S202). When determining that it has been input (S202: YES), the control unit 10 stores the input dimensions of the site grid 525 (step S203). When determining that it has not been input (S202: NO), the control unit 10 stores a preset initial value as a dimension of the site grid 525 (step S204).

  Next, the control unit 10 determines whether or not an operation for the site grid 525 has been received on the touch panel (step S205). When it is determined that an operation for the site grid 525 has been received (S205: YES), the control unit 10 determines whether the operation is “movement” of the site grid 525 (step S206). When the controller 10 detects contact at two locations simultaneously in step S206 and detects that the contact location has moved (dragged) in the same direction while maintaining contact, the controller 10 determines that it is “move”. . When the control unit 10 determines that it is “move” (S206: YES), it specifies a destination that is a non-contact point (step S207), and sets the position, scale, and viewpoint angle of the destination. Store in the temporary storage unit 11. (Step S208). In addition, the control unit 10 redisplays the site grid 525 based on the changed position, scale, and viewpoint angle (step S209).

  If it is determined in step S206 that the operation is not “movement” (S206: NO), the control unit 10 determines whether or not the operation is “rotation” (step S210). In step S210, the control unit 10 detects contact at one location, and determines that it is “rotation” when it detects that the location has moved (dragged) while being in contact. When determining that the rotation is “rotation” (S210: YES), the control unit 10 identifies the angle of the viewpoint according to the moving direction and moving distance of the contact location (step S211), and the position of the site grid 525 after the change The scale and the angle of the viewpoint are stored in the temporary storage unit 11 (S208). In addition, the control unit 10 redisplays the site grid 525 based on the changed position, scale, and viewpoint angle (S209).

  When it is determined in step S210 that the operation is not “rotation” (S210: NO), the control unit 10 determines that the operation is “expansion / contraction”. The control unit 10 detects contact at two locations, determines that the location has moved (dragged) in the opposite direction while maintaining the contact, determines “enlargement”, and maintains the contact in the direction toward each other. Is moved (dragged), it is determined to be “reduced”. The control unit 10 specifies the scale after the change from the current scale according to the moving distance of the contact location (step S212), and stores the position, scale, and viewpoint angle of the changed appearance image in the temporary storage unit 11. Store (S208). In addition, the control unit 10 redisplays the site grid 525 based on the changed position, scale, and viewpoint angle (S209).

  If it is determined in step S205 that an operation for the site grid 525 is not accepted (S205: NO), the control unit 10 stores the current position, scale, and viewpoint angle of the site grid 525 in the temporary storage unit 11. (Step S213).

  Next, the control unit 10 determines whether or not the operation on the site grid 525 has ended (step S214). When the change of the operation on the touch panel is not detected (for example, when the operator's finger is released from the touch panel or the operator stops the finger on the touch panel), the control unit 10 has finished the operation on the site grid 525. to decide. If it is determined that the operation has not ended (S214: NO), the control unit 10 returns the process to step S205.

  When it is determined that the operation on the site grid 525 has ended (S214: YES), the control unit 10 recognizes the three-dimensional space in the captured image based on the position, scale, and viewpoint angle of the site grid 525 after the change. (Step S215). At this time, for example, the control unit 10 sets the center of the site grid 525 as the origin, the frontage direction as the X axis, the depth direction as the Y axis, and the normal direction with respect to the site grid 525 as the Z axis, and a three-dimensional space in the captured image. Can be recognized. Moreover, the control part 10 can calculate the dimension of a house model etc. superimposed on a captured image based on the dimension input about the site grid 525 and the scale memorize | stored in step S208.

  After recognizing the captured image as a three-dimensional space, when receiving a selection of a house model to be superimposed on the captured image, the appearance image of the house model is placed at the origin in the recognized three-dimensional space. At this time, the orientation of the appearance image is adjusted so that the frontage direction of the house model is the X-axis direction and the depth direction is the Y-axis direction, and the appearance image of the house model is such that the bottom surface of the house model contacts the surface of the site grid 525. By displaying, the adjustment by the operator can be saved. FIG. 17 is a schematic diagram illustrating an example of a screen on which an appearance image of a house model is arranged.

  The site grid 525 represents the ground in the captured image, and it is not necessary to display the appearance image of the house model separately from the ground. Therefore, an operation for moving or rotating the appearance image on the touch panel of the architectural image display device 1 is accepted. In this case, the moving direction and the rotating direction may be limited. For example, when an operation for moving the appearance image is received on the touch panel, the control unit 10 converts the movement in the X-axis and Y-axis directions in the three-dimensional space, and the movement in the Z-axis direction is always set to zero. The appearance image may be moved. When an operation for rotating the appearance image is received on the touch panel, it is converted into rotation around the Z axis in the three-dimensional space, and the outline image is rotated with the rotation around the X axis and the Y axis always being zero. You may do it. In this way, by limiting the movement in the Z-axis direction and the rotation around the X-axis and the Y-axis, as shown by the two arrows in FIG. Become.

  In addition, when there is an object (for example, a wall or a wall) that restricts the movement of the appearance image of the house model in the site of the captured image, not only the movement in the Z-axis direction but also the X-axis direction or It is good also as a structure which restrict | limits the movement of a Y-axis direction. For example, the architectural image display device 1 causes the display unit 13 to display a plurality of grid lines parallel to the X-axis direction and a plurality of grid lines parallel to the Y-axis direction on the XY plane, and to display a wall or wall on the touch panel. Accepts selection of grid lines corresponding to. When selection of a grid line parallel to the X axis (Y axis) is received, each point on the appearance image at the movement destination is expressed as follows: Y coordinate of the point on the appearance image <Y coordinate of the grid line (X of each point) When the condition of coordinates <X coordinate of the grid line) is satisfied, the movement adsorbed to the ground is performed, and the Y coordinate of each point on the appearance image ≧ the Y coordinate of the grid line (X coordinate of each point ≧ X coordinate of the grid line) In such a case, the movement range is limited by not transmitting the movement operation to the object of the appearance image.

  In order to simplify the description, selection of a grid line parallel to the X axis (Y axis) is accepted. However, designation of a line in an arbitrary direction within the XY plane is accepted, and an appearance image is set with this line as a boundary. Of course, the movement range may be limited. In the present embodiment, the X coordinate (Y coordinate) of each point on the appearance image is compared with the X coordinate (Y coordinate) of the selected grid line. Objects such as rectangular parallelepipeds and spheres are arranged in a three-dimensional space, contact between the arranged objects and the selected grid line is determined, and whether to limit the movement range is determined according to the determination result. May be.

  FIG. 18 is a flowchart illustrating an example of a processing procedure of processing related to movement restriction by the control unit 10. When setting the movement range for the appearance image of the house model, the control unit 10 displays the grid lines as described above on the display unit 13. The control unit 10 determines whether or not selection of a grid line has been accepted (step S231), and when selection of a grid line is accepted (S231: YES), the X coordinate (Y coordinate) of the selected grid line is temporarily stored. It memorize | stores in the memory | storage part 11 (step S232). When the selection of the grid line has not been received (S231: NO), the control unit 10 executes the processes after step S233, which will be described later, assuming that the movement range is not limited.

  Next, the control unit 10 determines whether or not a movement operation for the appearance image has been received on the touch panel (step S233). When it is determined that the moving operation has been received (S233: YES), the control unit 10 converts the movement of the appearance image received on the touch panel into the movement in the X axis and Y axis directions in the three-dimensional space, and in the XY plane. Is specified (step S234). At this time, by setting the amount of movement in the Z-axis direction to zero, it is possible to move the appearance image while adsorbing to the ground (site grid 525).

  Next, the control unit 10 determines whether or not movement restriction is involved by determining whether or not the X coordinate (Y coordinate) of the selected grid line is stored in the temporary storage unit 11 (step S235). ).

  When it is determined that the movement is restricted (S235: YES), the control unit 10 determines that each point on the appearance image at the movement destination is the X (Y) coordinate of the point of the appearance image <the X (Y) coordinate of the grid line. It is determined whether or not the condition is satisfied (step S236). If it is determined that the condition is satisfied (S236: YES), the control unit 10 stores the position of the movement destination specified in step S234 in the temporary storage unit 11 (step S237), and causes the display unit 13 to redisplay the appearance image. (Step S238). If it is determined that the condition is not satisfied (S236: NO), the control unit 10 limits the moving range by not transmitting the moving operation to the object of the appearance image, and shifts the processing to step S243 described later.

  If it is determined in step S235 that there is no movement restriction (S235: NO), the control unit 10 stores the position of the movement destination specified in step S234 in the temporary storage unit 11 (S237), and displays an appearance image on the display unit. 13 is displayed again (S238).

  If it is determined in step S233 that a movement operation for the appearance image has not been received (S233: NO), the control unit 10 determines whether a rotation operation for the appearance image has been received on the touch panel (step S239). When it is determined that the rotation operation has been received (S239: YES), the control unit 10 converts the rotation operation of the appearance image received on the touch panel into the rotation around the Z axis in the three-dimensional space, and the rotation around the Z axis. An angle is specified (step S240). And the control part 10 calculates the viewpoint angle to an external appearance image based on the specified rotation angle, and memorize | stores the calculated viewpoint angle in the temporary memory part 11 (step S241). Further, the control unit 10 redisplays the appearance image based on the calculated viewpoint angle (step S242). If it is determined in step S239 that a rotation operation for the appearance image has not been received (S239: NO), the control unit 10 executes the process of step S243.

  Next, the control unit 10 determines whether or not the operation on the appearance image is completed (step S243). If the control unit 10 determines that the operation on the appearance image has not ended (S243: NO), the process returns to step S233, and if the control unit 10 determines that the operation on the appearance image has ended (S243: YES), this flowchart. The process by is terminated.

  The tool bar 50 displayed on the main screen of the architectural image display apparatus 1 includes a tool icon 53 for setting the light irradiation direction and the like for the appearance image of the house model that has been arranged. The winning method can be set as appropriate. FIG. 19 is an explanatory diagram for explaining the light source setting process. When the tool icon 53 is selected on the main screen displayed on the display unit 13 of the building image display device 1, the control unit 10 displays a toolbar 530 for receiving settings relating to the light source on the screen. The tool bar 530 includes buttons 531 and 532 for turning on / off the light source, a button 533 for changing the position of the light source, a slider 534 for changing the intensity of light emitted from the light source, and a setting for returning the light source setting to the initial value. A tool icon 535 and a tool icon 536 for ending the light source setting process are included.

  In the initial setting, the light source is off. When the light source is turned on by selecting the button 531, the control unit 10 arranges an object (not shown) of the light source in the three-dimensional space. The light source object has parameters such as the XYZ coordinates of the light source itself, the direction of the light source, and the light intensity as parameters in the three-dimensional space. By changing these parameters with the button 533 and the slider 534, how the light strikes the appearance image can be changed. In addition to the parameters described above, parameters such as the type of light source (directional light, point light source, etc.), the color temperature of light, and the like may be included.

Embodiment 3 FIG.
In the above-described embodiment, the architectural image display device 1 is configured to operate stand-alone. However, a database in which the customer ID and the identification information of the house model selected as a candidate in the past or the identification information of the three-dimensional shape data of the house under design is stored in association with an external server is created. It is good also as a structure which memorize | stores and can read from the building image display apparatus 1. FIG.

  FIG. 20 is an explanatory diagram showing the configuration of the architectural image display system. The architectural image display system includes an architectural image display device 1 and a building information server 2. The building image display device 1 and the building information server 2 are connected wirelessly via a network N. The architectural image display device 1 is as described above except that it further includes a communication unit. As shown in FIG. 20, the building information server 2 includes a database 200 in which identification information of a house model selected in the past or design data of a house under design is stored in association with a customer ID. When the selection tab 133 is selected on the main screen, the control unit 10 of the building image display apparatus 1 displays a login screen, receives input of customer ID and password information, and receives the received information as the building information server 2. And the identification information or design data of the corresponding house model is received. The control unit 10 creates an appearance image of the house model to be superimposed from the three-dimensional shape data based on the received house model identification information or design data. As a result, the operator can always display a composite image in which the appearance image of the house model based on the contents examined in the past is superimposed on the captured image of the planned construction site, and the progress of the design meeting can be displayed. It can be used to promote the sale of homes, improving convenience.

Embodiment 4 FIG.
In the above-described embodiment, the object of the appearance image to be displayed is the house model. However, the present invention is not limited to a house model, and may be a model of a building such as a store or a combined house building, a warehouse model, or the like. In addition, the object of the appearance image may be a so-called external structure such as a structure outside the house, a fence, a gate, an approach, or the like, equipment such as a system kitchen or system bath installed in the house, or furniture. It may be.

  FIG. 21 is an explanatory diagram illustrating an example of a screen when displaying an appearance image of furniture installed in a house. The screen example shown in FIG. 21 corresponds to the screen example in FIG. 7, and the background image is not a captured image obtained by photographing a planned building site, but a captured image in a house to be installed, and an image to be superimposed is a housing model. The only difference is that it is not the appearance image of the furniture, but the appearance image of the furniture based on the data of the three-dimensional shape of the furniture. In the example of FIG. 21, an example in which an appearance image of a chair is superimposed on a captured image is shown. Since the process performed by the control unit 10 is the same as that described above, the description thereof is omitted. The captured image in the house shown in FIG. 21 can also be recognized as a three-dimensional space by using the same method as in the second embodiment. In addition, when a moving operation is accepted for an appearance image of furniture arranged in a house, the movement may be limited to the movement adsorbed on the floor in the house. Further, by accepting designation of the position of the wall, the movement range may be limited so that the appearance image of the furniture does not move to the outside of the wall.

  Moreover, it is good also as a structure which displays merchandise information about the external appearance image of the furniture arrange | positioned in a house. FIG. 22 is an explanatory diagram showing an example of a screen when displaying product information. FIG. 22 shows an example in which the appearance images of four furniture items “sofa”, “rug”, “chair”, and “light” are superimposed on the captured image in the house. The control unit 10 of the building image display device 1 displays the product information such as the product name, color, size, and price of the superimposed furniture in a list on the display unit 13 together with the captured image in the house and the appearance image of the furniture. . Moreover, it is good also as a structure which displays the total amount of furniture, the contact information of the furniture sales company, etc. When the control unit 10 receives the selection of the appearance image of the furniture to be newly superimposed, the control unit 10 adds the furniture product information to the list. When the furniture appearance image is deleted from the screen, the control unit 10 displays the furniture product information. Remove from the list.

  Furthermore, a three-dimensional object for measurement may be displayed so that the dimensions of the structure in the captured image and the structure such as furniture superimposed on the captured image may be measured. When displaying a three-dimensional object for measurement, the control unit 10 of the architectural image display device 1 executes a space recognition process similar to that of the second embodiment in order to recognize a captured image as a three-dimensional space. 23 and 24 are explanatory diagrams illustrating examples of setting screens displayed on the display unit 13 of the architectural image display device 1 when the space recognition process is executed. The operator takes a picture in a state where a rectangular marker having a known dimension such as A4 size, A3 size, postcard size or the like is placed on the room (floor), and the obtained captured image is displayed on the display unit 13 of the architectural image display device 1. To display. The operation unit 14 of the architectural image display device 1 accepts designation of dimensions for the marker image 70 which is a marker image. Specifically, a standard size dimension is specified by receiving input of the width dimension (X) and depth dimension (Y) through the tool icon 61 arranged on the toolbar 60 or by receiving selection of the tool icons 62 to 64. Accept.

  Next, the control unit 10 of the architectural image display device 1 displays the fit frame 71 on the display unit 13 as shown in FIG. This fit frame 71 has the same function as the above-mentioned site grid 525, with the center of the fit frame 71 as the origin, the width direction as the X axis, the depth direction as the Y axis, and the normal direction relative to the fit frame 71 as the origin. The Z axis is a reference for defining a three-dimensional space in the screen. The architectural image display device 1 accepts an operation of matching the fit frame 71 with the marker image 70 by accepting changes in the tilt, display position, scale, and orientation of the fit frame 71 on the touch panel. The operation method of the fit frame 71 is exactly the same as the operation method of the site grid 525.

  When the fit frame 71 and the marker image 70 match, the control unit 10 of the architectural image display device 1 has set the position of the origin determined by the fit frame 71, the X-axis, Y-axis, and Z-axis directions, and The scale is stored in the temporary storage unit 11, and the captured image is recognized as a three-dimensional space.

  After recognizing the captured image as a three-dimensional image, the control unit 10 of the architectural image display device 1 causes the display unit 13 to display a three-dimensional object for measurement. FIG. 25 is an explanatory view showing an example of a screen during dimension measurement. In the example shown in FIG. 25, a measurement cube 75 is displayed as a three-dimensional object for measurement. Initial values (for example, each dimension of 500 mm) are set for the width dimension, the depth dimension, and the height dimension of the measurement cube 75, respectively, and the control unit 10 of the architectural image display device 1 sets the dimension and setting of the measurement cube 75. The size of the measurement cube 75 is adjusted based on the reduced scale of the three-dimensional space, and is displayed superimposed on the captured image.

  In addition, the measurement cube 75 can change its display position, scale, and orientation by accepting an operation on the touch panel of the architectural image display device 1 as well as an object such as a house model and a structure in the house. . For example, the control unit 10 of the architectural image display apparatus 1 detects the contact of two fingers in the vicinity of the measurement cube 75 and then the two fingers approach each other in the height direction of the measurement cube 75 (or are separated from each other). ) When an operation that slides in the direction is detected, it is determined that an operation for changing the height of the measurement cube 75 is accepted, and the height of the measurement cube 75 is changed to a height corresponding to the amount of movement of the two fingers. . By accepting the same operation in the width direction and the depth direction of the measurement cube 75, the width dimension and the depth dimension of the measurement cube 75 can be changed. The control unit 10 stores the display position, dimensions, and orientation of the measurement cube 75 after the change in the temporary storage unit 11.

  When the control unit 10 of the architectural image display device 1 accepts a double-click operation of the measurement cube 75 on the touch panel, the control unit 10 reads and displays each dimension of the measurement cube 75 stored in the temporary storage unit 11. Displayed on the unit 13. By arranging such a measurement cube 75 in the vicinity of the structure to be measured and comparing it with each dimension of the measurement cube 75, the dimension of the structure to be measured can be measured.

  In the present embodiment, the indoor recognition is performed with the marker placed on the floor, and the space recognition process is executed using the marker image 70 included in the obtained captured image. If a pattern that can define the X and Y directions is included on the floor surface of a carpet or carpet, a space recognition process is executed using a flooring or carpet pattern without placing a marker. It is good also as a structure.

  Further, in this embodiment, the suction movement on the floor surface has been described. However, when an image of a structure such as a table or cabinet is included in the captured image, height information is given to the structure, and the tableware The range of movement may be limited so that a three-dimensional object such as a home appliance moves by suction on the structure. Thereby, the image of various household goods, such as tableware and a household appliance, can be easily superimposed on a captured image, and the operativity at the time of showing an actual use image of a room to a customer improves.

  In Embodiment 1-4, it was set as the structure which uses tablet type PC as the building image display apparatus 1. FIG. However, it is not limited to a tablet PC. A desktop PC, a notebook PC, or a mobile phone may be used. The architectural image display device 1 can be realized as long as the display device displays a screen based on the processing of the CPU.

  It should be understood that the embodiment disclosed above is illustrative in all respects and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the meanings described above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 1 Architectural image display apparatus 10 Control part 12 Storage part 13 Display part 14 Operation part 1P Control program

Claims (10)

An appearance image of a building or a structure arranged inside or outside the building is created based on the three-dimensional shape data of the building or structure, and the building planned place of the building or the planned placement place of the structure In the architectural image display device that displays the appearance image on the display unit by superimposing the captured image on the captured image,
Receiving means for receiving the display position and scale setting of the appearance image of the building or structure, and the setting of the viewpoint to the building or structure, respectively;
Based on the setting accepted by the accepting means, a creation means for re-creating the appearance image of the building or structure based on the three-dimensional shape data of the building or structure;
Means the appearance image said producing means, and re-displayed by being superimposed on the shooting image,
Means for accepting an operation for designating an arbitrary location to be invisible among the appearance images of the building or structure;
Means for creating a translucent foreground image of the captured image;
Means for displaying the created foreground image superimposed on the captured image and the appearance image;
Means for making an area of the foreground image corresponding to a location designated by the operation when the operation is accepted in a state where the foreground image is displayed superimposed on the photographed image and the appearance image; An architectural image display device comprising: Based on the display position, scale, and viewpoint of the appearance image of the building or structure, the image of a person standing in the vicinity of the building or structure or the image of the car arranged in the vicinity is used as the display position, scale, and viewpoint. The architectural image display device according to claim 1, further comprising means for displaying in correspondence. Means for superimposing a two-dimensional plane image having a known width dimension and depth dimension on the photographed image to be displayed on the display unit;
Means for accepting the setting of the display position and scale of the image of the two-dimensional plane displayed on the display unit, and the setting of the viewpoint for the two-dimensional plane;
And a means for setting a three-dimensional orthogonal coordinate system comprising coordinate axes orthogonal to the two-dimensional plane and two coordinate axes orthogonal to each other in the two-dimensional plane, based on the setting received by the means;
The building according to claim 1 or 2, wherein an appearance image of the building or structure is displayed on the two-dimensional plane in a space defined by a three-dimensional orthogonal coordinate system. Image display device. And a means for restricting movement of the display position of the appearance image to movement in a direction parallel to the two-dimensional plane when the setting of the display position of the appearance image is received by the receiving means. The architectural image display device according to claim 3 . The apparatus further comprises means for restricting movement of the viewpoint to movement of the viewpoint around a coordinate axis orthogonal to the two-dimensional plane when setting of the viewpoint to the building or structure is received by the receiving means. The architectural image display device according to claim 3 . Means for displaying an image of a rectangular parallelepiped having a bottom surface on the two-dimensional plane and having a known width dimension, depth dimension, and height dimension on the display unit;
Means for accepting the setting of the display position and scale of the image of the cuboid displayed on the display unit, and the setting of the viewpoint for the cuboid;
Based on the setting received by said means, said rectangular image, as well as the width of the rectangular parallelepiped, claim 3 of 5 further comprising a means for displaying depth dimension, and the height The architectural image display device according to one. An appearance image of a building or a structure arranged inside or outside the building is created based on the three-dimensional shape data of the building or structure, and the building planned place of the building or the planned placement place of the structure In the architectural image display method for displaying the appearance image superimposed on the photographed image obtained by photographing
Accepting the setting of the display position and scale of the appearance image of the building or structure, and the setting of the viewpoint to the building or structure,
Based on the accepted settings, recreate the appearance image of the building or structure based on the three-dimensional shape data of the building or structure,
Appearance image recreated and re-displayed by being superimposed on the shooting image,
In the appearance image of the building or structure, accepting an operation for designating an arbitrary place to be invisible,
Create a semi-transparent foreground image of the captured image,
Display the created foreground image superimposed on the captured image and the appearance image,
When the operation is accepted in a state where the foreground image is superimposed on the captured image and the appearance image, the area of the foreground image corresponding to the location specified by the operation is made opaque. A featured architectural image display method. Create a semi-transparent foreground image of the captured image,
Display the created foreground image superimposed on the captured image and the appearance image,
Accepting designation of any location to be invisible among the appearance images of the building or structure,
The architectural image display method according to claim 7 , wherein the foreground image at a location where the designation is received is made opaque. An external appearance image of a building or a structure arranged inside or outside the building is created on a computer connected to the display unit based on the data of the three-dimensional shape of the building or the structure, and the building schedule of the building In a computer program that superimposes the appearance image on a photographed image obtained by photographing a place or a planned placement place of the structure, and outputs the image to the display unit,
In the computer,
Processing for accepting the setting of the display position and scale of the appearance image of the building or structure, and the setting of the viewpoint to the building or structure, respectively;
A process of re-creating an external appearance image of the building or structure based on the three-dimensional shape data of the building or structure based on the accepted setting ;
Appearance images created and re-displayed by being superimposed on the shooting image processing,
A process of accepting an operation for designating an arbitrary location to be invisible among the appearance images of the building or structure,
Processing for creating a foreground image in which the photographed image is translucent;
Processing for displaying the created foreground image superimposed on the captured image and the appearance image; and
When the operation is accepted in a state where the foreground image is displayed superimposed on the photographed image and the appearance image, processing for making the foreground image region corresponding to the location specified by the operation opaque A computer program that is executed. In the computer,
Processing for creating a foreground image in which the photographed image is translucent;
Processing to display the created foreground image superimposed on the captured image and the appearance image;
A process of accepting designation of an arbitrary location to be invisible among the appearance images of the building or structure, and
The computer program according to claim 9 , wherein a process for making the foreground image opaque at a location where a designation is received is executed.

Images