JP6696256B2 - Decoration simulation device, decoration simulation method, and program - Google Patents

Description

  The present invention relates to a decoration simulation device, a decoration simulation method, and a program, and more particularly to a technique of synthesizing an image of a decorative article such as a curtain with an image of a room or the like.

  When consumers purchase window decorations such as curtains and blinds, in addition to design and color, compatibility with the interior of the room is an important decision point. Therefore, the user considers the purchase by looking at the actual goods in the store, catalogs, pamphlets, images on the website, etc. and imagining them by replacing them with their own room. However, when you actually buy the curtain and attach it to the room, you may be dissatisfied with the image you had before purchasing.

  In order to avoid such a feeling of dissatisfaction, a mechanism of prior confirmation before purchase using a composite image has been proposed. For example, Patent Document 1 discloses a mechanism for performing a simulation for changing the pattern of a curtain by using a two-dimensional image. Patent Document 1 describes a technique for synthesizing an image by synthesizing an image having a pattern on an object having a curved surface in a two-dimensional background image by deforming the pattern so as to be in a natural state according to the curved surface. Has been done.

Japanese Patent No. 3679347

  However, in Patent Document 1, as an assumption, an indoor image with a plain curtain is used as a background image, and a different color or pattern is applied to the plain curtain in the background image for display. Therefore, the method of Patent Document 1 cannot be applied to an image in which a curtain is not mounted or an image in which a patterned curtain is mounted. Further, in particular, when a curtain simulation is performed, the actual appearance of the color and shape of the curtain greatly changes depending on the position and size of the window in the image to be combined, the lighting environment of the room, and the like. Therefore, the image may be different from the case where the curtain is actually installed in the room simply by synthesizing the picture of the room and the picture of the curtain.

  The present invention has been made in view of such a problem, and it is possible to generate a composite image as if an object having a desired decoration (color or pattern) is arranged in a space in a captured image. A decorative simulation device, a decorative simulation method, and a program are provided.

A first aspect of the present invention for solving the above-mentioned problems includes a storage unit that stores and stores a plurality of sample images having an object to be decorated and a plurality of material images that form the color and pattern of the object, and an arbitrary storage unit. An input means for inputting an image, a setting means for receiving a setting of an object placement area for the input image and a material image applied to the object placement area and the setting of the material image, and comparing the input image with each other, Extraction means for extracting from the storage means a single sample image in which the geometric shape of the predetermined image or the tint of the image is similar to the input image, and the setting for the object area in the extracted sample image A material composite image generation means for generating a material composite image to which the material image set by the means is applied, and the object composite image from the generated material composite image. Extracting the region, a synthesizing means for synthesizing the object arrangement region set on the input in the image, a decoration simulation device, characterized in that it comprises a.

  According to the first aspect of the present invention, when an object arrangement area is set in an arbitrary input image and a material image such as a color or a pattern to be applied is set, a sample image similar to the input image is extracted, and a sample image in the sample image is extracted. A material composite image is generated by combining the material images in the object area. Further, a composite image is generated by extracting the object area of the generated material composite image and arranging it in the object arrangement area in the input image. As a result, even if a plain object is not placed in the input image or an object with a pattern is already placed, an object to which a desired color or pattern is applied using a similar sample image Can be generated, and a high-quality composite image can be obtained as if the image was captured by placing it in the input image. Features such as the object arrangement position and the lighting environment of the space can be made equal to the sample image, and a high-quality composite image with high affinity when the images are combined can be generated.

In the first invention, a learning process using a plurality of sample images stored in the storage means is performed to compare the input image with the input image and a geometric shape of a predetermined object. or further comprising a identifier generating means for generating an identifier for color images to identify the single sample image most similar, said extracting means, compared to the input image using the discriminator Then, it is desirable to extract a single sample image in which the geometric shape of a predetermined object or the tint of the image is most similar to the input image. By using the classifier, an image similar to the input image can be accurately extracted from a larger number of sample images, and a high-quality composite image with less discomfort can be generated.

  Further, in the first invention, each sample image stored in the storage unit is associated with a mask image corresponding to an object area, size information, and object information including perspective information in advance and stored, and the setting unit It is preferable that the setting information of the object arrangement area size information and the setting of the object information including the perspective information be accepted, and the extraction means extract a sample image similar to the input image based on the object information. As a result, it becomes easy to extract a sample image in which the position and perspective of the object area are similar.

  In the first invention, the material composite image generation means converts the coordinate value of a rectangular area corresponding to an object area in the sample image into a coordinate value of a normalized material coordinate system corresponding to the material image, By performing a curve conversion process for converting the material coordinate system by a periodic function along at least one of the vertical axis and the horizontal axis, the pixel value of each pixel of the material image developed in the material coordinate system is calculated. It is desirable to set to the corresponding pixel on the rectangular area. As a result, the material image is deformed according to the distortion of the curved surface of the object and is combined with the mask area corresponding to the object in the sample image. Therefore, for example, a material composite image in which the material pattern is distorted according to the folds of the curtain can be obtained.

  1st invention WHEREIN: The deformation | transformation means which deform | transforms the object area | region extracted from the material synthetic | combination image produced | generated by the said material synthetic | combination image production | generation means according to the size information and perspective information of the said object arrangement area set by the said setting means. Further provision is desirable. As a result, the input image does not necessarily have to be taken from the front direction, the restrictions on the input image are eased, and a decoration simulation device that is easy to use can be realized.

  In the first invention, it is preferable that the synthesizing unit transforms the material-synthesized image by Poisson blending based on the input image and synthesizes the material-synthesized image with the input image. As a result, it is possible to match the object to be combined with the tint of the input image and blend the boundary portion, and a natural combined image can be obtained.

  According to a second aspect of the present invention, a control unit of a computer having a storage unit that stores and stores a plurality of sample images having an object to be decorated and a plurality of material images constituting the color and pattern of the object displays an arbitrary image. A step of inputting; a step of accepting a setting of an object placement area for the input image and a material image applied to the object placement area; and a setting of the material image; and a sample image similar to the input image And a step of generating a material composite image in which the set material image is applied to the object area in the extracted sample image, the object area is extracted from the generated material composite image, and the input And a step of synthesizing the set object placement area in the image, the decoration simulation method.

According to a second aspect of the present invention, a control unit of a computer having a storage unit that stores and stores a plurality of sample images having an object to be decorated and a plurality of material images constituting the color and pattern of the object displays an arbitrary image. A step of inputting, a step of accepting the setting of the object arrangement area for the input image and a material image applied to the object arrangement area and the setting of the material image, and comparing the input image with the input Extracting from the storage unit a single sample image in which the geometrical shape of the image and a predetermined object or the tint of the image are most similar; and applying the set material image to the object region in the extracted sample image Decoration simulation, which includes the step of generating the material composite image described above, and the step of extracting the object area from the generated material composite image and combining the extracted object area with the set object placement area in the input image. Is the way.

According to a third aspect of the present invention, a computer is configured to store a plurality of sample images each having an object to be decorated and a plurality of materials constituting the color and pattern of the object, and an input unit for inputting an arbitrary image. And setting means for accepting the setting of the object placement area for the input image and the setting of the material image to be applied to the object placement area and the setting of the material image, and comparing the input image with the input means . Extraction means for extracting from the storage means a single sample image in which the geometric shape of the image and a predetermined object or the tint of the image are most similar, and the setting means is set for the object area in the extracted sample image. Material composite image generation means for generating a material composite image to which a material image is applied, and the object area is extracted from the generated material composite image. Out a synthesizing means, a program for functioning as to be combined with the object arrangement region set on the input in the image.

  According to the third invention, it is possible to cause the computer to function as the decoration simulation device of the first invention.

  According to the present invention, there are provided a decoration simulation device, a decoration simulation method, and a program capable of generating a composite image as if an object having a desired decoration (color or pattern) is arranged in a space in a captured image. Can be provided.

Block diagram showing the hardware configuration of the decoration simulation device 1 Example of sample image data 31 stored in the storage unit 12 Example of material data 32 stored in the storage unit 12 Block diagram showing the functional configuration of the decoration simulation device 1 The flowchart which shows the flow of the process which the control part 11 of the decoration simulation device 1 performs. (A) Example of the input image 41, (b) Mask image 41m showing the object placement area 41p set in the input image 41, (c) Size information of the object placement area 41p, (d) Perspective information of the object placement area 41p The figure which shows the example of extraction of the sample image similar to the input image 41. The figure explaining the material synthetic | combination image generation process of step S104. The figure explaining the transformation process of step S105. The figure explaining the synthetic | combination process of step S106. The figure explaining Poisson blending (blending synthesis) in the synthesis process.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a hardware configuration of the decoration simulation device 1. The hardware configuration of FIG. 1 is an example, and various configurations can be adopted according to the use and purpose.

  As shown in FIG. 1, the decoration simulation device 1 includes a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 17, and the like. It is connected via the bus 18.

  The control unit 11 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU calls a program stored in the storage unit 12, the ROM, the recording medium, or the like into the work memory area on the RAM and executes the program to drive and control each device connected via the bus 18, and the decoration simulation device 1 The processing to be described later is realized. The ROM is a non-volatile memory, and permanently holds a computer boot program, programs such as BIOS, data, and the like. The RAM is a volatile memory, and temporarily stores programs, data, and the like loaded from the storage unit 12, the ROM, the recording medium, and the like, and includes a work area used by the control unit 11 to perform various processes.

  The storage unit 12 is an HDD (Hard Disk Drive) or the like, and stores a program executed by the control unit 11, data necessary for executing the program, an OS (Operating System), and the like. As for the programs, a control program corresponding to the OS and an application program for causing a computer to execute the processes described below are stored. These respective program codes are read by the control unit 11 as needed, transferred to the RAM, read by the CPU, and executed as various means.

  The media input / output unit 13 (drive device) inputs and outputs data, and is a medium such as a CD drive (-ROM, -R, -RW, etc.), a DVD drive (-ROM, -R, -RW, etc.), or the like. It has an input / output device. The communication control unit 14 is a communication interface that has a communication control device, a communication port, and the like, and mediates communication between a computer and a network, and controls communication with other computers via the network. The network may be wired or wireless.

  The input unit 15 inputs data and has, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. Through the input unit 15, operation instructions, operation instructions, data input, etc. can be given to the computer. The display unit 16 includes a display device such as a liquid crystal panel, and a logic circuit or the like (video adapter or the like) for realizing a video function of a computer in cooperation with the display device. The input unit 15 and the display unit 16 may be integrated as in a touch panel display.

  The peripheral device I / F (Interface) unit 17 is a port for connecting the peripheral device to the computer, and the computer transmits and receives data to and from the peripheral device via the peripheral device I / F unit 17. The peripheral device I / F unit 17 is configured by USB (Universal Serial Bus), IEEE1394, RS-232C, or the like, and usually has a plurality of peripheral device I / Fs. The connection form with the peripheral device may be wired or wireless. The bus 18 is a path that mediates exchange of control signals, data signals, etc. between the respective devices.

  The decoration simulation device 1 may be configured by one computer, or a plurality of computers may be configured via a network. For example, when the decoration simulation device 1 is composed of a server and a client terminal, the client terminal may accept data input, the server may perform various processes, and the client terminal may display the process result. In the following description, as a simple configuration example, an example in which the decoration simulation device 1 is configured by one computer will be described.

Next, the data structure of the decoration simulation device 1 will be described.
FIG. 2 is a diagram showing an example of the sample image data 31 stored in the storage unit 12 of the decoration simulation device 1. The storage unit 12 of the decoration simulation device 1 stores, as the sample image data 31, a plurality of sample images 31g and object information 31m, 31p, 31s of each sample image 31g. The sample image 31g is a two-dimensional image having an object to be decorated. The object will be described as an example of window ornaments such as curtains and blinds, but the present invention also includes articles having planar portions such as furniture, walls, and doors as objects to be decorated. The object information is a mask image 31m corresponding to the object area in the sample image 31, size information 31s of the object, and perspective information 31p indicating a rectangular area (coordinates of four points) circumscribing the object. The sample image 31g and the object information 31m, 31s, 31p may be acquired from the furniture or interior catalog data, or may be registered by the user via the input unit 15 or the like.

  For example, in FIG. 2, as information about the “sample image 1”, a sample image 31g in which a room in which a plain curtain is arranged as an object to be decorated is photographed, and a binary mask in which the object area is separated from other areas The image 31m, the object size information 31s is "width: 800 mm, height: 1900 mm", and the perspective information 31p is "P1: (383, 51), P2: (379, 509), P3: (530, 528). , P4: (537,41) ”is stored. Similarly, for the “sample image 2”, the “sample image 3”, and the “sample image 4”, a sample image 31 g in which the room in which the object (curtain) to be decorated is arranged is photographed, a mask image 31 m of the object, Object size information 31s and perspective information 31p are held. In the present embodiment, a photograph or the like is shown as an example of the sample image 31g, but the present invention is not limited to the photographed image, and the image created by computer graphics or the like is used as the sample image 31g. Good.

  FIG. 3 is a diagram showing an example of the material data 32 stored in the storage unit 12 of the decoration simulation device 1. In the storage unit 12 of the decoration simulation device 1, a plurality of material images 32g and size information 32s of the material images 32g are stored as the material data 32. The material image 32g is a patch image representing one unit of color or pattern (texture). The size information 32s is a numerical value indicating the height and width sizes of the material image 32g. The material image 32g and the size information 32s may be acquired from furniture or interior catalog data, or may be registered by the user via the input unit 15 or the like.

  For example, FIG. 3 shows an example in which the patch image (material image 32g) of the plant pattern and the size information 32s “width: 120 mm, height: 70 mm” are held as the information regarding “material 1”. Similarly, for "material 2", "material 3", and "material 4", the material image 32g indicating the color and pattern and the size information 32s of the material image 32g are stored.

  Next, a functional configuration of the decoration simulation device 1 according to the present invention will be described with reference to FIG.

  As shown in FIG. 4, the control unit 11 of the decoration simulation device 1 includes an image input unit 21, a setting unit 22, a sample image extraction unit 23, a classifier 24, a material composite image generation unit 25, a transformation unit 26, and an image composition unit. 27 and a composite image output unit 28.

  The image input unit 21 receives an input of an arbitrary image on which a decoration object is to be arranged. In the following description, the image input to the image input unit 21 will be referred to as the input image 41. A decoration object is an article to which decoration such as color or pattern is applied. For example, a curtain will be described as an example. The input image 41 is a two-dimensional image such as a photograph taken inside the room.

  The setting unit 22 receives the setting of the object placement area 41p for the input image 41, the setting (designation) of the material image 32 applied to the object placement area 41p, and the setting for the designated material image.

  The sample image extraction unit 23 extracts a sample image 31g similar to the input image 41 from the sample image data 31 stored in the storage unit 12. The process of extracting the sample image 31g will be described later. The sample image extraction unit 23 may identify and extract the sample image 31g similar to the input image 41 by using the classifier 24. The discriminator 24 is generated by the learning process performed by the control unit 11. In the learning process, the control unit 11 takes in a plurality of sample image data 31 stored in the storage unit 12 as learning data, learns the characteristics of the sample image data 31, and generates the discriminator 24. The discriminator 24 will be described later.

  The material composite image generation unit 25 sets the material image 32g (designated material 36) set by the setting unit 22 for the object (object area) arranged in the space within the extracted sample image 35g extracted by the sample image extraction unit 23. ) Is applied to generate the material composite image 37. The material image 36g of the designated material 36 is acquired from the material data 32 stored in the storage unit 12. The process of generating the material composite image 37 will be described later.

  The transformation unit 26 transforms an object (decoration object 37a) extracted from the material composite image 37 generated by the material composite image generation unit 25 according to the size and perspective of the object arrangement area set in the input image 41g, and deforms the object. A post-material composite image 38 is obtained. The transformation process will be described later.

  The image composition unit 27 extracts an object (decoration object 37a) extracted from the material composition image 37 generated by the material composition image generation unit 25 or an object extracted from the material composition image 38 transformed by the transformation unit 26 (decorated object after transformation). 38a) is arranged in the object arrangement area 41p in the input image 41 to generate the composite image 5. The image composition processing will be described later.

  The combined image output unit 28 outputs the combined image 5 generated by the image combining unit 27. The output includes display on the display unit 16, storage in the storage unit 12, transmission to a network or the like via the communication control unit 14, and the like.

  Next, with reference to FIG. 5, a flow of a decoration simulation process executed by the control unit 11 of the decoration simulation device 1 will be described.

  First, the control unit 11 receives an input of an image to be simulated (step S101). The input image 41 is a two-dimensional image of the interior of the room. Next, the control unit 11 receives an input of setting information regarding the decoration of the object (step S102). Specifically, the control unit 11 sets the object placement area 41p for the input image 41 input in step S101, the setting (designation) of the material image 32g to be applied to the object placement area 41p, and the size for the designated material image 32g. Settings such as the information 32s are accepted. The user sets the object placement area 41p by performing an input operation such as designating four points (rectangular area) on the input image 41 via the input unit 15. The user also selects the material image 32g to be applied to the object placement area 41p from the material data 32 stored in the storage unit 12. At this time, the control unit 11 preferably displays a list of a plurality of material images 32g of the material data 32 stored in the storage unit 12 on the display unit 16 in a selectable manner.

  Next, the control unit 11 extracts a sample image 31g similar to the input image 41 input in step S101 from the storage unit 12 (step S103). In step S103, the control unit 11 may identify and extract the sample image data 31 similar to the input image 41 by using the identifier 24. The discriminator 24 is generated by the learning process performed by the control unit 11. In the learning process, the control unit 11 takes in a plurality of sample image data 31 (sample image 31g, mask image 31m, size information 31s, perspective information 31p) stored in the storage unit 12 as learning data, and the sample image data 31 A classifier 24 that learns the features and identifies the indoor image is generated.

  The discriminator 24 is preferably an Exemplar-SVM, for example. Exemplar-SVM is a method of creating an entire classifier with a large number of one-to-many weak classifiers. In the Exemplar-SVM, the object information (mask image 31m, size information 31s, perspective information 31p) attached to the sample image 31g of the sample image data 31 can be reused, which is preferable in comparison with other methods. .. The control unit 11 uses the Exemplar-SVM to learn the sample image data 31 registered in advance in the storage unit 12, and uses the discriminator 24 that determines whether or not the input image 41 input in step S101 is an indoor image. To generate. The sample image data 31 is positive data, and an arbitrary image group that is not the indoor space is negative data. The feature vector used at this time is a concatenated vector of Gist features, size information, and perspective information (resolution-normalized). Exemplar-SVM is described in "Ensemble of Exemplar-SVMs for Object Detection and Beyond (ICCV2011)", (http://www.cs.cmu.edu/~tmalisie/projects/iccv11/).

  As shown in FIG. 7, the control unit 11 inputs the input image 41 to the discriminator 24. The classifier 24 extracts a sample image 35g similar to the input image 41 from the sample image data 31 stored in the storage unit 12. In the sample image extraction process of step S103, it is desirable to extract the sample image 35g having a similar geometric shape of the object and the tint of the image (illumination environment etc.) to the input image 41. As a result of the sample image extraction process, a sample image (extracted sample image 35g) having a similar window position of the input image 41 and the lighting environment (position of the lighting equipment, tint of the image, etc.) is extracted.

  Next, the control unit 11 generates the material composite image 37 (step S104). The material composite image 37 is an image in which the material image 36g designated in step S102 is composited with the object area of the sample image 35g extracted in step S103. The object area is an area specified by the mask image 31m which is preset (stored) in each sample image 31g. In the example of FIG. 8, a white value area in the mask image 35m represented by a binary value of white and black is an object area.

  As shown in FIG. 8, the control unit 11 stores, as the extracted sample image data 35, a sample image 35g, a mask image 35m corresponding to the object region of the sample image 35g, object region size information 35s, and object region perspective information 35p. Obtained from the part 12. Further, the control unit 11 acquires the designated material 36 (material image 36g, size information 36s) designated in the setting process of step S102 from the material data 32 of the storage unit 12. The control unit 11 synthesizes the material image 36g of the designated material 36 so as to match the size and perspective of the object area of the sample image 35g. As a result, the material composite image 37 in which the entire image of the material image 36g is transformed according to the size information 35s and the perspective information 35s of the object area of the sample image 35g is obtained.

  In the material composite image generation processing in step S104, it is desirable to use the method described in Japanese Patent No. 3679347, for example. Japanese Patent No. 3679347 converts the coordinate values of a rectangular area circumscribing the object area in the extracted sample image 35g into the coordinate values of the normalized material coordinate system corresponding to the material image 36g, and the coordinate values of the material coordinate system. Is converted by a periodic function along at least one of the vertical axis and the horizontal axis (curve conversion processing), and the pixel value of each pixel of the material image developed in the material coordinate system is converted into a rectangular area by the curve conversion processing. Setting to the corresponding pixel above is described. When the designated material 36g is combined with the sample image 35g by this method, the material image is distorted into a curve along at least one axis (for example, the horizontal axis in the case of a curtain image) of the material image 36g by conversion based on the periodic function. Is converted into a certain image and set in the object area (rectangular area) of the sample image 35g. Therefore, for example, it is possible to obtain the decorative object 37a in which the pattern of the material is distorted in accordance with the folds of the object (curtain). Further, in the material composite image generation processing, it is desirable that the material image 36g is composed while retaining the shadow information and the like of the object area of the sample image 35g. As a result, the object (curtain) of the sample image 35g can represent not only the deformation based on the depth (perspective) in the image space but also the curved distortion of the object itself. Further, the shadow in the sample image 35g is also reflected in the material composite image 37.

  Next, the control unit 11 performs a transformation process of transforming the decoration object 37a of the material composite image 37 generated in step S104 so as to fit the object arrangement area of the input image 41 (the area specified by the mask image 41m) (step). S105). The object arrangement area of the input image 41 is set by the process of step S102, and is held as the mask image 41m, the size information 41s, and the perspective information 41p in association with the input image 41.

  The transformation process of step S105 includes resizing and perspective conversion of the rectangular area 37p corresponding to the decoration object 37a of the material composite image 37. As shown in FIG. 9, first, the control unit 11 obtains the perspective information and the size information of the decoration object 37a of the material composite image 37 from the extracted sample image data 35 (see FIG. 8), and stores the object placement area 41p of the input image 41. A projective transformation matrix for converting the perspective information 41p and the size information 41s is calculated. Then, the material projection image 37 is multiplied by the obtained projective transformation matrix. As a result, the size and perspective of the material composite image 37 (decorative object 37a) are transformed to match the size and perspective of the object placement area 41p of the input image 41. Hereinafter, the image obtained by the transformation process of step S105 is referred to as a transformed material composite image 38. A region corresponding to the decoration object in the post-deformation material composite image 38 is called a post-deformation decoration object 38a.

  The control unit 11 synthesizes the decoration object 38a that has been resized and parsed by the transformation process of step S105 with the input image 41 (step S106). In the combining process of step S106, the control unit 11 extracts a mask region (a white value region of the mask image 38m) from the post-deformation material composite image 38 and combines it with the input image 41. At the time of this synthesis, so-called "familiar synthesis" may be performed in which images are converted and pasted by, for example, Poisson blending (or also referred to as "Poisson Image Editing"). The method of the combining process in step S106 is not limited to Poisson blending, and other known techniques may be used.

  FIG. 11 is a diagram for explaining the image synthesizing process by Poisson blending. In Poisson blending, the tint depends on the image of the pasting destination image (input image 41g), and the gradient of the image is synthesized using the image of the pasting source image (post-deformation decorative object 38a). In Poisson blending, the image data of the pasting source is treated as a set of differential values, the differential values are saved as much as possible, the values of the pasting boundaries are matched, and natural image synthesis is performed. What realizes this is the simultaneous differential equation (Poisson equation), which can be solved by the Gauss-Seidel method.

  FIG. 11 shows an image of processing by Poisson blending. FIG. 11A shows an image before the Poisson blending process, and FIG. 11B shows an image after the Poisson blending process.

  In FIG. 11A, the image data 81 of the pasting source is arranged as it is in the image data 83 of the pasting destination. Therefore, at both ends of the pasting range 85, that is, at the pasting boundary 87, there is a large difference between the image data values of the two. If there is such a difference, in the actual image, the image data 81 that is the pasting source will appear to be floating, and a sense of discomfort will occur.

  On the other hand, in FIG. 11B, the pasting image data 81 is converted by Poisson blending so that the tint depends on the pasting image data 83. Therefore, at the pasting boundary 87, there is no great difference in the values of the pasted image data 89 after conversion and the pasted image data 83. As a result, the actual image does not feel very strange.

  In this way, by intentionally converting the hue of the image of the decoration object (curtain or the like) 37a (38a) and bringing it closer to the hue of the input image 41g, the user not only has the color and pattern of the curtain alone to be attached, The color and pattern of the curtain to be attached to the room can be selected by comprehensively considering the color tones of the room wallpaper, floor, and other furniture.

  When the composite image 5 is obtained by the composite processing in step S106, the control unit 11 outputs the composite image 5 (storage in the storage unit 12, display on the display unit 16, etc.) (step S107), and a series of decoration simulations are performed. The process ends.

  If the user confirms the composite image 5 as a result of the output in step S107 and the result of the composition is unsatisfactory, or if another color or pattern is to be tried, the process returns to step S102 and the material data 32 is redesignated. Alternatively, the object placement area 41p may be reset, and the processing from step S103 may be repeated.

  As described above, according to the decoration simulation device 1 of the present invention, the user inputs the desired input image 41g, sets the object placement area 41p for the input image 41g, and applies the material such as the color or pattern to be applied. When the image 36g is set, the control unit 11 of the decoration simulation device 1 extracts a sample image 35g similar to the input image 41g from the storage unit 12, and synthesizes the designated material image 36g in the object area in the sample image 35g. The image 37 is generated. Further, the control unit 11 generates a composite image 5 in which the decoration object 37a (deformed decoration object 38a) of the generated material composite image 37 is combined with the object arrangement area 41p in the input image 41g. As a result, even when a plain object is not arranged in the input image 41g or an object with a pattern is already arranged, the decorative object 37a (transformed) is transformed using the sample image similar to the input image 41. Since the post-decoration object 38a) is generated and combined with the input image 41, the features such as the arrangement (size and perspective) of the object and the lighting environment of the space can be made equal to the sample image, and the compatibility with the combination is high. A quality decoration simulation image (composite image 5) can be generated.

  The preferred embodiments of the decoration simulation device and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to the examples. It is obvious to those skilled in the art that various modifications or modifications can be conceived within the scope of the technical idea disclosed in the present application, and naturally, those modifications and modifications are also included in the technical scope of the present invention. Understood.

1 …………………… Decoration simulation device 11 …………………… Control unit 12 …………………… Storage unit 13 ………………… Media input / output unit 14 …………… ...... Communication control unit 15 …………………… Input unit 16 …………………… Display unit 17 …………………… Peripheral equipment I / F unit 18 ………………… Bus 21… ……………… Image input unit 22 …………………… Material and object placement area setting unit 23 …………………… Sample image extraction unit 24 …………………… Identifier 25 ………… ………… Material composite image generation unit 26 …………………… Transformation unit 27 …………………… Image composition unit 28 …………………… Composite image output unit 31 ……………… Sample image data 31g ……………… Sample image 31m ……………… Mask image 31s ……………… Size information 31p ……………… Perspective information 32 ………………… Material data 32g ……………… Material image 32s ……………… Size information 35 …………………… Extracted sample image data 35g ……………… Extracted sample image 35m ……………… Mask image 35s… …………… Size information 35p ……………… Perspective information 36 …………………… Specified material 36g ……………… Material image 36s ……………… Size information 37 ……………… …… Material composite image 37a ……………… Decorative object 37m ……………… Mask image 37p ……………… Perth (rectangular area)
37q ……………… Perspective after transformation (rectangular area)
38 …………………… Composite image after deformation 38a ……………… Decorated object after deformation 38m ……………… Mask image after deformation 41 …………………… Input image 41m ………… …… Mask image 41p ……………… Object placement area (rectangular area)
5 …………………… Composite image 81 …………………… Paste source image data 83 …………………… Paste destination image data 85 ………………… Paste range 87 …………………… Border of pasting 89 …………………… Image data of pasting source after transformation

Claims (8)

Storage means for accumulating and storing a plurality of sample images having an object to be decorated, and a plurality of material images constituting the color and pattern of the object;
An input means for inputting an arbitrary image,
Setting means for receiving the setting of the object placement area for the input image and the material image to be applied to the object placement area and the setting of the material image,
Extraction means for extracting, from the storage means, a single sample image in which the input image has a geometrical shape of a predetermined object or the tint of the image most similar to the input image ,
A material composite image generation means for generating a material composite image in which the material image set by the setting means is applied to the object area in the extracted sample image,
A synthesizing means for extracting the object area from the generated material composite image and synthesizing it with the object arrangement area set in the input image;
A decoration simulation device comprising: By performing a learning process using a plurality of sample images stored in the storage unit, the input image and the geometric shape of the predetermined object or the tint of the image are compared with the input image. Further comprising a discriminator generating means for generating a discriminator for discriminating the most similar single sample image,
The extracting means extracts a single sample image having a geometrical shape of a predetermined object or a color shade of the image most similar to the input image by comparing the input image using the classifier. The decoration simulation device according to claim 1, wherein: In each sample image stored in the storage unit, a mask image corresponding to the object area, size information, and object information including perspective information are stored in advance in association with each other,
The setting means receives the size information of the object placement area and the setting of object information including perspective information,
The decoration simulation device according to claim 1, wherein the extraction unit extracts a sample image similar to the input image based on the object information. The material composite image generation means converts the coordinate value of the rectangular area corresponding to the object area in the sample image into the coordinate value of the normalized material coordinate system corresponding to the material image, and the material coordinate system is changed vertically. The pixel value of each pixel of the material image developed in the material coordinate system is corresponded on the rectangular area by performing a curve conversion process for conversion by a periodic function along at least one of the axis and the horizontal axis. decoration simulation apparatus according to any one of claims 1 to 3, characterized in that the set pixels.   It further comprises a deforming unit that deforms the object region extracted from the material composite image generated by the material composite image generating unit according to the size information and perspective information of the object arrangement region set by the setting unit. The decoration simulation device according to any one of claims 1 to 4.   The composition unit converts the material composite image by Poisson blending based on the input image, and combines the material composite image with the input image. Decoration simulation device. A control unit of a computer having a storage unit that stores and stores a plurality of sample images having an object to be decorated and a plurality of material images forming the color and pattern of the object,
Inputting an arbitrary image,
A step of receiving the setting of the object arrangement area for the input image and the material image applied to the object arrangement area and the setting of the material image;
Extracting from the storage unit a single sample image in which the input image has a geometrical shape of a predetermined object or the tint of the image most similar to the input image ;
Generating a material composite image in which the set material image is applied to the object area in the extracted sample image,
A step of extracting the object area from the generated material composite image and combining with the set object arrangement area in the input image;
A decoration simulation method comprising: Computer,
Storage means for accumulating and storing a plurality of sample images having an object to be decorated and a plurality of materials constituting the color and pattern of the object;
An input means for inputting an arbitrary image,
Setting means for accepting the setting of the object placement area for the input image, the setting of the material image applied to the object placement area, and the setting of the material image;
Extraction means for extracting, from the storage means, a single sample image in which the input image has a geometrical shape of a predetermined object or the tint of the image most similar to the input image ,
A material composite image generation means for generating a material composite image in which the material image set by the setting means is applied to the object area in the extracted sample image,
A synthesizing means for extracting the object area from the generated material composite image and synthesizing it with the object arrangement area set in the input image,
Program to function as.

Images