JP7139236B2 - Image processing device, image processing method and image processing program - Google Patents

Description

The present invention relates to an image processing device, an image processing method, and an image processing program.

2. Description of the Related Art Conventionally, in order to intuitively present information to a user, there is an AR (Augmented Reality) technology in which images or videos in a real space are processed by a computer and virtual information is superimposed thereon. For example, in recent years, there is an application that virtually places a product such as furniture in a video captured by a user. A user who uses such an application can easily know the atmosphere and size feeling when the product is installed in the actual room before purchase.

However, with the above-described application, when it is desired to place an AR image of new furniture in a place where existing furniture is installed, it is troublesome to move the existing furniture to another place. On the other hand, Patent Literature 1 discloses a method using a DR (depleted reality) technique for erasing "things" from an original image of the physical space. Specifically, in Patent Literature 1, furniture is removed from the original image, and a new image of the furniture is arranged on a 3D model based on the removed image. As a result, a 3D model is generated in which existing furniture is virtually replaced with new furniture.

JP 2015-125774 A

However, in the technique disclosed in Patent Document 1, the degree of impairment reality for the original image in the real space is adjusted based on the user's settings. For this reason, if the user wants to remove only the desired furniture from the original image, it is necessary for the user to set the degree of such depletion reality. For this reason, the setting by the user is required for each original image, which increases the user's time and effort.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus, an image processing method, and an image processing program capable of easily replacing an article in an image.

An image processing apparatus according to an aspect of the present invention acquires an original image captured by a virtual object acquisition unit that acquires a virtual object of an article based on information on the article included in user information, and an imaging device. an image acquisition unit, a target object detection unit that detects a target object from the original image based on the article, and an image processing unit that generates a processed image in which the target object in the original image is replaced with the virtual object; characterized by comprising

An image processing apparatus according to another aspect of the present invention includes an image acquisition unit that acquires an original image captured by an imaging device used by a user, and a target object that detects a target object arranged in a predetermined area from the original image. It is characterized by comprising a detection unit, a virtual object acquisition unit that acquires a virtual object of an article, and an image processing unit that generates a processed image in which the target object in the original image is replaced with the virtual object.

According to the image processing apparatus according to the aspect of the present invention, it is possible to automatically replace an appropriate physical object in the original image with a virtual object of the article desired by the user. This allows the user to easily replace the article in the image without moving the article in the physical space or setting the degree of depletion reality.
According to the image processing apparatus according to the aspect of the present invention, it is possible to automatically replace an article arranged in a predetermined area in an original image with a virtual object of an arbitrary article. This allows the user to easily replace the item in the image as described above.

1 is a block diagram showing a schematic configuration of an image processing system according to a first embodiment; FIG. 1 is a block diagram showing a schematic configuration of a server device according to a first embodiment; FIG. The figure which shows an example of the cart information page of goods selection content. The figure which shows an example of the product information page of product selection content. 4 is a flowchart showing an example of an image processing method of the server device according to the first embodiment; FIG. 4 is a diagram schematically showing the flow of processing an image processed in the first embodiment; 4 is a flowchart showing an example of image processing of the server device of the first embodiment; 4A and 4B are diagrams showing an example of a physical space and a processed image according to the first embodiment; FIG. 10 is a flowchart showing an example of an image processing method of the server device according to the second embodiment;

[First embodiment]
A first embodiment of the present invention will be described based on the drawings.
[Image processing system 1]
FIG. 1 is a block diagram showing a schematic configuration of an image processing system 1 of this embodiment.
As shown in FIG. 1, the image processing system 1 of this embodiment includes a user terminal 10 and a server device 20 functioning as the image processing device of the present invention. The user terminal 10 and the server device 20 are communicably connected via a network (for example, the Internet).

The image processing system 1 of the present embodiment is, for example, a part of an electronic commerce system, and the server device 20 registers information of products sold at an online shop. In this image processing system 1, the server device 20 performs image processing related to DR (degraded reality) and AR (augmented reality) on an image (original image) captured by the user terminal 10 in the real space. A processed image is generated in which the article inside is virtually replaced with the desired article. By viewing the processed image displayed on the user terminal 10, the user can intuitively know the atmosphere, size, etc., when the article in the physical space is replaced with the desired article.
A specific configuration of such an image processing system 1 will be described in detail below.

[User terminal]
The user terminal 10 is, for example, a computer such as a smart phone or a tablet terminal, and has basic hardware of such a computer.
Specifically, the user terminal 10 includes a terminal communication unit 11, an input operation unit 12, a terminal storage unit 13, a terminal control unit 14, a display 15, and a camera 16, as shown in FIG.

The terminal communication unit 11 is connected to a network via, for example, a LAN, and communicates with other devices on the network.
The input operation unit 12 outputs an operation signal by user operation to the terminal control unit 14 . As the input operation unit 12, for example, a touch panel provided integrally with the display 15 can be exemplified.

The terminal storage unit 13 is composed of a data recording device such as a memory or a hard disk. Various programs and the like for controlling the user terminal 10 are stored in the terminal storage unit 13 . The terminal storage unit 13 also temporarily stores original images to be processed by the server device 20, which will be described later.

The terminal control unit 14 includes an arithmetic circuit such as a CPU (Central Processing Unit) and a storage circuit such as a RAM (Random Access Memory), and controls each unit of the user terminal 10 . The terminal control unit 14 develops programs (software) such as predetermined applications among the programs stored in the terminal storage unit 13 or the like in RAM, and performs various processes in cooperation with the programs developed in RAM. Run. As a result, the terminal control unit 14 can communicate with the server device 20 via the network, and can use various services and view various data provided by the server device 20, for example.
In addition, the terminal control unit 14 transmits an image (original image) captured by the camera 16 to the server device 20 by the user's operation of the input operation unit 12 .
Further, the terminal control unit 14 causes the display 15 to display (output) an image (processed image) image-processed by the server device 20 .

The camera 16 corresponds to the imaging device of the present invention, and is composed of a lens, an image sensor, and the like. Images captured by the camera 16 are temporarily stored in the terminal storage unit 13 and transmitted to the server device 20 by the terminal control unit 14 .

[Configuration of server device 20]
FIG. 2 is a block diagram showing a schematic configuration of the server device 20. As shown in FIG.
As shown in FIG. 2 , the server device 20 is configured by a computer and includes a communication section 21 , a storage section 22 and a control section 23 .
The communication unit 21 is connected to a network, for example via a LAN or the like, and communicates with other devices on the network.

The storage unit 22 is configured by a data recording device such as a memory or a hard disk, for example.
The storage unit 22 stores various programs and various data for performing image processing, which will be described later. The storage unit 22 also includes a user database 221 , a product database 222 , a virtual object database 223 and an image processing database 224 .

The product database 222 records product information related to products sold at the online shop. This product information includes, for example, product ID, product name, category information, and the like.
The user database 221 stores user information about users who use the image processing system 1 . User information is managed for each user, and includes, for example, user IDs, user names, user attributes, history information, recommendation information, and the like. The history information includes browsing history, bookmark information, cart information, purchase history, and the like.
The virtual object database 223 stores virtual objects associated with each product (for example, product ID) registered in the product database 222 . A virtual object is, for example, 3D image data expressing the structure of a product in three-dimensional coordinates.
Image processing information for performing image processing is stored in the image processing database 224 . This image processing information stores, for example, learning data for performing image recognition.

In this embodiment, the storage unit 22 of the server device 20 exemplifies a configuration including the product database 222, the user database 221, the virtual object database 223, and the image processing database 224. may be stored on the device.

The control unit 23 is composed of an arithmetic circuit such as a CPU and a memory circuit such as a RAM. , performs various processing. By reading and executing the image processing program stored in the storage unit 22, the control unit 23 obtains an image acquisition unit 231, a virtual object acquisition unit 232, a target object detection unit 233, an image It functions as the processing unit 234 and the like. Each function will be described in detail later.

[Image processing method]
Next, each process performed in the image processing system 1 will be described.
(pretreatment)
In this embodiment, the server device 20 performs processing for storing product information (for example, product ID) in the user database 221 before performing image processing.
Specifically, the server device 20 transmits product selection content to the user terminal 10 based on the product selection content request received from the user terminal 10 . Thereby, the product selection content is displayed on the display 15 of the user terminal 10 .
Note that the product selection content provided by the server device 20 to the user terminal 10 is, for example, part of a service that provides online shopping, and may be provided via a general-purpose application such as a browser. It may be provided through an application for providing the service.

On the user terminal 10 , the user performs an operation of selecting a product offered at the online shop while looking at the product selection content page displayed on the display 15 . Here, the operation of "selecting a product" is, for example, an operation of viewing detailed information on the product, registering the product in a bookmark, or adding the product to the cart.

Based on the operation information received from the user terminal 10, the server device 20 stores the product information (for example, product ID) of the product selected by the user as history information (for example, viewing history, bookmark information, or cart information) in the user database 221. Memorize.
Hereinafter, products stored in the history information of the user database 221 are generally referred to as user-selected products.

After at least one user-selected item is stored in the user database 221, the user taps the "Place" button from any page of the item-selected content (eg, see FIG. 3 or 4). The server device 20 executes the image processing program based on the operation information received from the user terminal 10, and performs the image processing described below.

(Image processing)
FIG. 5 is a flowchart showing image processing in the server device 20. As shown in FIG.
First, the virtual object acquisition unit 232 refers to the user-selected product stored in the user database 221, and acquires the virtual object associated with the product ID of the user-selected product from the virtual object database 223 (step S1; virtual object acquisition step).

Here, based on the operation information received from the user terminal 10, the virtual object acquisition unit 232 may select which storage area of the browsing history, bookmark information, or cart information in the user database 221 to refer to. .
For example, when receiving operation information including the fact that the “Place” button in the page showing cart information has been tapped, the virtual object acquisition unit 232 refers to the cart information and corresponds to the product selected by the user in the cart information. You may get a virtual object that In this case, a plurality of virtual objects may be obtained.
In addition, when receiving operation information including the fact that the “Place” button in the page showing product information of a specific product has been tapped, the virtual object acquisition unit 232 refers to the browsing history and selects the first A virtual object corresponding to the newly stored user-selected item may be obtained.
In the following, for simplification of explanation, a case where one virtual object is acquired will be explained.

The image acquisition unit 231 also transmits a camera activation request to the user terminal 10 . Upon receiving the request, the user terminal 10 causes the camera 16 to start operating. When the user points the camera 16 at a desired space (for example, a part of the user's room), the camera 16 captures an original image of the real space as a subject. Then, the terminal control unit 14 transmits the original image captured by the camera 16 to the server device 20 . Thereby, the image acquisition unit 231 acquires the original image transmitted from the user terminal 10 (step S2; image acquisition step).

Next, the target object detection unit 233 detects a real object included in the acquired original image (step S3). Here, a physical object is image information of an article (for example, furniture) that exists in the physical space, and can be detected by image recognition technology. Note that when a plurality of articles exist in the physical space, the target object detection section 233 may detect a plurality of physical objects included in the original image.
For example, the upper part of FIG. 6 shows an example of the original image. In this example, a table and a chair are each detected as real objects.

Next, the target object detection unit 233 calculates the feature amount of the virtual object acquired in step S1 and the feature amount of the real object detected in step S3. Then, the similarity of the feature amount between the virtual object and the real object is calculated (step S4).
Note that the feature amount of the virtual object may not be calculated by the target object detection unit 233, but may be included in the information of the virtual object acquired in step S1. Also, the feature amount of the virtual object corresponds to the feature amount of the user-selected product to which the virtual object corresponds.

Next, the target object detection unit 233 determines whether or not there is a physical object whose feature amount similarity to the virtual object is equal to or greater than a threshold value (step S5). If No in step S5, the threshold is decreased by a predetermined value (step S6), and the process returns to step S5.

On the other hand, in the case of Yes in step S5, the target object detection unit 233 identifies, as the target object, the real object whose feature amount similarity with the virtual object is equal to or greater than the threshold (step S7). The above steps S3 to S7 correspond to the target object detection step of the present invention.
According to the target object detection step (steps S3 to S7) of the present embodiment, the physical object having the highest degree of similarity to the user-selected product among the physical objects present in the original image is detected as the target object.
For example, in the example of the original image shown in the upper part of FIG. 6, if the virtual object corresponds to a product on a rectangular table, the target object is the “real object of the round table” instead of the “real object of the chair”. be done.
Further, if the virtual object corresponds to a square table product and only the "real object of the chair" is detected from the original image, by repeatedly decreasing the threshold value, finally, A "real chair object" is identified as the target object.

Next, the image processing unit 234 processes the original image to generate a processed image (step S8; image processing step).
FIG. 7 is a flow chart showing the image processing in step S8.
As shown in FIG. 7, the image processing unit 234 first calculates various conditions regarding the target object (step S81). Here, the conditions include, for example, the distance from the user terminal 10 to the target object, the arrangement direction of the target object, the background information of the target object, the ground plane information of the target object, and the like.

Note that the image processing unit 234 may calculate the arrangement direction of the target object based on the category of the user-selected product. Specifically, the category of the user-selected product is referred to from the product database 222 and the front information associated with the category is acquired from the image processing database 224 . Then, the arrangement direction of the target object may be calculated by detecting the front of the target object based on the acquired front information. For example, the front information corresponding to the category of the table is information indicating the central portion between any two adjacent legs among the four legs. Further, the front information corresponding to the chair category is, for example, information indicating a portion located on the opposite side of the backrest portion in the horizontal direction.

Next, the image processing unit 234 determines the placement direction of the virtual object based on the placement direction of the target object or the physical objects placed around the target object (step S82).
For example, when the front information can be acquired based on the target object, the rotational position of the virtual object about the vertical central axis is set so that the arrangement direction of the virtual object is the same as the arrangement direction of the target object.
Further, when the front information based on the target object cannot be obtained and when the front information can be obtained based on the real objects arranged around the target object, the rotational position of the virtual object is determined according to the front information. set.
For example, in the example of the original image shown in the upper part of FIG. 6, the front information is not acquired based on the target object (round table), and based on the front information of the real object (chair) placed around it, Set the placement direction of the virtual object (square table).

Next, the image processing unit 234 generates a mask image based on the background information of the target object, and superimposes the mask image on the display area (target area) of the target object in the original image (step S83). This results in a depleted reality process that removes the target object from the original image to produce a depleted image. It should be noted that the process of step S83 can utilize a known impairment reality (DR) technique.
An example of an impaired image is shown in the middle part of FIG. In this example, the image of the table detected as the target object is removed from the original image.

Next, the image processing unit 234 superimposes the virtual object on the degraded image (step S84).
Here, the arrangement direction of the virtual object in the degraded image is the arrangement direction set in step S82. Also, the vertical position of the virtual object in the degraded image is set based on the ground plane on which the real object was grounded. Also, the horizontal central position of the virtual object in the degraded image is set based on the horizontal central position of the real object in the original image.
It should be noted that the process of step S84 can utilize known augmented reality (AR) technology.

As a result, a processed image is generated in which the target object in the original image is replaced with the virtual object. For example, the lower portion of FIG. 6 shows an example of a processed image. In this example, instead of the round table that exists in the real space, a square table that is a virtual object is displayed in the room.

Note that when a plurality of target objects are specified, the processing of steps S81 to S84 may be performed for each target object. In this case, each of the multiple target objects in the original image is replaced with a virtual object.
Alternatively, if multiple target objects are identified, some target objects may be selected. This selection process may be a process based on a user's operation, or may be a process of the image processing unit 234 that selects a target object located in a predetermined area such as the center of the screen.

Returning to FIG. 5, the image processing unit 234 transmits the generated processed image to the user terminal 10 and causes the display 15 of the user terminal 10 to display the processed image (step S9).
For example, FIG. 8 shows examples of the physical space RS and the processed image PI displayed on the display 15 of the user terminal 10 when the physical space RS is captured. As in the example shown in FIG. 8, by capturing an image of the physical space, the user can see a processed image in which the article in the physical space is virtually replaced with the user-selected product. As a result, the user can intuitively know the atmosphere, size, etc., when the article in the physical space is replaced with the user-selected article.

Note that when the camera 16 is capturing images in the moving image mode, the image acquisition unit 231 acquires original images as frame images at predetermined intervals. That is, after step S9, if the predetermined stop condition is not satisfied (step S10; No), the image acquisition unit 231 acquires the next original image (step S11), and returns to step S8. Here, the target object detection unit 233 performs processing for tracking the target object specified in the first flow, and the target object tracked by the target object detection unit 233 is used in the second flow and subsequent flows.
Note that when a predetermined stop condition is satisfied, such as when the user performs a stop operation (step S10; Yes), the flow ends.

[Action and effect of the first embodiment]
The server device 20 of the present embodiment functions as an image acquisition unit 231, a virtual object acquisition unit 232, a target object detection unit 233, and an image processing unit 234 by the control unit 23 reading and executing an image processing program. The virtual object acquisition unit 232 acquires the virtual object of the user-selected product based on the information on the user-selected product, and the image acquisition unit 231 acquires the original image captured by the camera 16 . The target object detection unit 233 detects the target object from the original image based on the information on the user-selected product. The image processing unit 234 generates a processed image by replacing the target object in the original image with a virtual object.
In this embodiment, by acquiring the original image, the server device 20 can automatically replace an appropriate physical object in the original image with the virtual object of the user-selected product. As a result, the user can easily replace the article in the original image without having to move the article in the physical space or set the degree of depletion reality.

In this embodiment, the target object detection unit 233 detects, as the target object, a physical object whose similarity to the user-selected product is equal to or greater than a threshold among the physical objects present in the original image.
With such a configuration, even if the original image contains multiple types of physical objects, the physical object related to the user-selected product can be easily detected as the target object.

In this embodiment, the target object detection unit 233 decreases the threshold when the degree of similarity between the physical object and the virtual object is smaller than the threshold.
According to such a configuration, even if the real object in the original image is greatly different from the product selected by the user, it can be detected as the target object.

In this embodiment, the image processing unit 234 sets the placement direction of the virtual object in the processed image based on the placement direction of the target object or the physical objects placed around the target object.
With such a configuration, the virtual objects can be arranged with a natural look without the user's trouble.

[Second embodiment]
Next, a second embodiment will be described.
In the first embodiment described above, the target object detection unit 233 detects the target object related to the user-selected product from the original image. That is, in the first embodiment, it is necessary for the user to select the product in advance. On the other hand, in the second embodiment, the target object detection unit 233 detects a real object arranged in a predetermined area of the original image as the target object, and the user does not need to select the product in advance. It is different from the first embodiment in that point.

An image processing method in the server device 20 of the second embodiment will be described below. The server device 20 executes the image processing program based on the operation information received from the user terminal 10, and performs the image processing described below.

FIG. 9 is a flow chart showing image processing in the server device 20 of the second embodiment. In addition, in FIG. 9, the same code|symbol is attached|subjected to the same process as 1st Embodiment.
First, the image acquisition unit 231 transmits a camera activation request to the user terminal 10 . Upon receiving the request, the user terminal 10 transmits the original image captured by the camera 16 to the server device 20, as in the first embodiment. Thereby, the image acquiring unit 231 acquires the original image transmitted from the user terminal 10 (step S2).

Next, the target object detection unit 233 detects, as a target object, a physical object arranged in a predetermined area in the acquired original image (step S21). Here, the predetermined area can be arbitrarily set within the original image, and may be, for example, the central area within the original image, or the area where the camera 16 is focused in the original image.

Next, the virtual object acquisition unit 232 acquires the virtual object of the article related to the target object (step S22).
For example, the virtual object acquisition unit 232 analyzes the target object and estimates which of the products registered in the product database 222 corresponds to the target object. Then, based on the product estimated to correspond to the target object (estimated product), the virtual object of the product belonging to the same category as the estimated product may be acquired.
Alternatively, the virtual object acquisition unit 232 may calculate the feature amount of the target object and acquire virtual objects whose similarity between the target objects is equal to or greater than a threshold.
To simplify the explanation, it is assumed here that any one virtual object is acquired, and the subsequent processing is the same as in the first embodiment.

According to the above processing, the user can view a processed image in which an article placed in a predetermined area of the physical space is virtually replaced with a product related to the article by capturing an image of the physical space.

[Action and effect of the second embodiment]
In the second embodiment, the target object detection unit 233 detects a target object arranged in a predetermined area from the original image.
In the second embodiment as described above, the server device 20 automatically replaces an article arranged in a predetermined area in the original image with a virtual object of an arbitrary article by acquiring the original image. This allows the user to easily replace the article in the image without moving the article in the physical space or setting the degree of depletion reality.

Also, in the second embodiment, the virtual object acquisition unit 232 acquires the virtual object of the product based on the target object, so the user does not need to select the product in advance. Therefore, when there is an article that the customer wants to purchase, but the article to be purchased has not been decided yet, the server device 20 can propose a candidate article for purchase.

[Modification]
The present invention is not limited to the above-described embodiments, and includes modifications shown below as long as the object of the present invention can be achieved.

(Modification 1)
In each of the embodiments described above, the case where the virtual object acquisition unit 232 acquires one virtual object has been described for the sake of simplification of description, but the present invention is not limited to this. That is, the above description can be applied even when acquiring a plurality of virtual objects.

For example, in the first embodiment, when a certain physical object has a degree of similarity equal to or greater than a threshold value with a plurality of virtual objects, the virtual object acquiring unit 232 associates the physical object with the plurality of virtual objects, A real object is detected as a target object. Then, the target object may be replaced with any virtual object associated with the target object, and the virtual object to be placed may be switched at predetermined time intervals.
Further, when there are a plurality of target objects associated with a plurality of virtual objects, a combination of virtual objects to be arranged may be created, and the combination may be switched to a different combination every predetermined time.

Further, in the second embodiment, the number of acquisitions of virtual objects acquired by the virtual object acquisition unit 232 may be set to an arbitrary number in advance. Then, when a plurality of virtual objects are acquired, the virtual object to be replaced with the target object may be switched at regular intervals.
Such a modified example is particularly useful when the image processing system 1 forms part of an electronic commerce system. For example, since a plurality of virtual objects can be sequentially substituted for a product in the original image without requiring the user's trouble, the user can easily compare and consider a plurality of products.

(Modification 2)
In the first embodiment described above, a physical object whose degree of similarity to a virtual object is equal to or greater than a threshold is detected as a target object, but the present invention is not limited to this.
For example, the target object detection unit 233 may analyze the obtained physical object and determine the category to which the physical object corresponds. Then, the target object detection unit 233 may detect a real object belonging to the same category as the virtual object as the target object.
The target object detector 233 can also use any other method such as machine learning to detect the target object.

(Modification 3)
In the first embodiment, the virtual object acquisition unit 232 acquires the virtual object of the user-selected product based on the information of the product (user-selected product) stored in the history information of the user database 221. The invention is not so limited. For example, based on product information stored in the recommendation information of the user database 221, the virtual object of the product may be acquired.

(Modification 4)
In the second embodiment, in step S22, the virtual object acquisition unit 232 acquires the virtual object based on the target object, but may acquire the virtual object based on the product information included in the user information.
Here, the product information included in the user information is, for example, product information registered in history information (browsing history, bookmark information, cart information, purchase history, etc.) or recommendation information.
According to such a modification, as in the second embodiment, it is possible to receive from the server device 20 a proposal for a product that can be a candidate for purchase. Also, in such a modified example, it is possible to receive proposals for articles of various categories without being bound by the category of the target object.

(Modification 5)
In each of the above-described embodiments, the server apparatus 20 exemplifies the configuration for performing the above image processing, but the user terminal 10 may perform the above image processing. That is, in the user terminal 10 in which the image processing program of the present invention is installed, the terminal control unit 14 reads out and executes the image processing program, so that the image acquisition unit, the virtual object acquisition unit, and the target object detection unit of the present invention and may function as an image processing unit.
Also, the user terminal 10 may perform part of the processing in the server device 20 . For example, the server device 20 may transmit the mask image and the virtual object to the user terminal 10, and the user terminal 10 may superimpose the received mask image and virtual object on the original image. That is, the image processing apparatus of the present invention may be configured as a system composed of a plurality of communicable computers.

(Modification 6)
Articles of the present invention are not limited to furniture. That is, the present invention can be applied to various articles such as clothing, accessories, home appliances, or building equipment, which are expected to be replaced or renewed.

(Modification 7)
Although the image processing system 1 is part of the electronic commerce system in each of the above embodiments, the present invention is not limited to this. That is, the image processing system 1 of the present invention may be a stand-alone system or may be combined with another system. Also, the article of the present invention is not limited to being in the form of a product, and may be an article not for sale or a transferable article.

In addition, the specific structure and procedure for carrying out the present invention can be appropriately changed to another structure within the scope of achieving the object of the present invention.

DESCRIPTION OF SYMBOLS 1... Image processing system 10... User terminal 11... Terminal communication part 12... Input operation part 13... Terminal storage part 14... Terminal control part 15... Display 16... Camera 20... Server device 21... Communication unit 22 Storage unit 221 User database 222 Product database 223 Virtual object database 224 Image processing database 23 Control unit 231 Image acquisition unit 232 Virtual object acquisition unit 233 target object detection section, 234... image processing section;

Claims (10)

a virtual object acquisition unit that acquires a virtual object of the article based on the article information included in the user information;
an image acquisition unit that acquires an original image captured by an imaging device;
a target object detection unit that detects a target object from the original image based on the information of the article;
an image processing unit that generates a processed image in which the target object in the original image is replaced with the virtual object ;
The image processing apparatus , wherein when the virtual object acquisition unit acquires a plurality of the virtual objects, the image processing unit switches the virtual object to be replaced with the target object at predetermined time intervals . The image processing device according to claim 1,
The target object detection unit includes:
An image processing apparatus that detects, as the target object, a physical object having a degree of similarity to the product equal to or greater than a threshold among physical objects present in the original image. The image processing device according to claim 2,
The image processing apparatus, wherein the target object detection unit reduces the threshold when the degree of similarity between the physical object and the virtual object is smaller than the threshold. an image acquisition unit that acquires an original image captured by an imaging device used by a user;
a target object detection unit that detects a target object arranged in a predetermined area set in the original image from the original image;
a virtual object acquisition unit that acquires a virtual object of an article;
an image processing unit that generates a processed image in which the target object in the original image is replaced with the virtual object ;
The image processing apparatus , wherein when the virtual object acquisition unit acquires a plurality of the virtual objects, the image processing unit switches the virtual object to be replaced with the target object at predetermined time intervals . The image processing device according to claim 4,
The image processing apparatus, wherein the virtual object acquisition unit acquires the virtual object of the article based on the target object or information of the article included in user information of the user. . The image processing device according to any one of claims 1 to 5,
The image processing device, wherein the image processing unit sets the arrangement direction of the virtual object in the processed image based on the arrangement direction of the target object. An image processing method for performing image processing by a computer,
a virtual object acquisition step of acquiring a virtual object of the article based on the information of the article included in the user information;
an image acquisition step of acquiring an original image captured by the imaging device;
a target object detection step of detecting a target object from the original image based on the information of the article;
an image processing step of generating a processed image in which the target object in the original image is replaced with the virtual object ;
The image processing method, wherein in the virtual object acquiring step, when a plurality of the virtual objects are acquired, the image processing step switches the virtual object to be replaced with the target object at predetermined time intervals . An image processing method for performing image processing by a computer,
an image acquisition step of acquiring an original image captured by an imaging device used by a user;
a target object detection step of detecting a target object arranged in a predetermined area set in the original image from the original image;
a virtual object obtaining step of obtaining a virtual object of an article;
an image processing step of generating a processed image in which the target object in the original image is replaced with the virtual object ;
The image processing method, wherein in the virtual object acquiring step, when a plurality of the virtual objects are acquired, the image processing step switches the virtual object to be replaced with the target object at predetermined time intervals . An image processing program that is read and executed by a computer,
said computer,
a virtual object acquisition unit that acquires a virtual object of an article based on the article information included in the user information;
an image acquisition unit that acquires an original image captured by an imaging device;
a target object detection unit that detects a target object from the original image based on the information of the article; and
function as an image processing unit that generates a processed image in which the target object in the original image is replaced with the virtual object ;
The image processing program , wherein when the virtual object acquisition unit acquires a plurality of the virtual objects, the image processing unit switches the virtual object to be replaced with the target object at predetermined time intervals . An image processing program that is read and executed by a computer,
said computer,
an image acquisition unit that acquires an original image captured by an imaging device used by a user;
a target object detection unit that detects a target object arranged in a predetermined area set in the original image from the original image;
a virtual object acquisition unit that acquires a virtual object of an article; and
function as an image processing unit that generates a processed image in which the target object in the original image is replaced with the virtual object ;
The image processing program , wherein when the virtual object acquisition unit acquires a plurality of the virtual objects, the image processing unit switches the virtual object to be replaced with the target object at predetermined time intervals .

Images