JP2024037556A - Information processing device and program - Google Patents

Abstract

To acquire transmittance of a transparent region of an image even if no special device or environment is used.SOLUTION: An information processing device includes a processor that defines an object region 53a as a region of an object including a transparent region from an image, acquires information indicating a first region 54a as a region other than the object region 53a in the image, a second region 54b as the transparent region and a third region 54c as a region other than the second region 54b in the object region 53a, from the image and the object region 53a, and determines transmittance of the second region 54b on the basis of the image and the information.SELECTED DRAWING: Figure 8

Description

The present invention relates to an information processing device and a program.

In the field of image processing, such as creating web advertisements and pop advertisements, objects in an image are composited onto another background, but when pasting an image of an object that includes transparent areas onto another background, , the original background is reflected in the transparent area. For example, Patent Document 1 discloses a prior patent that calculates the transparency of a transparent area necessary for high-precision compositing. A technique is disclosed in which background attributes are determined for each pixel of image data, and transparency data and color data of a transparent object are calculated based on color data of pixels for each attribute. Further, in Patent Document 2, a measuring device includes an imaging position acquisition unit and a physical quantity calculation unit, and the imaging position acquisition unit is configured to detect a state in which a space or a space and a transparent object existing in the space are interposed between the imaging position acquisition unit and the background. The physical quantity calculation unit calculates the physical quantity of the transparent object based on the position in the captured image corresponding to the background imaging position and the background imaging position for the captured image. A technique for calculating is disclosed.

Japanese Patent Application Publication No. 2001-143085 Japanese Patent Application Publication No. 2018-159671

Here, when obtaining the transparency of a transparent region of an image using a special device or environment, it is difficult to reduce the burden on the user who obtains the transparency.
An object of the present invention is to obtain the transparency of a transparent area of an image without using any special equipment or environment.

The invention according to claim 1 includes a processor, and the processor determines an object area that is an area of the object including a transparent area from an image, and determines the object area in the image from the image and the object area. obtaining information indicating a first area that is an area other than the area, a second area that is the transparent area of the target object area, and a third area that is an area other than the second area;
The information processing apparatus determines the transparency of the second area based on the image and the information.
The invention according to claim 2 is characterized in that the acquired information is image information consisting of three values indicating each of the first area, the second area, and the third area. The information processing device according to item 1.
In the invention according to claim 3, the values indicating each of the second area and the third area are determined by the difference between the color information of the first area and the color information of the second area and the third area. 3. The information processing apparatus according to claim 2, characterized in that: .
The invention according to claim 4 is the information processing apparatus according to any one of claims 1 to 3, wherein the target object area is determined using a machine learning model.
The invention according to claim 5 is characterized in that the object area is any one of a plurality of objects included in the image. It is an information processing device.
The invention according to claim 6 is the information processing apparatus according to claim 1, characterized in that the information processing apparatus receives a correction instruction from a user regarding the second area displayed on the screen.
The invention according to claim 7 is characterized in that the modification instruction is to change the boundary between the second area and the third area. It is a processing device.
The invention according to claim 8 is characterized in that the boundary change selects either the second area or the third area and changes a specified portion in the other area to the one. 8. The information processing apparatus according to claim 7.
The invention according to claim 9 is the information processing apparatus according to claim 6, wherein the modification instruction is to change the degree of transparency.
The invention according to claim 10 provides a computer with a function of determining an object area that is an area of an object including a transparent area from an image, and a function of determining an object area other than the object area in the image from the image and the object area. a first area that is a region, a second area that is the transparent area of the target object area, and a third area that is an area other than the second area; This program implements a function of determining the degree of transparency of the second area based on information.

According to the first aspect of the invention, it is possible to obtain the transparency of a transparent area of an image without using any special equipment or environment.
According to the invention of claim 2, it is possible to reduce the burden of information processing compared to the case where the information to be acquired is not information of an image consisting of three values indicating each of the first area, the second area, and the third area. can.
According to the invention of claim 3, when the configuration in which the values indicating each of the second area and the third area are determined by the difference between the color information of the first area and the color information of the second area and the third area is not adopted. In comparison, it is possible to reduce the burden of information processing.
According to the fourth aspect of the invention, it is possible to reduce the burden of information processing compared to a case where a configuration in which the target object area is determined using a machine learning model is not adopted.
According to the fifth aspect of the invention, it is possible to improve the accuracy of transparency compared to a case where the object area is any one of a plurality of objects included in the image.
According to the invention of claim 6, the usability can be improved compared to the case where a configuration is not adopted in which a modification instruction from a user is accepted for the second area displayed on the screen.
According to the invention of claim 7, the usability can be improved compared to the case where a configuration in which the modification instruction changes the boundary between the second area and the third area is not adopted.
According to the invention of claim 8, the operability is improved compared to the case where a configuration in which the boundary change selects one of the second area and the third area and changes a designated portion in the other area to one is not adopted. It is possible to improve the
According to the invention of claim 9, the usability can be improved compared to the case where a configuration in which the modification instruction changes the transparency is not adopted.
According to the tenth aspect of the invention, it is possible to obtain the transparency of a transparent area of an image without using any special equipment or environment.

1 is a diagram showing an example of the overall configuration of an image processing system to which this embodiment is applied. 1 is a diagram illustrating an example of a hardware configuration of an image processing apparatus according to the present embodiment. FIG. 2 is a diagram showing an example of the hardware configuration of a mobile terminal according to the present embodiment. 1 is a block diagram illustrating an example of a functional configuration of an image processing apparatus according to a first embodiment; FIG. It is a figure explaining the processing of a target object area estimation part. FIG. 3 is a diagram illustrating processing of a try map generation unit. FIG. 3 is a diagram illustrating processing of a try map generation unit, in which (a) is a diagram illustrating a mask image and pixels of the image, and (b) is a table illustrating calculation results of differences in pixel values. FIG. 3 is a diagram illustrating processing of a try map generation unit. FIG. 3 is a diagram illustrating an image when an image and a new background are combined, where (a) shows a combined image when combined by processing of the image processing device according to the first embodiment, and (b), A composite image as a comparative example when composited using conventional processing is shown. FIG. 2 is a block diagram illustrating an example of the functional configuration of an image processing device according to a second embodiment. It is a figure explaining a user correction part, (a), (b), and (c) are figures explaining the operation which changes transparency. FIG. 7 is a block diagram illustrating an example of a functional configuration of an image processing device according to a third embodiment. It is a figure explaining the processing of a user correction part, (a) shows before processing of a user correction part, (b) shows after processing. FIG. 7 is a diagram illustrating processing of an image processing apparatus according to a fourth embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Overview of this embodiment]
In this embodiment, an object area that is an area of the object including a transparent area is determined from an image, and a first area that is an area other than the object area in the image and a first area of the object area are determined from the image and the object area. Provided is an information processing device that obtains information indicating a second area that is a transparent area and a third area that is an area other than the second area, and determines the degree of transparency of the second area based on the image and information. .

Here, the transparent area refers to an area in an image where the background of the object is visible through a transparent member that transmits light. In addition to having a flat surface, such a transparent member may have a partially or entirely curved or bent surface, or such a surface may be regular or irregular. Further, the background corresponds to the first area, and the transparent area corresponds to the second area.
Transmittance refers to the degree of light passing through the second region.
The information processing apparatus will be described using an image processing apparatus 10 (see FIG. 1) that performs image processing as an example, but it may also be a mobile terminal 20 (see FIG. 1).

[Overall configuration of image processing system]
FIG. 1 is a diagram showing an example of the overall configuration of an image processing system 1 to which this embodiment is applied. As illustrated, this image processing system 1 includes an image processing device 10 and a mobile terminal 20. The image processing device 10 is connected to a communication line 30. The mobile terminal 20 can be wirelessly connected to the communication line 30 via the access point 40 by wireless communication. Note that in the figure, only one image processing device 10 and one mobile terminal 20 are shown, but a plurality of them may exist. Furthermore, the communication line 30 may be, for example, a LAN (Local Area Network) or the Internet.

The image processing device 10 is a device that processes images from the mobile terminal 20, and estimates a target object area based on the image, and uses the image and the estimation result to determine transparent areas outside the target area, transparent areas inside the target object area, An image (hereinafter referred to as a tri-map image) representing a non-transparent region within the object is generated, and the degree of transparency is estimated based on the image and the tri-map image. Note that the image processing device 10 may be realized by, for example, a personal computer (PC).

The mobile terminal 20 is a terminal device that captures an image of a target object and transmits the image to the image processing device 10. A camera application is preferably installed on the mobile terminal 20. This camera application captures an image of an object and transmits the captured image to the image processing device 10, for example, by an operation of an operator of the mobile terminal 20. Note that the mobile terminal 20 may be realized by, for example, a smartphone.

[Hardware configuration of image processing device]
FIG. 2 is a diagram showing an example of the hardware configuration of the image processing device 10 in this embodiment. As shown in the figure, the image processing device 10 includes a processor 10a, a RAM (Random Access Memory) 10b, an HDD (Hard Disk Drive) 10c, and a communication interface (hereinafter referred to as "communication I/F") 10d. , a display device 10e, and an input device 10f.

The processor 10a executes various software such as an OS (Operating System) and applications to realize various functions described below.

The RAM 10b is a memory used as a working memory of the processor 10a. The HDD 10c is, for example, a magnetic disk device that stores input data for various software, output data from various software, and the like.

The communication I/F 10d sends and receives various information to and from the mobile terminal 20 via the communication line 30.
The display device 10e is, for example, a display that displays various information. The input device 10f is, for example, a keyboard or mouse used by the user to input information.

[Hardware configuration of mobile terminal]
FIG. 3 is a diagram showing an example of the hardware configuration of the mobile terminal 20 in this embodiment. As illustrated, the mobile terminal 20 includes a processor 21, a RAM 22, a ROM 23, an input/output device 24, an audio input mechanism 25, an audio output mechanism 26, an imaging mechanism 27, a wireless circuit 28, and an antenna 29. Equipped with.

The processor 21 implements each function of the mobile terminal 20 by loading various programs stored in the ROM 23 and the like into the RAM 22 and executing them.

The RAM 22 is a memory used as a working memory of the processor 21 and the like. The ROM 23 is a memory that stores various programs executed by the processor 21.

The input/output device 24 is a device that displays various information and receives operation input from the user, and is, for example, a touch panel. The audio input mechanism 25 is a device that inputs audio from the outside, and is, for example, a microphone. The audio output mechanism 26 is a device that outputs audio to the outside, and is, for example, a speaker.

The imaging mechanism 27 is a device that images a subject, and is, for example, a camera.

The wireless circuit 28 is a circuit that controls wireless communication. The antenna 29 is a device that transmits a wireless communication signal output from the wireless circuit 28 or receives a wireless communication signal and outputs it to the wireless circuit 28. Wireless communication includes communication based on wireless communication standards such as 5G (5th Generation), LTE (Long Term Evolution), Wi-Fi (registered trademark), Bluetooth (registered trademark), and NFC (Near Field Communication).

The image processing device 10 according to the present embodiment can estimate the degree of transparency from one image without using special equipment or environments as in the past, and thereby achieve highly accurate automation of composition. . Various embodiments will be described below.

[First embodiment]
FIG. 4 is a block diagram showing an example of the functional configuration of the image processing device 10 according to the first embodiment. As illustrated, the image processing device 10 according to the first embodiment includes an object area estimation section 11, a tri-map generation section 12, and a transparency estimation section 13.

The object area estimation unit 11 estimates an object area, which is an area of the object including a transparent area, from the image. Thereby, the object area estimation unit 11 calculates an object area mask image representing the object area.

The trymap generation unit 12 divides the object area into a transparent area and a non-transparent area based on the image and the object area determined by the object area estimation unit 11. Thereby, the tri-map generation unit 12 generates a tri-map image representing the positions of three areas: the outside of the object area, the non-transparent area and the transparent area within the object area.

The transparency estimation unit 13 estimates the transparency based on the image and the trimap image generated by the trimap generation unit 12. Thereby, the transparency estimating unit 13 prepares a transparency image.

First, the object area estimation unit 11 will be explained using FIG. 5.
FIG. 5 is a diagram illustrating the processing of the target object area estimation unit 11.
The object area estimating unit 11 performs area estimation using a machine learning model 52 from an image 51, as shown in FIG. A mask image 53 representing the area 53a is calculated.

The image 51 is photographed and transmitted by the mobile terminal 20 (see FIG. 1). Further, as shown in FIG. 5, the image 51 is an image of an apple G3 contained in a transparent bag G1 having a square hole G2 cut out for carrying as a handbag. A plurality of vertical lines appear in the background of the image 51. The transparent bag G1 is made of vinyl, for example, but may also be made of a plastic bottle.
The plurality of vertical lines appearing as the background are visible through the transparent bag G1. That is, the plurality of vertical lines appear directly on the outside of the outer edge G4 of the transparent bag G1 and on the square hole G2 of the bag G1, and also on the inside of the outer edge G4 of the transparent bag G1. It is visible in parts other than G2 and apple G3.

The apple G3 contained in the transparent bag G1 is an example of the target object, and the target object area 53a is an example of the area of the target object.

The machine learning model 52 used by the object area estimating unit 11 is trained to estimate an object area 53a including a transparent area. To explain further, the machine learning model 52 is arbitrary, and a deep learning model such as U-Net or DeepLab V3+ having an encoder-decoder structure may be utilized. Furthermore, a two-stage model may be used or the model structure and loss function may be devised for the purpose of improving accuracy.

In the mask image 53, the boundary between the target object region 53a and a region 53b other than the target object region 53a is estimated with high accuracy by the machine learning model 52. Note that the region 53b in FIG. 5 is indicated by diagonal lines.

Next, the try map generation unit 12 will be explained using FIGS. 6, 7, and 8.
6 to 8 are diagrams illustrating the processing of the try map generation unit 12.
FIG. 6 shows a mask image 53 on the left and a trimap image 54 on the right. The mask image 53 has an object area 53a and an area 53b, and the trimap image 54 has an outside area 54a, a transparent area 54b, and a non-transparent area 54c.

The tri-map generation section 12 divides the object region 53a of the mask image 53 estimated by the object region estimation section 11 into a transparent region 54b and a non-transparent region 54c. Then, the trimap generation unit 12 generates a trimap image 54 representing the outside of the object area 54a, the transparent area 54b, and the non-transparent area 54c.
An area 54a outside the target object area of the try map image 54 can be an area 53b of the mask image 53.

Although the method of dividing the transparent area 54b and the non-transparent area 54c is arbitrary, it may be calculated by utilizing color information of the image outside the object area 54a, for example. That is, the trimap generation unit 12 divides the transparent area 54b and the non-transparent area 54c based on the difference between the color information outside the object area 54a and the color information inside the object area.
The outside object area 54a is an example of a first area, the transparent area 54b is an example of a second area, and the non-transparent area 54c is an example of a third area. The try map image 54 is an example of information indicating the first area, the second area, and the third area.

Here, (a) of FIG. 7 is a diagram explaining the pixels of the mask image 53 and the image 51, and (b) is a table showing the calculation result of the difference in pixel values.
As shown in FIG. 7A, the set of pixel values in the area 51a of the image 51 corresponding to the object area 53a of the mask image 53 is Iin, and the pixel values in the area 51b of the image 51 corresponding to the area 53b are Let the set of Iout be Iout. Further, the pixel value of a certain pixel in the area 51a of the image 51 is expressed as m, and the pixel value of a certain pixel in the area 51b of the image 51 is expressed as n.

Then, as shown in FIG. 7B, the difference between the pixel values (RGB) in the image of the target 511 of the target 51a to be classified in the image 51 and the pixel values of all pixels outside the target object area 51b. Calculate and get its minimum value. In the example shown in FIG. 7(b), the minimum value is 9.

Although the explanation will be repeated, Iin represents a set of pixel values within the object region in the image, and Iout represents a set of pixel values outside the object region in the image. m represents an element of Iin, and n represents an element of Iout.
In the above formula, ||・||1 represents the L1 norm, but the method of converting the RGB 3 values into a certain value is arbitrary, and the L2 norm may be used, such as the average value of each component, the sum of squares, You may calculate the square mean or the like.

Further, the color difference calculation is not limited to the RGB space, but may be performed after converting to the Lab space or the HSV space, and the color space to be calculated is arbitrary.

If the minimum value of the difference is less than or equal to the threshold value, the target pixel is determined to be a transparent region 54b because it is close to a background pixel, and if it is larger than the threshold value, it is determined to be a non-transparent region 54c. The threshold value may be set in advance, or may be set automatically, such as by using the difference between the average pixel value within the object area and the average pixel value outside the object area as the threshold value.

As shown in FIG. 8, the trimap generation unit 12 creates a trimap image based on the calculated classification results (see FIG. 7(b)). For example, if Diff is less than or equal to the threshold value, it is set to 0.5, and when it is larger than the threshold value, it is set to 1. More specifically, the trimap image is created by creating an image consisting of three values: 0 for the outside of the object area 54a, 0.5 for the transparent area 54b, and 1 for the non-transparent area 54c. In the example of FIG. 8, the three values indicating each region of the trimap image are 0, 0.5, and 1.0, but the value is not limited to this.
0, 0.5, and 1.0 shown in parentheses in FIG. 8 are examples of three values.

In addition to the color information method described above, there are other ways to create a tri-map image, such as using a deep learning model that receives the image and object area as input and divides the object area into regions. .

Next, the transparency estimation section 13 will be explained.
The transparency estimating unit 13 calculates the transparency in the transparent area based on the image and the trimap image. Here, the transparency is an α mask used in the synthesis method of the following formula.

Here, I represents the image, F represents the foreground component, B represents the background component, and α represents the degree of transparency or opacity of the foreground component.

Alternatively, the transparency is ρ used in Environment Matching.

Here, m is a mask image representing a transparent area, B is a background image, M is an image representing refraction, and ρ represents transparency. In the above equation, M may be set to B if the influence of refraction in the transparent region is not considered. As described above, the definition of transparency is based on the synthesis method, and the synthesis method is not limited to the above two.

The method for calculating transparency is to prepare a large set of images, trimap images, and transparency images, and have deep learning learn the relationships between them to estimate the transparency of the test image. Alternatively, a method of setting constraint conditions and calculating by optimization may be used.

In this way, the first embodiment makes it possible to calculate transparency from an RGB image without using any special equipment or environment. Therefore, when composing an image and a new background, a natural-looking composite image can be created. This will be explained below.

FIG. 9 is a diagram illustrating an image when the image 51 and the new background 55 are combined, and (a) is a combined image 56 when combined by the processing of the image processing device 10 according to the first embodiment. , and (b) shows a composite image 156 as a comparative example when composite is performed using conventional processing.
As shown in FIG. 9A, the image 51 is a background in which a plurality of vertical lines appear, and the plurality of vertical lines can be seen through the transparent bag G1. On the other hand, the new background 55 is free of vertical lines. As described above, in the first embodiment, since the transparency is calculated from one RGB image, the transparent bag G1 is replaced with the new background 55 in the composite image 56. Moreover, in the first embodiment, it is possible to calculate the transmittance without using any special equipment or environment.

On the other hand, in the composite image 156 shown in FIG. 9(b), if the area is simply cut out and pasted as is, a plurality of vertical lines, which are the original background, remain on the transparent bag G1. Therefore, if conventional processing is used instead of the first embodiment, it is difficult to naturally synthesize the images.

In this way, in the first embodiment, by estimating the degree of transparency from one image, it is possible to automate highly accurate composition.
To further explain, in order to further promote automation of high-precision composition, by photographing a new background after photographing the image 51 with the mobile terminal 20, the trouble of specifying the background can be omitted, and the operation can be simplified. This makes it possible to achieve highly accurate synthesis.
Further, the size and position of the object in the image 51 displayed on the screen may be changed by a touch operation or a pinch operation by the user, so that a composite image 56 that reflects the user's intention may be created. For example, even if the new background 55 has a different size from the image 51, it is possible to make the composite image 56 look natural.

[Second embodiment]
FIG. 10 is a block diagram showing an example of the functional configuration of the image processing device 10 according to the second embodiment. As shown in the figure, the image processing device 10 according to the second embodiment includes an object area estimation section 11, a tri-map generation section 12, and a transparency estimation section 13, as in the case of the first embodiment. In addition to the above, the image synthesis section 14 and the user correction section 15, which are not provided in the first embodiment, are also provided.
Therefore, unlike the first embodiment, in which the transparency is determined automatically and the user cannot make fine adjustments, the second embodiment allows the transparency to be adjusted based on the user's instructions for the synthesis result. Changes can be made.

The image compositing unit 14 performs a process of compositing the image 51 and the transparency image into a compositing destination image that is a new background (for example, see FIG. 9). In addition, the image synthesis unit 14 performs processing on transparent areas according to the degree of transparency regarding the synthesis of the area cut out from the image 51 and the synthesis destination image. In this way, the image composition unit 14 creates a composite image.

The user correction unit 15 changes the transparency in the transparent area of the composite image according to a modification instruction from the user, and creates a corrected composite image according to the changed transparency.

FIG. 11 is a diagram illustrating the user correction unit 15, and (a), (b), and (c) are diagrams illustrating an operation for changing transparency.
As shown in FIGS. 11A to 11C, the correction by the user correction unit 15 is performed by operating the slider 62 of the transparency adjustment unit 61 while the composite image 56 is displayed on the screen. The composite image 56 displays a transparent area 54b and a non-transparent area 54c.

When the user performs a sliding operation on the slider 62, the user correction unit 15 receives an instruction to correct the transparency of the transparent area 54b. Then, the user correction unit 15 changes the transparency of the transparent area 54b in the composite image 56 in real time according to the received correction instruction. Note that the non-transparent area 54c is not modified by the slide operation.

In the transparency adjustment section 61, as the slider 62 moves to the left, the transparency of the transparent area 54b decreases, and as the slider 62 moves to the right, the transparency increases. For example, in FIG. 11(a), the transparent area 54b and the background can be distinguished, but in FIG. It becomes difficult. Furthermore, in the same figure (c), it is in a completely transparent state, and the outline of the transparent area 54b is difficult to see.
When the user performs a confirmation operation, the user correction unit 15 creates a corrected composite image (see FIG. 10).

The transparency can be changed by a simple operation of sliding the slider 62, and the transparency can be easily adjusted.
Note that the transparency can be changed, for example, if the transparency is an α mask, by weakening or strengthening α values that are not 0 or 1 as a whole, but the changing method is arbitrary.
The operation of the slider 62 of the transparency adjustment section 61 is an example of a correction instruction by the user, and is an example of changing the transparency.

[Third embodiment]
FIG. 12 is a block diagram showing an example of the functional configuration of the image processing device 10 according to the third embodiment. As illustrated, the image processing device 10 according to the third embodiment includes an object area estimation section 11, a tri-map generation section 12, a transparency estimation section 13, and a user correction section 16. .
Therefore, unlike the first embodiment, in which the transparent and non-transparent regions are automatically determined and the user cannot make fine adjustments, the third embodiment allows the user to modify the regions according to the user's instructions. It can be carried out.
Note that although the image processing device 10 according to the third embodiment does not include the image synthesis unit 14 included in the second embodiment, it may be provided.

The user correction unit 16 receives correction instructions from the user regarding the boundary between the transparent area and the non-transparent area, changes the boundary according to the received correction instruction, and creates a corrected trimap image according to the changed boundary.
Thereafter, the transparency estimation unit 13 creates a transparency image based on the image and the corrected trimap image.

FIG. 13 is a diagram illustrating the processing of the user correction unit 16, in which (a) shows the state before the processing by the user correction unit 16, and (b) shows the state after the processing. The upper rows of each of (a) and (b) are overwritten images 56 obtained by overwriting the boundary between the transparent area and the non-transparent area on the image, and the lower rows of each are the tri-map images 54.
As shown in FIG. 13(a), when the overwritten image 56 is displayed on the screen, the user checks the transparent area 54b and the non-transparent area 54c and finds that the image of apple G3 and the non-transparent area 54c match. do not have. That is, there is a region 57 in which the non-transparent region 54c protrudes toward the transparent region 54b. This protruding area 57 also exists in the trimap image 54 shown in the lower part of FIG.

Therefore, the user performs a touch operation on the transparent area 54b of the overwritten image 56 with the mouse as in step P1 of FIG. 13A, and then performs a drag operation to move it toward the protruding area 57 as in step P2. . Through such a user operation, the user correction unit 16 corrects the tri-map image 54 so that the protruding area 57 becomes smaller.

When the user operation to reduce the protruding area 57 is completed, the protruding area 57 disappears in the overwritten image 56 shown in FIG. 13(b), and the non-transparent area 54c matches the image of the apple G3. In this example, the user correction unit 16 performs correction to delete the protruding area 57 caused by the user's operation from the trimap image 54 shown in the lower part of FIG. 2(b).

In addition, in FIG. 13, the correction instruction for the protruding area 57 is explained in the case of performing a touch operation on the transparent area 54b before the drag operation, but the present invention is not limited to this, and may be performed when a touch operation is performed on the non-transparent area 54c.
Further, after touching the protruding area 57, the correction instruction may be given by performing a mouse operation to trace the outline of the apple G3.

In this manner, the user correction unit 16 allows the user to trace the area using a touch operation or a mouse operation to change or add the area to the transparent area 54b or non-transparent area 54c.

The user operation on the screen is an example of a correction instruction by the user, and is an operation to change the transparent area 54b relative to the non-transparent area 54c, and is an operation to change the boundary between the transparent area 54b and the non-transparent area 54c. This is an example of a boundary change.
Further, a correction instruction by touching the transparent area 54b or the non-transparent area 54c with a mouse is an example of selecting either one of the transparent area 54b or the non-transparent area 54c, and a correction instruction by dragging the mouse is an example of selecting one of the transparent area 54b or the non-transparent area 54c. This is an example of changing a specified part to one side.

[Fourth embodiment]
FIG. 14 is a diagram illustrating processing of the image processing device 10 according to the fourth embodiment.
The fourth embodiment shown in the figure is a case where, when an image 58 including a plurality of objects is displayed on the screen, necessary portions are cut out and the above-described processing is performed on the cut out image 51.
More specifically, the image 58 on the left side displayed on the screen includes an apple G3 in a transparent bag G1 and a lemon G5 in a transparent bag G1.

When the user wants to use the apple G3 in the transparent bag G1, as shown in the central image 58, the user specifies the lower right position of the rectangle to be trimmed with the point 58a, and defines the range of the trimming frame 58b on the screen. .
In this way, the area outside the trimming frame 58b is removed, and the area surrounded by the trimming frame 58b is cut out as the image 51. Therefore, it is possible to cope with the case where there are a plurality of objects in the transparent bag G1, and it is also possible to improve the accuracy of the transparency of the transparent region 54b (see, for example, FIG. 6).
The apple G3 in the transparent bag G1 and the lemon G5 in the transparent bag G1 included in the image 58 are examples of a plurality of objects included in the image. Note that the object area 53a to the transparent area 54b and the non-transparent area 54c (see FIG. 6, for example) are examples of object areas.

In each of the above embodiments, the processor refers to a processor in a broad sense, and includes a general-purpose processor (e.g., CPU: Central Processing Unit, etc.), a dedicated processor (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, etc.) FPGA: Field Programmable Gate Array, programmable logic device, etc.)
Further, the operation of the processor in each of the above embodiments may be performed not only by one processor, but also by a plurality of processors located at physically separate locations. Further, the order of each operation of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate.

<Additional notes>
(((1)))
a processor, the processor determines an object area that is an area of the object including a transparent area from an image, and determines from the image and the object area a first area that is an area other than the object area in the image; and a second region that is the transparent region of the target object region and a third region that is a region other than the second region,
determining the transparency of the second area based on the image and the information;
Information processing device.
(((2)))
The information to be acquired is image information consisting of three values indicating each of the first area, the second area, and the third area (((1))). Information processing device.
(((3)))
The value indicating each of the second area and the third area is determined by the difference between the color information of the first area and the color information of the second area and the third area. The information processing device described in 2))).
(((Four)))
The information processing device according to any one of ((1)) to ((3)), wherein the target object area is determined using a machine learning model.
(((Five)))
According to any one of (((1))) to (((4))), the object area is any one of a plurality of objects included in the image. information processing equipment.
(((6)))
The information processing apparatus according to ((1)), further comprising the step of accepting a correction instruction from a user regarding the second area displayed on the screen.
(((7)))
The information processing device according to ((6))), wherein the modification instruction is to change the boundary between the second region and the third region.
(((8)))
In ((7))), the boundary change is characterized in that one of the second area and the third area is selected and a portion specified in the other area is changed to the selected one. The information processing device described.
(((9)))
The information processing device according to ((6))), wherein the modification instruction is to change the transparency.
(((Ten)))
to the computer,
A function to determine an object area, which is an area of the object including a transparent area, from an image;
From the image and the object area, a first area which is an area other than the object area in the image, a second area which is the transparent area in the object area, and an area other than the second area. a third area;
a function of determining the transparency of the second area based on the image and the information;
A program that makes this possible.

According to the invention (((1))), it is possible to obtain the transparency of a transparent area of an image without using any special equipment or environment.
According to the invention (((2))), the burden of information processing is reduced compared to the case where the information to be acquired is not image information consisting of three values indicating each of the first area, the second area, and the third area. can be achieved.
According to the invention (((3))), the value indicating each of the second area and the third area is determined by the difference between the color information of the first area and the color information of the second area and the third area. Compared to the case where this method is not adopted, the burden of information processing can be reduced.
According to the invention (((4))), it is possible to reduce the burden of information processing compared to the case where a configuration in which the object region is determined using a machine learning model is not adopted.
According to the invention (((5))), it is possible to improve the accuracy of transparency compared to the case where the object area is one of a plurality of objects included in the image. .
According to the invention (((6))), the usability can be improved compared to the case where a configuration is not adopted that accepts correction instructions from the user for the second area displayed on the screen.
According to the invention (((7))), the usability is improved compared to the case where a configuration in which the modification instruction changes the boundary between the second area and the third area is not adopted. be able to.
According to the invention (((8))), when the boundary change does not adopt a configuration in which one of the second area and the third area is selected and a specified portion in the other is changed to one of the areas, Compared to this, it is possible to improve the operability.
According to the invention (((9))), the usability can be improved compared to the case where a configuration in which the modification instruction changes the transparency is not adopted.
According to the invention (((10))), it is possible to obtain the transparency of a transparent area of an image without using any special equipment or environment.

DESCRIPTION OF SYMBOLS 10... Image processing device, 10a... Processor, 51... Image, 53a... Object area, 54... Trimap image, 54a... Outside object area, 54b... Transparent area, 54c... Non-transparent area, G1... Transparent bag, G3…Apple

Claims (10)

Equipped with a processor,
The processor includes:
Determine an object area that is an area of the object including a transparent area from the image,
From the image and the object area, a first area which is an area other than the object area in the image, a second area which is the transparent area in the object area, and an area other than the second area. obtain information indicating the third area;
determining the transparency of the second area based on the image and the information;
Information processing device. The information processing apparatus according to claim 1, wherein the information to be acquired is image information consisting of three values indicating each of the first area, the second area, and the third area. 2. A value indicating each of the second area and the third area is determined by a difference between color information of the first area and color information of the second area and the third area. The information processing device described in . The information processing apparatus according to any one of claims 1 to 3, wherein the target object area is determined using a machine learning model. The information processing apparatus according to any one of claims 1 to 3, wherein the object area is any one of a plurality of objects included in the image. The information processing apparatus according to claim 1, wherein the information processing apparatus receives a correction instruction from a user regarding the second area displayed on the screen. 7. The information processing apparatus according to claim 6, wherein the modification instruction is to change a boundary between the second area and the third area. 8. The information processing according to claim 7, wherein the boundary change selects one of the second area and the third area and changes a specified portion in the other area to the selected one. Device. 7. The information processing apparatus according to claim 6, wherein the modification instruction is to change the degree of transparency. to the computer,
A function to determine an object area, which is an area of the object including a transparent area, from an image;
From the image and the object area, a first area which is an area other than the object area in the image, a second area which is the transparent area in the object area, and an area other than the second area. a third area;
a function of determining the transparency of the second area based on the image and the information;
A program that makes this possible.