JP5867133B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The disclosed technology relates to an image processing apparatus, an image processing method, and an image processing program.

  There are known services for distributing various images (including moving images) such as maps, satellite photographs, and aerial photographs to users. In such a service, a form in which a superimposed image obtained by superimposing an additional image such as an advertisement (hereinafter referred to as a superimposing image) on a part of an original image used by a user is also known.

  Conventionally, in the above service, in order not to impair the convenience of the user, a fixed area previously determined as an area not noticed by the user (hereinafter referred to as a non-attention area), for example, an edge of the original image It is customary to superimpose a superimposing image on a part or the like with a small size.

JP 2008-8700 A JP-A-2005-331413

  However, depending on the image used by the user and the purpose of use of the image, the position and size of the important area that the user actually pays attention is not fixed, so the non-focused area is not fixed. There is a case. For example, if a user wants a satellite photo of some typhoon clouds, the area where the clouds appear is likely to be important if the user is focusing on the clouds. If the user focuses on the situation, the area where the clouds are reflected is likely not important. In addition, the size and position of the area where the clouds are reflected vary depending on the image.

  In the disclosed technology, even when the position and size of the attention area noted by the user varies for each image used by the user, the superimposed image is displayed in an area that does not impair the convenience of the user. The purpose is to superimpose.

In the disclosed technique, an image to be superimposed is selected from a plurality of images in accordance with an image selection instruction, and based on the image selection instruction or input information about a target object, the selected image to be superimposed is selected. The attention area noted by the user is identified from the inside. In addition, the disclosed technology is based on the shape and size of the non-attention region excluding at least the attention region, and the shape and size of the image for superimposition among the selected images to be superimposed. A superimposing area where the superimposing image can be superimposed is determined. The image for superimposition is stored in advance in the storage unit as an image to be superimposed on the selected image to be superimposed. In the disclosed technique, the superimposition image is superimposed on the determined superimposition region of the superimposition target image.
When determining the superimposition region, a region having the same aspect ratio as that of the superimposition image and having a maximum area that does not deviate from the non-target region is obtained. It is determined as a superimposing region that can be superposed at a certain magnification. Then, the image for superimposition is superimposed on the determined overlapping region at a magnification that maximizes the area.

According to another aspect of the disclosed technology, an image to be superimposed is selected from a plurality of images in response to an image selection instruction, and the selection is performed based on the image selection instruction or input information about a target object. A non-attention area that is not noticed by the user is identified from the images to be superimposed. The disclosed technique superimposes the superimposing image in the non-focused region based on the shape and size of the specified non-focused region in the selected superimposition target image and the shape and size of the superimposing image. The overlapping region that can be determined is determined. In the disclosed technique, the superimposition image is superimposed on the determined superimposition region of the superimposition target image.
When determining the superimposition region, a region having the same aspect ratio as that of the superimposition image and having a maximum area that does not deviate from the non-target region is obtained. It is determined as a superimposing region that can be superposed at a certain magnification. Then, the image for superimposition is superimposed on the determined overlapping region at a magnification that maximizes the area.

  In the disclosed technology, even when the position and size of the attention area noted by the user varies for each image used by the user, the superimposed image is displayed in an area that does not impair the convenience of the user. It has the effect that it can superimpose.

It is a schematic block diagram of the image processing apparatus which concerns on 1st-4th embodiment. It is a figure which shows an example of an original image table. It is a figure which shows an example of a recognition result table. It is a figure which shows an example of the image table for superimposition. It is a schematic block diagram of the image recognition part which concerns on 1st-4th embodiment. It is explanatory drawing explaining an example of an image recognition process. It is explanatory drawing explaining the specific example of the character recognition process of a character recognition part. It is a schematic block diagram which shows an example of the hardware constitutions in the aspect which implement | achieves an image processing apparatus by a program. It is a flowchart which shows an example of the image process which concerns on 1st Embodiment. It is a flowchart which shows an example of attention area specific processing. It is a flowchart which shows an example of the largest area search process. It is explanatory drawing which illustrates notionally the flow of a process from the input of image selection information to specifying an attention area. It is a figure which shows the specific example of the image for superimposition, a non-attention area | region, and a superimposition area | region. It is explanatory drawing which illustrates typically the maximum area search process shown in FIG. It is a figure which shows an example of the search result in the case of searching for a superimposition area | region from the non-attention area | region, fixing an aspect ratio. (A) is a figure which shows the example of an object recognition result of the original image of superimposition object, (B) is a figure which shows the example of the superimposed image which superimposed the image for superimposition on the original image. It is explanatory drawing which illustrates typically the maximum area search process shown in FIG. It is a figure which shows an example of the image table for superimposition in which the some image for superimposition with which the tag for matching is mutually equal is registered. It is a flowchart which shows the image processing which concerns on 2nd Embodiment. It is a flowchart which shows the image processing which concerns on 3rd Embodiment. It is the image processing which concerns on 3rd Embodiment, Comprising: It is a flowchart of the image processing in the case of using a priority. It is a flowchart of an attention area specification process in the case of specifying an attention area using a priority. (A) is a figure which shows an example of the priority table which shows the designation | designated result of a priority, (B) is an example of the area table which registered the position and size of each area | region after merging according to the priority. FIG. It is the figure which represented the position of the superimposition area | region candidate extracted in each of the non-attention area P02 and the non-attention area P03 from which a priority mutually differs in a table | surface form. (A) is an example of a non-attention area, (B) is a diagram in which the magnification is changed without changing the inclination of the image for superimposition within the range that falls within the non-attention area, and (C) is superimposed on the original image. It is the figure which changed the inclination and magnification of the image for superimposition within the range which fits in a non-attention area, and was superimposed on the original image. It is explanatory drawing explaining the method of the largest area search process in the case of enabling change of the inclination of the image for superimposition. (A) is an example of a non-attention area, (B) is a diagram in which the magnification is changed without changing the aspect ratio of the image for superimposition within the range that falls within the non-attention area, and superimposed on the original image, (C) FIG. 4 is a diagram in which an aspect ratio and a magnification of an image for superimposition are changed and superimposed on an original image within a range that falls within a non-attention area. It is a flowchart which shows an example of the largest area search process in case the change of an aspect ratio is possible. It is explanatory drawing which illustrates typically the maximum area search process shown in FIG. It is a figure which shows the specific example of the designation | designated screen which designates superimposition conditions. (A) is a figure which shows an example of the image table for superimposition, (B) is a figure which shows an example of the superimposition condition table which registered the superimposition conditions. It is the figure which plotted an example of the search result when changing inclination and searching a maximum area.

  Hereinafter, embodiments of the disclosed technology will be described in detail with reference to the drawings.

  [First Embodiment]

  FIG. 1 shows an image processing apparatus 10 according to the present embodiment. The image processing apparatus 10 performs a process of superimposing an additional image (for example, an image representing an advertisement or some information) on an image designated by the user. Hereinafter, the additional image is referred to as a superimposing image, the original image on which the superimposing image is superimposed is referred to as an original image, and an image in which the superimposing image is superimposed on the original image is referred to as a superimposed image. For example, the superimposed image is provided to the user using a predetermined means (set to a viewable state), but the process of providing the superimposed image to the user is performed with the image processing apparatus 10. May be performed by another apparatus, or may be performed by the image processing apparatus 10. In this embodiment, the description about the process which provides a superimposition image to a user is abbreviate | omitted.

  The image processing apparatus 10 includes an input unit 12, an image selection unit 14, a region specifying unit 16, a superimposition image selection unit 18, a superposition region determination unit 20, a superposition unit 22, an output unit 24, an image storage unit 26, and a recognition result storage unit. 28, a superimposition image storage unit 30, a superimposition image storage unit 32, and an image recognition unit 40. Here, the case where the image storage unit 26, the recognition result storage unit 28, the superimposing image storage unit 30, and the superimposed image storage unit 32 are provided in the image processing apparatus 10 will be described as an example. These storage units may be provided in a device different from the image processing device 10. In this embodiment, the case where the image recognition unit 40 is provided in the image processing apparatus 10 will be described as an example. However, the image recognition unit 40 is provided in a device different from the image processing apparatus 10. May be.

The image storage unit 26 stores a plurality of image data of original images. Further, in the image storage unit 26, also the original image table registering attribute information of each original image by the image data is stored in the images storage unit 26 has been stored. FIG. 2 shows an example of the original image table. In the original image table, information of original image ID, image path, selection tag, image size, and recognition result table ID is registered as attribute information of the original image.

  “Original image ID” is identification information given to each original image. “Image path” is an address of a storage area in which image data of each original image is stored. The “selection tag” is information given in advance as a keyword for selecting an original image from a plurality of original images registered in the original image table. As the selection tag, for example, information indicating the content of the image included in the original image may be registered. “Image size” is information indicating the size of the original image, and is expressed by “the number of pixels in the horizontal (= horizontal direction, x-axis direction) × the number of pixels in the vertical (= vertical direction, y-axis direction)”. Has been. The “recognition result table ID” is information for specifying a corresponding recognition result table. In the present embodiment, as will be described later, image recognition processing for recognizing an object included in the original image (referred to as a recognition target such as an object, a human face, or a character) is performed for each original image. The recognition result obtained by the processing is registered in the recognition result table. Information that identifies this recognition result table is a recognition result table ID. Accordingly, one recognition result table is associated with one original image, and the associated recognition result table ID is registered in the original image table.

The recognition result table is generated by the image recognition unit 40 and stored in the recognition result storage unit 28. FIG. 3 shows an example of the recognition result table. In the recognition result table, information indicating a recognition result, each information of a recognition result table ID, a category, recognition contents, a position (upper left), and an object size is registered.

  “Recognition result table ID” is identification information assigned to each recognition result table. On the other hand, “category”, “recognition content”, “position (upper left)”, and “object size” are provided for each object recognized from the original image by the image recognition unit 40.

  “Category” indicates the type of the recognition processing unit that recognized the object. In this embodiment, as will be described later, image recognition processing is performed by an image recognition unit 40 including a plurality of types of recognition processing units (see also FIG. 5), and the types of recognition targets recognized by the respective recognition processing units. Different. In this example, the recognition processing unit that has recognized the object is specified by the “category”. For example, for an object recognized by a recognition processing unit (object recognition unit 42) that recognizes an object, information indicating “object recognition” is registered as a category. For example, for an object recognized by the recognition processing unit (face recognition unit 44) that recognizes a face, information indicating “face recognition” is registered as a category. In addition, information indicating “text” as a category is registered as an object recognized by the recognition processing unit (character recognition unit 46) that recognizes characters. “Recognized content” is information indicating the content of the recognized object. For example, in the recognition result table shown in FIG. 3, three objects having the category “object recognition” are registered, and “house”, “car”, and “person” are recognized as the recognition contents of these three objects, respectively. It is registered.

  “Position (upper left)” is information indicating the position of the recognized object in the original image. In the present embodiment, the upper left coordinate value of the object when the lower left vertex of the original image is the origin (0, 0) of the xy coordinate system is registered as “position (upper left)”. The “object size” is information indicating the size of the object, and is represented by “the number of horizontal pixels × the number of vertical pixels”. In this example, the “object size” is described as the size of a rectangular area circumscribing the object. However, the present invention is not limited to this, and information representing the shape and size of each object is recognized. You may make it store in a result table.

  The superimposing image storage unit 30 stores a plurality of image data of superimposing images. The superimposing image storage unit 30 also stores a superimposing image table in which attribute information of each superimposing image whose image data is stored in the superimposing image storage unit 30 is registered. FIG. 4 shows an example of the image table for superimposition. In the superimposing image table, superimposing image ID, image path, matching tag, and image size information are registered as attribute information of the superimposing image.

  The “superimposition image ID” is identification information given to each superimposition image. “Image path” is an address of a storage area in which image data of an image for superimposition is stored. The “matching tag” is information given in advance as a keyword for selecting a superposition image from a plurality of superposition images stored. In the present embodiment, the “matching tag” is assigned so as to correspond to any of the keywords used as the “selection tag” registered in the original image table. The superimposition image to which the matching tag corresponding to the original image selection tag is assigned is selected by the superimposition image selection unit 18 described later as an image to be superimposed on the original image. The “image size” is information indicating the size of the image for superimposition and is represented by “the number of horizontal pixels × the number of vertical pixels”. In this embodiment, the case where the shape of the superimposition image is a rectangle will be described as an example, but the superimposition image may not be a rectangle.

  The superimposed image storage unit 32 stores the image data of the superimposed image.

  The input unit 12 causes the user to input a keyword for selecting an original image to be superimposed from the image storage unit 26 as image selection information. For example, the input unit 12 outputs an image or sound that prompts the user to input image selection information, and the user operates the operation unit such as a keyboard to input the image selection information according to the image or sound. . A user can designate an original image by inputting image selection information. In the present embodiment, the case where the image selection information is a keyword will be described. However, the image selection information may be information indicating the characteristics of the original image, or information indicating conditions for specifying the original image. .

  In addition, the input unit 12 causes the user to input a keyword for specifying an attention area noted by the user from the original image to be superimposed as information on the attention object (attention area identification information). The input unit 12 outputs, for example, an image or a sound that prompts the user to input attention area specifying information, and the user operates the operation part such as a keyboard according to the image or sound, for example. input. In the present embodiment, the case where the attention area specifying information is a keyword will be described. However, the attention area specifying information may be information indicating the characteristics of the attention area, or information indicating a condition for specifying the attention area. It may be. In some cases, the image selection information is “satellite photograph”, the attention area specifying information is “cloud”, and the like, but the image selection information and the attention area specifying information are both the same keyword (for example, “automobile”). Or the like. In the latter case, the input of the attention area specifying information may be omitted, and the image selection information may also be used as the attention area specifying information.

  The image selection unit 14 selects an original image to be superimposed from a plurality of original images stored in the image storage unit 26 based on the input image selection information.

  The area specifying unit 16 specifies an attention area to be noted by the user from the original image selected by the image selection unit 14 based on the input attention area specifying information (or image selection information).

  The superimposing image selection unit 18 selects a superimposing image to be superimposed on the original image from the superimposing images stored in the superimposing image storage unit 30. In the present embodiment, the superimposition image selection unit 18 selects a superposition image associated with a matching tag corresponding to the selection tag of the original image selected by the image selection unit 14.

  The superimposition region determination unit 20 determines a superimposition region that is a region in which the superimposition image selected by the superimposition image selection unit 18 is superimposed from a region excluding the attention region in the original image (hereinafter referred to as a non-attention region). . In the present embodiment, the magnification of the superimposition image when superimposing the superimposition image on the original image is variable, and the superimposition region determination unit 20 sets the superimposition image to the maximum size (maximum area) from the non-attention region. A region that can be superimposed with a magnification is searched, and the region is determined as a superimposed region.

  The superimposing unit 22 adds the superimposing image selected by the superimposing image selecting unit 18 to the superimposing region determined by the superimposing region determining unit 20 among the original images selected by the image selecting unit 14 within the superimposing region. Superposition is performed with the magnification changed so that the maximum size is obtained, and a superimposed image is generated.

  The output unit 24 outputs the superimposed image generated by the superimposing unit 22 to the superimposed image storage unit 32 and stores it in the superimposed image storage unit 32. The superimposed image stored in the superimposed image storage unit 32 is provided to the user.

  Here, the image recognition unit 40 will be described. FIG. 5 is a configuration diagram showing the configuration of the image recognition unit 40. The image recognition unit 40 includes a plurality of recognition processing units corresponding to the types of objects to be recognized. In the present embodiment, the image recognition unit 40 includes three recognition processing units, which are an object recognition unit 42, a face recognition unit 44, and a character recognition unit 46, as recognition processing units.

  The object recognition unit 42 recognizes an image of an object included in the original image, and outputs information indicating the type (content) of the object, the size of the object, and the position in the original image. The face recognition unit 44 recognizes a human face included in the original image and outputs information indicating the position and size of the recognized face. The character recognition unit 46 recognizes characters included in the original image and outputs information indicating the content, position, and size of the recognized characters.

  FIG. 6 is an explanatory diagram illustrating an example of image recognition processing. Each of the object recognition unit 42, the face recognition unit 44, and the character recognition unit 46 performs recognition processing on the original image shown in FIG. Accordingly, as shown in FIG. 6B, five objects O1 to O5 are recognized. Objects O1 to O3 are object objects recognized by the object recognition unit 42. The object O4 is a face object recognized by the face recognition unit 44. The object O5 is a character object recognized by the character recognition unit 46. And each information of each of these objects O1-O5 is registered into the recognition result table shown in FIG.6 (C).

  FIG. 7 is an explanatory diagram for explaining a specific example of the character recognition processing of the character recognition unit 46. Here, the characters included in the original image are imaged and not text information. The original image shown in FIG. 7A includes images representing a plurality of character strings, and each of the images representing the character strings is recognized as an object to be recognized by the character recognition unit 46. For example, when the character recognition unit 46 recognizes an image representing a character string of “Hachioji City” in the original image shown in FIG. 7A (hereinafter referred to as a character image) as a character object, the content “Hachioji City” of the character object is recognized. ”, Information indicating the position and size is output from the character recognition unit 46.

  Note that the character recognition unit 46 may extract, for example, a rectangular area including the entire character image as a character object area. In this case, for example, a rectangular area circumscribing the character image may be extracted as a character object area. Further, as shown in FIG. 7B, a rectangular area that includes a character image and is larger than the rectangular area circumscribing the character image is associated with the character image of “Hachioji City”, and the associated area is a character. You may extract as an area | region of an object. In FIG. 7B, not only in Hachioji City but also in other character images such as “Akiruno City” and “Okutama Town”, an area larger than the rectangular area circumscribing the character image is similarly displayed in each character image. It is tied. Therefore, as shown in FIG. 7B, some of the recognized object regions may overlap each other. In addition, the area | region tied in this way does not need to be a rectangle, For example, a circle may be sufficient.

  Note that not only the character recognition unit 46 but also the object object recognized by the object recognition unit 42, a rectangular area circumscribing the object image may be extracted as the object object area, or larger than the rectangular area. A rectangular area may be extracted as the area of the object object. Similarly, for the face object recognized by the face recognition unit 44, a rectangular area circumscribing the face image may be extracted as the face object area, or a rectangular area larger than the rectangular area may be extracted. (See also FIG. 6B). Hereinafter, a case where the shape of the area of each object recognized by the image recognition unit 40 is a rectangle will be described as an example.

  In the present embodiment, the case where the image recognition unit 40 includes three recognition processing units will be described as an example. However, the number and types of recognition processing units constituting the image recognition unit 40 are as follows. However, it is not limited to these. For example, a recognition processing unit corresponding to an object that is supposed to be included in the original image can be provided according to the type of the original image that is the target of image recognition.

The image processing apparatus 10 can be realized by a computer 50 shown in FIG. 8, for example. The computer 50 includes a CPU (Central Processing Unit) 52, a display 54, a keyboard 56, a communication IF (Interface) 58, a first memory unit 60 of the non-volatile memory 66 and second memory unit 6 8 nonvolatile, they Are connected to each other via a bus 70.

  The first storage unit 60 stores an image processing program 62 and an image recognition program group 64 for causing the computer 50 to function as the image processing apparatus 10. The CPU 30 reads the image processing program from the first storage unit 60 and develops it in the memory 66, and executes a process included in the image processing program.

  The image processing program 62 includes an input process, an image selection process, a region specifying process, a superimposing image selection process, a superimposing region determination unit process, a superimposing process, and an output process. The CPU 52 operates as the input unit 12 shown in FIG. 1 by executing the input process. The CPU 52 operates as the image selection unit 14 shown in FIG. 1 by executing the image selection process. The CPU 52 operates as the area specifying unit 16 illustrated in FIG. 1 by executing the area specifying process. The CPU 52 operates as the superimposing image selection unit 18 shown in FIG. 1 by executing the superimposing image selection process. Further, the CPU 52 operates as the overlapping area determination unit 20 illustrated in FIG. 1 by executing the overlapping area determination process. The CPU 52 operates as the superimposing unit 22 shown in FIG. 1 by executing the superimposing process. The CPU 52 operates as the output unit 24 shown in FIG. 1 by executing the output process.

  The image recognition program group 64 includes an object recognition program, a face recognition program, and a character recognition program. The CPU 52 operates as the object recognition unit 42 illustrated in FIG. 5 by executing the object recognition program. The CPU 52 operates as the face recognition unit 44 shown in FIG. 5 by executing the face program. The CPU 52 operates as the character recognition unit 46 shown in FIG. 5 by executing the character recognition program.

  The second storage unit 68 includes storage areas corresponding to the storage units of the image storage unit 26, the recognition result storage unit 28, the superimposition image storage unit 30, and the superimposed image storage unit 32.

  The communication IF 58 is an interface for communicating with other devices.

  Next, as an operation of the present embodiment, image processing performed by the image processing apparatus 10 will be described with reference to flowcharts of FIGS. 9 to 11 and explanatory diagrams of FIGS. FIG. 9 is a flowchart illustrating an example of image processing performed by the image processing apparatus 10. FIG. 10 is a flowchart illustrating an example of processing for specifying a region of interest (hereinafter referred to as region-of-interest specifying processing). FIG. 11 is a flowchart illustrating an example of processing for searching a non-attention area for an area in which an image for superimposition can be superimposed at a magnification of the maximum size (largest area) (hereinafter referred to as maximum area search processing). FIG. 12 is an explanatory diagram conceptually illustrating the flow of processing from the input of image selection information until the region of interest is identified. FIG. 13 is a diagram illustrating specific examples of the superimposing image, the non-attention area, and the overlapping area. FIG. 14 is an explanatory diagram schematically illustrating the maximum area search process shown in FIG.

  In the present embodiment, before the image processing described with reference to FIG. 9 is started, image recognition by the image recognition unit 40 is performed, and the recognition result for each original image stored in the image storage unit 26. Assume that the table is already stored in the recognition result storage unit 28.

  In step 100, for example, the input unit 12 displays an image that prompts the user to input image selection information on the display 54, and causes the user to input image selection information. The user inputs image selection information by operating an operation unit such as a keyboard 56, for example. FIG. 12A shows an example of the input screen displayed on the display 54. This input screen has an area 72 for inputting a keyword as image selection information and a search button 74. For example, when the user operates the keyboard 56 to input a keyword as image selection information in the area 72 and then clicks the search button 74 with a mouse or the like (not shown), the input of the image selection information is completed, and the original image Search (selection) of is started. Note that the input unit 12 may communicate with other devices connected to a network such as a communication line or a LAN, and acquire image selection information input from the other devices.

  In step 102, the image selection unit 14 selects an original image from a plurality of original images stored in the image storage unit 26 based on image selection information input by the user. For example, the image selection unit 14 refers to the image table and selects an original image having a selection tag corresponding to the image selection information input by the user.

  When a plurality of keywords are associated as the original image selection tags, the image selection unit 14 selects the original image when the image selection information corresponds to at least one of the plurality of keywords. May be selected. Further, when the user inputs a plurality of keywords as image selection information, the image selection unit 14 selects an original image provided with a selection tag corresponding to at least one of the plurality of keywords. May be. Further, the image selection unit 14 may select an original image to which selection tags corresponding to all of the plurality of keywords are assigned. Further, if the image selection information and the selection tag are similar, the image selection unit 14 determines that the selection tag corresponds to the image selection information even if they are not completely matched, and selects the original image. You may do it.

  Taking the original image table shown in FIG. 12B as an example, the original image having the selection tag corresponding to the image selection information “car” input by the user is the original image with the original image ID “A01”. Therefore, the image selection unit 14 selects the original image (FIG. 12C) with the original image ID “A01”. When there are a plurality of original images having selection tags corresponding to the image selection information input by the user, for example, a list of the corresponding original images is displayed on the display 54 and a desired original image is used. Can be selected.

  In step 104, for example, the input unit 12 displays an image that prompts the user to input image selection information on the display 54 and causes the user to input attention area specifying information. For example, the user operates the operation unit such as the keyboard 56 to input attention area specifying information. The input unit 12 may communicate with another device connected to a network such as a communication line or a LAN, and acquire attention area specifying information input from the other device. Further, as described above, when the image selection information is used as attention area specifying information, step 104 (input of attention area specifying information) is omitted. Therefore, in FIG. 9, step 104 is indicated by a broken line.

  In step 106, the region specifying unit 16 specifies a target region from the original image selected by the image selecting unit 14 based on the input target region specifying information (or image selection information).

  A specific example of the attention area specifying process will be described with reference to FIG. In step 200, the area specifying unit 16 acquires the image recognition result (that is, information registered in the recognition result table) of the original image selected by the image selection unit 14. In step 202, the area specifying unit 16 determines whether each object registered in the recognition result table is a target object. Here, the attention object refers to an object having recognition contents corresponding to the attention area specifying information (or image selection information). In the example described above, an object whose recognition content is “automobile” is extracted as an object of interest (see also FIG. 12D). Note that if the region-of-interest specifying information (or image selection information) and the recognized content are similar, the region specifying unit 16 recognizes that the recognized content is the region-of-interest specifying information (or image selection information) even if they are not completely matched. Information), and may be extracted as an object of interest.

  In step 204, the area specifying unit 16 determines whether there are a plurality of objects of interest. If there are a plurality of objects of interest, the area specifying unit 16 merges (combines) the areas of the object of interest in step 206. In step 208, the region specifying unit 16 specifies the merged region as a region of interest. An area excluding the identified attention area is a non-attention area.

  On the other hand, if there is one object of interest, the area specifying unit 16 makes a negative determination in step 204, and specifies the area of the object of interest as the area of interest in step 210 (see also FIG. 12E). An area excluding the identified attention area is a non-attention area.

  In step 108, the superimposing image selection unit 18 selects a superimposing image to be superimposed on the original image selected by the image selecting unit 14 from the superimposing images stored in the superimposing image storage unit 30. In the present embodiment, it is assumed that the superimposition image selection unit 18 selects a superposition image associated with a matching tag corresponding to the selection tag of the original image selected by the image selection unit 14. For example, when a plurality of keywords are associated as selection tags for the original image selected by the image selection unit 14, a matching tag corresponding to at least one of the plurality of keywords is associated. A superimposing image may be selected. Further, an image for superimposition in which a plurality of matching tags corresponding to each of a plurality of keywords stored as selection tags may be selected. Also, the original image ID and the superimposing image ID are stored in association with each other in advance, and the superimposing image ID assigned with the superimposing image ID associated with the original image ID of the original image selected by the image selection unit 14 is stored. An image may be selected.

  In step 110, the superimposition region determination unit 20 determines a superimposition region that is a region in which the superimposition image selected by the superimposition image selection unit 18 is superimposed from the non-target region. In the present embodiment, the superimposition region determination unit 20 searches a non-target region for a region where the superimposition image can be superimposed at a magnification that has the maximum size (maximum area), and determines the region as a superimposition region. In the present embodiment, the superimposing unit 22 changes the magnification of the superimposing image so that the user's attention is easily directed to the superimposing image, and expands it to the largest possible size of the non-attention area (the original superimposing image). The image is reduced depending on the size of the image and superimposed on the original image. Therefore, the superimposition area determination unit 20 searches the non-attention area for an area where the superimposition image can be superimposed with the largest size, and determines this area as the superimposition area. In the present embodiment, the superimposing unit 22 maintains the state of the original superimposing image stored in the superimposing image storage unit 30 for the aspect ratio (aspect ratio) and inclination of the superimposing image, Only the magnification is changed.

  Hereinafter, a specific example of the maximum area search process will be described in detail with reference to FIG. In the present embodiment, as shown in FIG. 13B, the original image is divided into a lattice shape, and a non-attention area based on the lattice area is extracted from the non-attention area extracted by the area specifying unit 16. The superimposition region determination unit 20 performs a maximum region search process on a non-attention region based on the lattice region. Note that the smaller the lattice area, the smaller the error from the non-attention area extracted by the area specifying unit 16.

  In step 300, the superimposition area determination unit 20 sets the width of the minimum rectangular area that satisfies the aspect ratio of the superimposition image in width_base, with the grid area when the non-target area is divided into a grid as a unit, and sets the minimum Set the height of the rectangular area to height_base.

  In step 302, 0 is set to max_size. Here, max_size is a variable in which a maximum area value with one grid area as one unit is set.

  In step 304, the overlapping area determination unit 20 selects one grid area from the non-target areas as a starting point. Here, an example will be described in which the starting point is the lower left vertex of a rectangular region that is a candidate for the overlapping region, but the present invention is not limited to this, and any of the four vertices of the rectangular region that is a candidate for the overlapping region It can be. Furthermore, the overlapping area determination unit 20 sets 0 to width and 0 to height.

  In step 306, the overlapping area determination unit 20 adds width_base to width and adds height_base to height.

  In step 308, the overlapping area determination unit 20 starts from the currently selected grid area in the non-target area, and sets the current width (width) to the horizontal length and the height (height) to the vertical direction. It is determined whether or not a rectangular area having a length can be secured.

  In step 310, if the overlapping area determination unit 20 determines that the rectangular area can be secured as a result of the determination in step 308, the process proceeds to step 312.

  In step 312, the overlapping area determination unit 20 sets a value of height × width to the variable size.

  In step 314, the overlapping area determination unit 20 compares the variable max_size with the variable size, and determines whether or not the variable max_size is smaller than the variable size. If the determination result at step 314 is affirmative, the superimposition area determination unit 20 sets the value of the variable size to the variable max_size at step 316, stores information on the starting point at this time, and returns to step 306 for processing. repeat. Note that the starting point information stored in step 316 is held until the variable max_size is next updated. On the other hand, if the determination result at step 314 is negative, the overlapping area determination unit 20 returns to step 306 without performing the process at step 316.

  On the other hand, if it is determined in step 310 that the rectangular area cannot be secured as a result of the determination in step 308, the overlapping area determination unit 20 proceeds to step 318.

  In step 318, the overlapping area determination unit 20 determines whether or not the above process has been repeated starting from all the lattice areas of the non-target area. If the overlap region determination unit 20 makes a negative determination in step 318, in step 320, it changes the starting lattice region, sets width to 0, sets height to 0, and returns to step 306.

  If the determination is affirmative in step 318, the overlapping area determination unit 20 determines in step 322 that the rectangular area having the area of max_size at this time is the rectangular area having the maximum area. Then, the overlapping area determination unit 20 specifies the rectangular area having the maximum area as the overlapping area.

  The process example shown in FIG. 11 is schematically shown in FIG. In FIG. 14, the non-attention area is shown in gray. Here, as shown in FIG. 13A, a case where a square superposition image having an aspect ratio of “vertical (height): horizontal (width) = 1: 1” is superimposed on the original image is taken as an example. I will give you a description. In this example, first, as shown in FIG. 14A, a starting point is selected. In FIG. 14A, the starting point is a grid area filled with black. Since the image for superimposition is a square, in step 300, 1 is set to width_base and 1 is set to height_base.

  Then, as shown in FIG. 14B, first, the area 1 of the rectangular area having the width 1 and the height 1 is set to the variable max_size. Next, when 1 is added to width and height and the width and height are set to 2, a 2 × 2 square rectangular area can be secured as shown in FIG. Black area), rectangular area 4 is set to the variable max_size. Next, when 1 is added to width and height and the width and height are 3, the rectangular area 9 is set to the variable max_size as shown in FIG.

Next, 1 is added in the width and height, and if it is the width and height 4, as shown in FIG. 14 (E), in the non-target region, a rectangular region having a width and height 4 Can not be secured. Therefore, the variable max_size is not updated. Next, when 1 is added to width and height and the width and height are set to 5, as shown in FIG. 14 ( F ), a rectangular area having a width and height of 5 within the non-attention area. Can not be secured. If it is not possible to secure a rectangular area with a width and height of 4, it is obvious that a rectangular area with a width and height of 5 cannot be secured. Therefore, in the flowchart shown in FIG. 11, the width and height are updated. Without starting, the starting point is shifted to the next lattice area.

  Accordingly, when the lattice area shown in FIG. 14A is used as the starting point, the area having the largest area within the range satisfying the aspect ratio of the superimposed image is the area 9 when the width is 3 and the height is 3. Therefore, the overlapping area determination unit 20 increases the width in a state where 9 is set in the variable max_size, and shifts the starting point. Even if the shape of the non-attention area is distorted, the overlapping area determination unit 20 searches all the grid areas as starting points while changing the positions of the starting points.

  As a result of such processing, as shown in FIG. 13A, when the superimposition image has a square shape, and the gray area shown in FIG. 13B is a non-attention area based on the lattice area. Is determined by extracting a black area shown in FIG. 13C as an overlap area. In the determination of the overlapping area shown in FIG. 11, for example, some limitation may be provided, such as setting the maximum size of the overlapping area to be equal to or smaller than the size of the attention area.

  In the above description, the superimposing image has a square shape. However, the same processing may be performed when the superimposing image has a rectangular shape with a different width and height. In this case, in the processing illustrated in FIG. 11, when the aspect ratio of the image for superimposition is “width: height = 4: 3”, in step 300, 4 is set to width_base, and height_base Is set to 3. As a result, for example, when a superimposition region is searched from within the non-attention region shown in FIG. 15, three rectangles are searched: the smallest rectangle that satisfies 4: 3, the second largest rectangle, and the third largest rectangle. Among these, the maximum area search process of FIG. 11 ends with the largest third rectangular area 108 set to the variable max_size. Accordingly, the third rectangular area is determined as the overlapping area.

  In addition, here, an example has been described in which the superimposition area search process is performed in units of lattice areas when the non-target area is divided into a grid, but the superimposition area search process may be performed in units of pixels.

  In FIG. 9, after the overlapping region is determined in step 110, in step 112, the overlapping unit 22 adds the overlapping region determined by the overlapping region determining unit 20 among the original images selected by the image selecting unit 14. The superimposed image selected by the superimposed image selection unit 18 is superimposed to generate a superimposed image. At this time, the superimposing unit 22 superimposes the superimposed image on the superimposed region by changing the magnification of the superimposed image so that the superimposed image has the determined size of the superimposed region. Note that when the size of the original superimposition image is smaller than the size of the superimposition area, the superimposition unit 22 magnifies and superimposes the images with the same vertical and horizontal enlargement ratios. In addition, the superimposing unit 22 does not need to enlarge when the size of the original superimposing image is larger than the determined superimposing area size. May be. Furthermore, when the size of the superimposition area is larger than the size of the attention area, the superimposition area may be superimposed with the magnification changed so that the size of the image for superimposition becomes smaller than the size of the attention area. FIG. 16A shows an example of an original image to be superimposed, and FIG. 16B shows an example of a superimposed image obtained by superimposing a superimposition image on the original image of FIG. The region of the car object shown in FIG. In FIG. 16B, the superimposing image is superimposed on the non-attention area excluding the attention area. In this example, the image for superimposition is superimposed with a size smaller than the size of the region of interest.

  In step 114, the output unit 24 outputs the superimposed image generated by the superimposing unit 22 to the superimposed image storage unit 32 and stores it in the superimposed image storage unit 32.

  As described above, in the present embodiment, the original image is selected according to the designation of the user, the attention area noticed by the user is identified according to the designation of the user, and the attention area is excluded. The superimposition image is superimposed on the superimposition area determined from the attention area. Therefore, even when the position and size of the attention area noted by the user varies for each image used by the user, the image for superimposition is added to the area in the image that does not impair the convenience of the user. Can be superimposed.

  In the present embodiment, an example in which the image recognition processing by the image recognition unit 40 is executed before the image processing is started has been described. However, the present invention is not limited to this, and processing for specifying a region of interest in step 106 The image recognition process may be executed before the operation. Therefore, for example, after step 102 and before step 106, image recognition may be performed on the original image selected in step 102, an object may be extracted, and a recognition result table may be created.

  Further, in the present embodiment, the example in which the region having the maximum area is searched from the non-target region and determined as the overlapping region has been described. However, any region of the non-target region may be selected even if it is not the region having the maximum area. You may make it determine an area | region as a superimposition area | region. For example, an area that satisfies a predetermined condition of the non-attention area, for example, an area closest to the attention area may be determined as the overlapping area. Even in this case, even when the position and size of the attention area noted by the user varies for each image used by the user, the image is superimposed on an area that does not impair the convenience of the user in the image. An image for use can be superimposed.

  Furthermore, in the above description, the example in which the region specifying unit 16 specifies the region of interest based on the image selection information or the region of interest specifying information has been described. However, the present invention is not limited to this. For example, the area specifying unit 16 may specify a non-attention area that is not noticed by the user based on the image selection information or the attention area specification information. For example, the region specifying unit 16 may extract a region of an object excluding the target object and a region not recognized by the image recognition unit 40, and extract a region obtained by merging these regions as a non-target region. The overlapping area determination unit 20 determines the overlapping area from the non-attention area specified by the area specifying unit 16 as described above.

  In the above description, the example of the non-attention area excluding the attention area from the original image has been described. However, for example, an area where no object exists may be specified as the non-attention area.

  [Second Embodiment]

  Next, a second embodiment of the disclosed technique will be described. In the second embodiment, an example will be described in which, when a plurality of images for superimposition are superimposed on the original image, a superimposition region is determined for each image for superimposition and superimposed on the original image. Since the second embodiment is the same as the first embodiment described above except for image processing, the same reference numerals are given to the respective parts, and the description of the configuration is omitted. Hereinafter, the second embodiment will be described. Image processing according to the above will be described.

  FIG. 17 shows an example of the superimposing image table. In the superposition image table shown in FIG. 17, a plurality (two in this example) of superposition images having the same matching tag are registered. Therefore, when the method described in the first embodiment is used when selecting an image for superimposition, these two images for superimposition are used as superimposition images to be superimposed on the original image to which the selection tag of automobile is added. Will be selected. In the superimposing image table of this embodiment, the priority is also registered in correspondence with the superimposing image. The priority can be specified in advance by the provider of the image for superimposition, for example.

  FIG. 18 is a flowchart illustrating image processing according to the second embodiment.

  The processing from step 400 to step 408 is the same as the processing from step 100 to step 108 described with reference to FIG. However, in the present embodiment, a plurality of superimposing images may be selected in step 408.

  In step 410, the superimposition area determination unit 20 determines whether or not a plurality of superimposition images have been selected in step 408. If the determination in step 410 is affirmative, the superimposition region determination unit 20 determines a superimposition region in the non-attention region in order from the superimposition image having a higher priority in step 412.

  Also in the present embodiment, the superimposition area determination unit 20 searches the non-attention area for an area where the superimposition image can be superimposed at the maximum magnification, and determines the area as the superimposition area. Since the determination method of a superimposition area | region should just be performed similarly to the method demonstrated in 1st Embodiment, description is abbreviate | omitted here. However, among the plurality of superimposition images selected in step 408, the other superimposition images excluding the superimposition image having the highest priority do not overlap with the previously determined superimposition area in the non-attention area. Is determined as the overlapping region. That is, when determining a superimposition region, a superimposition region is determined from a region obtained by excluding a superimposition region previously determined for a superimposition image having a higher priority than a superimposition image for determining the superimposition region from a non-attention region. decide. As a result, a superimposition region having a larger area is assigned to a superimposition image with higher priority.

  In step 414, the superimposing unit 22 superimposes each superimposing image on each superimposing region determined for each superimposing image to generate a superimposed image.

  On the other hand, if a negative determination is made in step 410, the processing in step 416 and step 418 is performed. In addition, since the process of step 416 and step 418 is the same as the process of step 110 and step 112 demonstrated using FIG. 9 in 1st Embodiment, description is abbreviate | omitted. Furthermore, the process of step 420 performed after step 414 and step 418 is also the same as the process of step 114 described with reference to FIG. 9 in the first embodiment, and thus description thereof is omitted.

  As described above, even when a plurality of superimposing images are superimposed on each of the plurality of superimposing regions, each superimposing image can be superimposed on a region in the image that does not impair the convenience of the user.

  When the aspect ratio of the superimposing image is fixed and the aspect ratio of N superimposing images to be superimposed on the original image (N is an integer of 2 or more) is equal, a range that satisfies the aspect ratio first. The top N overlapping regions may be searched in descending order of area, and information on the overlapping regions may be retained. Then, each superimposition region may be determined so that a superimposition image with a higher priority is superimposed on a larger superimposition region.

  [Third Embodiment]

  Next, a third embodiment of the disclosed technique will be described. In the third embodiment, when one superimposition image is superimposed on the original image, a plurality of superposition region candidates are extracted from the non-target region, and one superposition region is determined from the plurality of superposition region candidates. Will be described. Since the third embodiment is the same as the first embodiment described above except for image processing, the same reference numerals are given to the respective parts, and the description of the configuration is omitted. Hereinafter, the third embodiment will be described. The image processing according to will be described with reference to FIG. FIG. 19 is a flowchart illustrating image processing according to the third embodiment.

  In this embodiment as well, image recognition by the image recognition unit 40 is performed before the image processing described with reference to FIG. 19 is started, and recognition of each original image stored in the image storage unit 26 is performed. It is assumed that the result table is stored in the recognition result storage unit 28.

  The processing from step 500 to step 508 is the same as the processing from step 100 to step 108 described with reference to FIG.

  In step 510, the superimposition region determination unit 20 searches for a region in which the superimposition image can be superimposed from the non-target region at a magnification that has the maximum size (largest area), and extracts the region as a superimposition region candidate. The search for the overlap region candidate may be performed in the same manner as the method described with reference to FIG. 11 in the first embodiment, for example. Here, the overlap region determination unit 20 compares the variable max_size with the variable size before returning to the processing of step 306 after the negative determination in step 314, and adds information on the starting point at this time if they are equal. Are stored until the variable max_size is updated next time (not shown). That is, in the present embodiment, when a plurality of rectangular areas having the same area as the variable max_size are searched, information on each of the rectangular areas is held as an overlap area candidate.

  In step 512, the overlapping area determination unit 20 determines whether or not a plurality of overlapping area candidates have been extracted. If the superimposition area determination unit 20 makes an affirmative determination in step 512, in step 514, the input unit 12 displays an image prompting the user to input position information for specifying the position of the superimposition area on the display 54, for example. Display and allow the user to input the location information. The user inputs position information by operating an operation unit such as the keyboard 56, for example.

  In step 516, the overlapping area determining unit 20 determines an overlapping area from among a plurality of overlapping area candidates based on the input position information.

  On the other hand, when a negative determination is made in step 512, the superimposition region determination unit 20 determines the superimposition region candidate as a superimposition region in step 518 because there is one superimposition region candidate.

  Since the processing of step 520 and step 522 performed after the overlapping region is determined is the same as step 112 and step 114 described with reference to FIG. 9 in the first embodiment, description thereof is omitted here.

  Thus, in this embodiment, since a superimposition area | region can be determined according to a user's designation | designated from several superimposition area | region candidates according to a user's designation | designated, the convenience of the user in this image is improved. The image for superimposition can be superimposed on an area that is not impaired.

  In the above description, the example in which the user designates the position each time and determines the overlapping area based on the position has been described. However, the present invention is not limited to this. For example, the overlapping area determining unit 20 is determined in advance. You may make it determine a superimposition area | region from several superimposition area | region candidates based on conditions. For example, when a condition such as determining the overlap region candidate closest to the attention region as the overlap region is set in advance, the overlap region candidate closest to the attention region is determined as the overlap region according to the condition. Also good. In addition, if the condition for determining the overlap region candidate at the position with the minimum distance from the center of the original image as the overlap region is set in advance, the overlap region candidate near the center of the original image is determined as the overlap region. You may make it do. Further, the conditions for determining the superimposition region may be set in advance by the user, or may be set by the provider who provided the superimposition image.

  Furthermore, the example in which a plurality of overlapping region candidates are extracted is not limited to the example described above. For example, even when the user sets the priority for each object and specifies the attention area and the non-attention area according to the priority, a plurality of overlapping area candidates may be extracted. Hereinafter, an example of image processing performed when priority is used will be described. FIG. 20 is a flowchart of image processing when priority is used. FIG. 21 is a flowchart of the attention area specifying process when the attention area is specified using the priority.

  The processing from step 600 to step 602 in FIG. 20 is the same as the processing from step 100 to step 102 described with reference to FIG.

  In step 604, the input unit 12 displays, for example, an image that prompts the user to input the priority for each object in the recognition result table corresponding to the selected original image, and displays the image to the user. Lets you enter the priority of the object. For example, the user operates the operation unit such as the keyboard 56 to input the priority. Further, as the attention area specifying information, superposition permission information indicating whether or not to superimpose the superposition image is also input for each priority.

  FIG. 22A is a diagram illustrating an example of a priority table indicating priority designation results. The priority table is stored in the second storage unit 68, for example. Here, the priority is specified in three stages from 0 to 2. 0 is the highest priority and 2 is the lowest priority. The priority table illustrated in FIG. 22A includes a recognition result whose content is “no recognition”, and this indicates a region that is not recognized as an object by the object recognition unit 42. In this example, an area where an object is not actually recognized is also registered in the priority table as a virtual object. The priority table is generated for each original image and for each user. Although not shown in the figure, superimposition of the superimposing image is prohibited in the object area of priority 0 by the designation of the user, and the superimposing image is superimposed on the object area of priority 1 or 2. Is permitted (superimposition permission information). The priority and superimposition permission information of each object differ depending on the user.

  In step 608, the area specifying unit 16 specifies the attention area and the non-attention area according to the priority. Here, the processing of the region specifying unit 16 will be described in detail with reference to FIG. In step 700, the region specifying unit 16 acquires information of a recognition result table associated with the image recognition result of the original image selected in step 602. In step 702, the area specifying unit 16 acquires the priority specified for each object from the priority table. In step 704, the area specifying unit 16 merges areas (objects) having the same priority. FIG. 22B shows an example of an area table in which the position and size of each area after merging are registered. The merged area is not limited to a rectangle, and may have a complicated shape other than a rectangle. In step 706, the area specifying unit 16 specifies the area having the highest priority among the merged areas as the attention area, and specifies each of the other areas as the non-attention area. In the example shown in FIG. 22B, each of the area having the priority 1 and the area having the priority 2 is a non-attention area. Here, an example in which the region with the highest priority is specified as the attention region will be described. However, depending on the designation of the user, for example, the region with the highest priority and the region with the second highest priority may be The region of interest may be specified.

  Subsequently, in Step 610 of FIG. 20, the superimposition region determination unit 20 sets, as a processing target, a priority non-attention region in which superimposition of the superimposition image is permitted among all non-attention regions. In step 612, the overlapping area determination unit 20 determines whether there are a plurality of non-attention areas set as processing targets. In the above example, there are two non-attention areas that are allowed to be superposed, ie, a non-attention area with priority 1 and a non-attention area with priority 2, and therefore an affirmative determination is made at step 612.

  If the determination in step 612 is affirmative, the superimposing area determination unit 20 extracts, in step 616, an area where the superimposing image can be superimposed with the maximum size as a superimposing area candidate for each non-attention area set as a processing target. . The processing in step 616 may be performed for each non-attention area in the same manner as in step 110 described with reference to FIG. 9 of the first embodiment, and thus description thereof is omitted here.

  In step 618, for example, the input unit 12 displays on the display 54 an image that prompts the user to input position information for designating any position of the plurality of overlapping area candidates, and displays the position to the user. Ask for information. For example, for the user, as shown in FIG. 23, the positions of the overlap region candidates extracted in each of the non-target region P02 and the non-target region P03 may be displayed in a table format, or the original image An image in which a frame image is superimposed on the position of each of the overlapping region candidates may be displayed. In response to this, the user operates the operation unit such as the keyboard 56 to input position information for designating the position of any one of the plurality of overlapping area candidates.

  In step 620, the overlapping area determination unit 20 determines an overlapping area from among a plurality of overlapping area candidates based on the input position information.

  On the other hand, if a negative determination is made in step 612, it indicates that there is one non-attention area set as a processing target. Accordingly, in step 614, the superimposition area determination unit 20 extracts, as superimposition area candidates, an area where the superimposition image can be superimposed with the maximum size for the non-attention area set as the processing target. The processing in step 616 may be performed in the same manner as step 110 described with reference to FIG. 9 of the first embodiment, and thus description thereof is omitted here.

  Since the processing of Step 622 and Step 624 performed after Step 614 or Step 620 (after the overlapping region is determined) is the same as Step 112 and Step 114 described with reference to FIG. 9 in the first embodiment, The description is omitted here.

  Note that also in step 614 and step 616, as described with reference to FIG. 19, a plurality of overlapping region candidates having the same area may be searched for each non-target region. In this case, as described with reference to FIG. 19, the overlapping area may be determined from a plurality of overlapping area candidates.

  In the present embodiment, the example in which the image recognition processing by the image recognition unit 40 is executed before the image processing is started has been described. However, the image recognition processing is performed in Step 506 in FIG. 19 or Step 604 in FIG. It is only necessary to be executed before the above process, and the present invention is not limited to the above example.

  Further, the processing content described in the third embodiment may be applied to the processing content described in the second embodiment. That is, it is also possible to determine a superimposition region so that a superimposition region with a higher priority is assigned to a larger superimposition region among a plurality of superimposition region candidates extracted from a plurality of non-attention regions having different priorities. .

  [Fourth Embodiment]

  Next, a fourth embodiment of the disclosed technique will be described. In the first to third embodiments, the example in which only the magnification can be changed without changing the inclination and the aspect ratio of the image for superimposition has been described, but in this embodiment, not only the magnification but also the inclination and the aspect ratio are changed. An example in which at least one of the aspect ratios can be changed will be described. In the fourth embodiment, the configuration of the image processing apparatus 10 is the same as that of the first embodiment described above. Therefore, the same reference numerals are given to the respective parts, and the description of the configuration of the image processing apparatus 10 is omitted. To do.

  First, a case where the magnification and inclination of the superimposition image can be changed will be described. For example, it is assumed that the non-attention area having the shape and size shown in FIG. When the image for superimposition is an image having the shape shown in FIG. 24B, the magnification of the image for superimposition is within a range in which the area is within the non-attention area without changing the inclination or aspect ratio of the image for superimposition. Even if it changes so that it may become the maximum, the area of the image for superimposition after a change does not become so large. However, as shown in FIG. 24C, the inclination of the image for superimposition is changed in accordance with the shape of the non-attention area, and the magnification of the image for superimposition is maximized within the range where the area falls within the non-attention area. If changed to, the area can be increased as compared with the case where the inclination is not changed.

  In this way, when changing not only the magnification of the image for superimposition but also the inclination, the superimposing area determination unit 20 rotates the grid rotated (or the inclination changed) with respect to the non-attention area as shown in FIG. The maximum area search process described with reference to FIG. 11 is performed starting from each lattice area when overlapped. In addition, the overlapping area determination unit 20 performs the inclination (rotation angle) in a step, for example, -15 °, -10 °, -5 °, 0 °, + 5 °, + 10 °, + 15 °. The maximum area search process is performed for each rotation angle. The search accuracy increases as the width of the change is reduced.

  The overlapping area determining unit 20 determines the area having the maximum area among the areas searched in this way as the overlapping area. The superimposing unit 22 superimposes the superimposing image on the superimposing region to generate a superimposing image. At this time, the superimposing image is rotated to the rotation angle when the superimposing region is searched. Superposition is performed by changing the magnification of the superimposition image so that the size of the determined superimposition area is obtained.

  Next, a case where the magnification and aspect ratio of the superimposing image can be changed will be described. For example, it is assumed that the non-attention area having the shape and size shown in FIG. When the image for superimposition is an image having the shape shown in FIG. 26B, the magnification of the image for superimposition is within a range in which the area falls within the non-attention area without changing the inclination or aspect ratio of the image for superimposition. Even if it changes so that it may become the maximum, the area of the image for superimposition after a change does not become so large. However, as shown in FIG. 26C, the aspect ratio of the image for superimposition is changed in accordance with the shape of the non-attention area, and the magnification of the image for superimposition is maximized within the range where the area falls within the non-attention area. If changed in this way, the area can be increased compared with the case where the aspect ratio is not changed.

  Thus, when the aspect ratio can be changed, it is not necessary to consider the aspect ratio in the maximum area search process. Therefore, for example, instead of the maximum area search process illustrated in FIG. 11, the overlapping area determination unit 20 may perform the maximum area search process illustrated in FIG. 27. In this embodiment as well, as shown in FIG. 13B, the original image is divided into a lattice shape with a predetermined size, and the lattice region is extracted from the non-attention region extracted by the region specifying unit 16. A non-attention area as a reference is extracted. The superimposition region determination unit 20 performs a maximum region search process on a non-attention region based on the lattice region.

  In step 800, the overlapping area determination unit 20 sets 1 to the variable max_size. Here, the smallest area 1 expressed in units of each grid area when the original image is divided into grids of a predetermined size is set as the initial value of the variable max_size.

  In step 802, the overlapping area determination unit 20 selects one grid area from the non-target areas as a starting point. Furthermore, the overlapping area determination unit 20 sets 1 to height which is a vertical variable, and sets 1 to width which is a horizontal variable (see also FIG. 28A).

  In step 804, the overlapping area determination unit 20 starts from the currently selected lattice area among the non-target areas, and sets the current width (width) as the horizontal length of the rectangular area. Search how much the length in the height direction can be extended. Then, the overlapping area determination unit 20 sets the maximum height that can be extended as a result of the search to height.

  In step 806, the overlapping area determination unit 20 sets a value of height × width to the variable size.

  In step 808, the overlapping area determination unit 20 compares the variable max_size with the variable size, and determines whether the variable max_size is smaller than the variable size. If the determination is affirmative at step 808, the superimposition area determination unit 20 sets the value of the variable size to the variable max_size at step 810, stores the starting point information at this time, and proceeds to step 812. The information on the starting point stored here is retained until it is updated next to the variable max_size. On the other hand, if the overlap region determination unit 20 makes a negative determination in step 808, the process proceeds to step 812 without performing the processing in step 810.

  In step 812, the overlapping area determination unit 20 determines whether or not the width can be increased in the non-attention area. If the determination is affirmative at step 812, the overlapping area determination unit 20 proceeds to step 814. In step 814, the overlapping area determination unit 20 adds 1 to the variable width, returns to step 804, and repeats the above processing.

  If the determination result in step 812 is negative, the overlapping region determination unit 20 determines in step 816 whether or not the above process has been repeated starting from all lattice regions in the non-target region. If the overlap region determination unit 20 makes a negative determination in step 816, in step 818, it changes the starting lattice region, sets width to 1, sets height to 1, and returns to step 804.

  If the determination in step 816 is affirmative, the overlapping area determination unit 20 determines in step 820 that the rectangular area having the area of max_size at this time is the rectangular area having the maximum area. Then, the overlapping area determination unit 20 specifies the rectangular area having the maximum area as the overlapping area.

  The process example shown in FIG. 27 is schematically shown in FIG. In FIG. 28, the non-attention area is shown in gray. In this example, first, as shown in FIG. 28A, a starting point is selected. In FIG. 28A, the starting point is a lattice area filled with black. Here, 1 is set as the initial value of the variable max_size.

  Then, as shown in FIG. 28B, the maximum area 4 when the width is 1 is set to the variable max_size. Next, when 1 is added to width and the width is 2, as shown in FIG. 28C, a rectangular area of 2 × 3 can be secured at maximum (black area). The rectangular area 6 when the width is 2 is set to the variable max_size. Next, when 1 is added to width and the width is 3, as shown in FIG. 28D, the maximum rectangular area 9 when the width is 3 is set to the variable max_size. Next, when 1 is added to width and the width is 4, as shown in FIG. 28E, the maximum rectangular area at the time of width 4 is 8, but at the time of width 3 The variable max_size is not changed because it is smaller than the maximum rectangular area 9. Next, when 1 is added to width and the width is set to 5, as shown in FIG. 28 (F), the maximum rectangular area when width is 5 is 5, but when width is 3, The variable max_size is not changed because it is smaller than the maximum rectangular area 9.

  Therefore, when the lattice area shown in FIG. 28A is used as the starting point, the largest area is the area 9 when the width is 3 and the height is 3. Therefore, the overlapping area determining unit 20 sets the variable max_size. With 9 set to, increase the width and shift the starting point. Even if the shape of the non-attention area is distorted, the overlapping area determination unit 20 searches all the grid areas as starting points while changing the positions of the starting points.

  As described above, in this example, since there is no restriction on the aspect ratio, the largest rectangular area that can be secured among non-target areas is searched. Here, an example has been described in which the superimposition area search process is performed in units of lattice areas when the non-target area is divided into a grid, but the superimposition area search process may be performed in units of pixels.

  The overlapping region determining unit 20 determines the region having the maximum area searched in this way as the overlapping region. In the determination of the overlapping area shown in FIG. 27, for example, some limitation may be provided, such as setting the maximum size of the overlapping area to be equal to or smaller than the size of the attention area. The superimposing unit 22 generates a superimposed image by superimposing the superimposing image on the superimposing region. At this time, the superimposing image has an aspect ratio and a magnification so that the superimposing image becomes the size of the determined superimposing region. Change and superimpose. In addition, when the size of the superimposition area is larger than the size of the attention area, the superimposition area may be superimposed with the magnification changed so that the size of the image for superimposition becomes smaller than the size of the attention area.

  Note that the image processing apparatus 10 can be configured to be able to change all of the inclination, the aspect ratio, and the magnification. In this case, as described with reference to FIG. 25, the overlapping region determination unit 20 changes the lattice inclination (rotation angle) stepwise with respect to the non-attention region and uses FIG. 28 for each inclination. The described maximum area search process is performed, and an area having the maximum size over the entire inclination is determined as the overlapping area.

  In this embodiment as well, as described in the second embodiment, when a plurality of superimposing images are superimposed on the original image, the superimposing regions may be determined in order from the superimposing images having the highest priority. . Further, when the number of superimposing images to be superimposed on the original image is N (N is an integer of 2 or more), the top N overlapping regions are searched first in descending order of area, and the position of the overlapping region is determined. And size information may be held. Then, each superimposition region may be determined so that a superimposition image with a higher priority is superimposed on a larger superimposition region. Further, as described in the third embodiment, the process of the present embodiment in which the slope and the aspect ratio are variable even when the overlapping region candidate is extracted from the non-attention regions having different priorities and the overlapping region is determined. The method can be applied to determine the overlap region.

  When providing a service such as superimposing a superimposition image on the original image and charging a fee for superimposition of the superimposition image from the system management side to the superimposition image provider side (for example, the advertiser side) In addition, the image processing apparatus 10 may be applied. In addition, in this service, when the inclination, aspect ratio, and size of the superimposition image can be changed, the fee that the provider bears may be varied depending on the inclination, aspect ratio, and size (area). Further, when the inclination, aspect ratio, and size of the image for superimposition can be changed, the provider of the image for superimposition allows the at least one of the inclination, aspect ratio, and size of the image for superimposition as the superimposing condition. A range may be specified. For example, the input unit 12 of the image processing apparatus 10 displays a designation screen on the display 54 and the provider operates the operation unit such as the keyboard 56 so that the designation information of the allowable range is input. May be. Alternatively, the input unit 12 displays a designation screen on a terminal device different from the image processing apparatus 10 via the communication IF 58, and the designation that the provider of the superimposing image inputs to the designation screen by operating the terminal device Information may be acquired via the communication IF 58.

  FIG. 29A shows an example of a designation screen 80 for designating an allowable range of inclination and aspect ratio. The designation screen 80 includes a designation area 82 for designating an allowable range of the inclination of the superimposition image and a designation area 84 for designating an allowable range of the aspect ratio of the superposition image.

  In the designation area 82, four conditions of “not tiltable”, “± 10 degrees”, “± 30 degrees”, and “allow any tilt” can be specified. For example, when the provider wants to prohibit the superimposition image from being tilted and superimposing it on the original image with the rotation angle of the superimposition image always being 0 degrees, it is possible to designate “not tiltable”. In addition, when the provider permits the change of the inclination of the image for superimposition within the range of the rotation angle from −10 degrees to +10 degrees, “± 10 degrees” can be designated. The same applies to ± 30 degrees. Further, if the provider does not want to set a limit on the tilt, “allow any tilt” can be designated.

  In the designation area 84, three conditions of "fixed aspect ratio", "aspect ratio ± 5% deformation allowable", and "aspect ratio ± 10% deformation allowable" can be specified. For example, if the provider of the superimposing image wants to fix the aspect ratio as it is in the original superimposing image, “fixed aspect ratio” can be designated. Further, when the provider allows the deformation of the aspect ratio from −5% to + 5% of the aspect ratio of the original superimposition image, “aspect ratio ± 5% deformation allowable” can be designated. Here, “aspect ratio−5%” means that the superimposition image is deformed so that “horizontal length / vertical length” becomes 0.95. “Aspect ratio + 5%” means that the image for superimposition is deformed so that “horizontal length / vertical length” is 1.05. Further, when the provider allows the deformation of the aspect ratio from −10% to + 10% of the aspect ratio of the original superimposition image, “aspect ratio ± 10% deformation allowable” can be designated.

  FIG. 29B shows a designation screen 86 for designating not only the allowable range of the inclination and aspect ratio of the image for superimposition but also the allowable size. The designation screen 86 is provided with a designation area 88 for designating the permissible size of the image for superimposition as well as the designation areas 82 and 84 described above.

  In the designation area 82, three sizes of small, medium and large can be designated. In general, the provider of the image for superimposition desires to make the image for superimposition as large as possible. However, for example, if the fee varies depending on the size, The provider may specify the size of the superimposing image.

  Note that a value representing an actual area or a magnification may be associated with abstract items such as small, medium, and large in advance. For example, a range of area r1 or more and less than r2 is associated with “small”, a range of area r2 or more and less than r3 is associated with “medium”, and an area r3 or more and less than r4 is associated with “large”. A range may be associated. The unit representing the area may be the number of lattice regions or the number of pixels. Furthermore, a range of magnification s1 to less than s2 is associated with “small”, a range of magnification s2 to less than s3 is associated with “medium”, and a magnification s3 to less than s4 is associated with “large”. Ranges may be associated with each other.

  The designation screens 80 and 88 and the designation areas 82, 84, and 86 are examples, and the method for designating the superimposition condition is not limited to the above.

  In this way, the superimposing conditions designated by the superimposing image provider can be registered in the superimposing image table. As shown in FIG. 30A, a superposition condition ID for specifying a superposition condition is registered in association with the superposition image ID, and as shown in FIG. 30B, the superposition condition ID is stored in the superposition condition table. The superimposing conditions may be registered in association with each other.

  The superimposing region determination unit 20 refers to the superimposing condition registered in association with the superimposing image selected by the superimposing image selection unit 18 and executes the maximum region searching process within a range satisfying the superimposing condition. That's fine. For example, when the specified tilt tolerance is ± 10 degrees, the rotation angle may be changed stepwise within a range from −10 degrees to +10 degrees to search for the maximum area. Further, when an allowable range is specified for the aspect ratio, the maximum area may be searched by changing the aspect ratio stepwise within the allowable range. For example, in the case where 1.25 to 1.40 is specified as the allowable range of aspect ratio, when the shape of the original superimposition image is “horizontal: vertical = 4: 3”, “horizontal length” / Vertical length (aspect ratio) "is 1.33. In addition to the original rectangular area having an aspect ratio of 1.33 of the image for superimposition, the overlapping area determining unit 20 has an aspect ratio within the specified allowable range of 1.25 (horizontal: vertical = 5: 4) and 1.40 (horizontal: vertical = The above-mentioned maximum area search process is executed so that the rectangular area satisfying each of 7: 5) is also searched. When the permissible ranges of inclination, aspect ratio, and size are specified as the superimposing conditions, the superimposing region determining unit 20 determines the superimposing region so as to have the maximum area within the range satisfying each condition. That's fine.

  By the way, in each of the above-described embodiments, an example in which a superimposition region is determined so that the area of the superimposition image is maximized and the magnification of the superimposition image is changed is described. In some cases, it may be desirable to reduce the change in tilt and aspect ratio at the expense of some area. In this case, for example, the overlapping region determination unit 20 first changes the overlapping condition other than the area step by step to search for a region with the maximum area under each condition, and determines the overlapping condition that maximizes the area. Extract. The superimposition region determining unit 20 determines the extracted superimposition condition, the maximum area searched under the condition, the maximum area searched when the amount of change of the other superimposition condition is 0 (minimum), and the other superimposition The largest area among the maximum areas searched within the condition change range is presented together with the other superposition conditions. The superimposing region provider selects one from a combination of the presented area and other superimposing conditions corresponding to the area. The overlapping region determining unit 20 can determine a region having the area indicated by the selected combination as the overlapping region.

  As an example, consider a combination of slope and area. First, the superimposition region determination unit 20 changes the inclination in a stepwise range from −90 degrees to +90 degrees (or within an allowable range designated by the superimposing image provider), and for each inclination. The maximum area search process described with reference to FIG. 27 is performed to search for the maximum area. In FIG. 31, the figure which plotted an example of the search result is shown. The overlapping region determination unit 20 shows a search result when the inclination is minimum, a search result when the area becomes maximum with respect to the inclination, and a search result when the area becomes maximum within the change range of the inclination. Present information. Although the presentation method is not particularly limited, for example, the graph shown in FIG. 31 may be displayed on the display 54 or may be displayed in a table format. In addition, the overlapping area determination unit 20 may perform control so as to display on a different terminal device different from the image processing device 10. The provider of the image for superimposition can select one from the presented search results (combinations). For example, the keyboard 56 may be selected by selection, and when the combination is displayed on another terminal device, the desired combination is selected by operating the other terminal device. The selection result may be transmitted to the image processing apparatus 10 via the communication IF 58.

  The superimposition area determination unit 20 acquires information indicating the selected combination, and determines a rectangular area corresponding to the combination as a superimposition area on which the superimposition image is superimposed.

  In addition, when considering the combination of aspect ratio and area, the same processing as described above may be performed. Furthermore, when considering a combination of inclination, aspect ratio, and area, the overlapping region determination unit 20 searches for a result in which the area is maximized in a three-dimensional graph with the inclination, aspect ratio, and area as axes. May be extracted and presented as described above.

  Here, an example in which a combination is presented and the provider of the superimposing image is selected has been described. However, the present invention is not limited to this, and a combination that satisfies a predetermined condition is selected without causing the provider to select the combination. Thus, the overlapping area may be determined. The predetermined condition includes, for example, a combination having the largest area, a combination having the smallest inclination, or a combination having the smallest inclination when the area is equal to or greater than a predetermined value among the combinations having the maximum.

  Alternatively, an evaluation value may be assigned to each of the inclination, the aspect ratio, and the area to obtain a combination that maximizes the total value of the evaluation values, and a region that satisfies this combination may be determined as the overlapping region. Note that the evaluation value may be such that, for example, the evaluation value decreases as the inclination increases. Regarding the aspect ratio, the evaluation value may be higher as the aspect ratio is closer to 1. About an area, you may make it an evaluation value become large, so that an area becomes large. In addition, the weighting is changed according to each parameter of inclination, aspect ratio, and area, the evaluation value of each parameter × the total value of weighting is obtained, the combination having the largest total value is obtained, and the region satisfying this combination is superimposed. It may be determined as a region.

  Moreover, when the superimposition area | region determination part 20 selects the combination which satisfy | fills a predetermined condition among the combinations that become the said maximum, you may apply the method of the said evaluation value. That is, the sum of the evaluation values is obtained for each maximal combination, and the combination region in which the total value is the largest is determined as the overlapping region.

  In each embodiment described above, the inclination, aspect ratio, and magnification of the superimposition image can be changed. However, the present invention is not limited to this. For example, the color of the superposition image may be changed. By changing the superimposition image so as to be conspicuous in accordance with the color around the region where the superimposition image is superimposed, the user's attention is more easily directed.

  Moreover, although each said embodiment demonstrated the example which identifies an attention area | region based on the keyword which the user input as image selection information or attention area identification information, it is not limited to this. For example, when the image processing apparatus 10 is applied to a service for distributing an image to a user from a used car sales site on the Internet, it is clear that the user pays attention to the car from the characteristics of the site. Therefore, the area specifying unit 16 specifies the image area of the car in the original image as the attention area, and the superimposition area determination unit 20 may determine the superposition area using the areas other than the attention area as non-attention areas. In this case, when the user accesses the used car sales site, the “car” that is a feature of the site is designated as attention area specifying information.

  Further, for example, the attention area specifying information may be input by causing the user to select a category from a category menu in which a plurality of categories are displayed in a tree shape on a display unit such as the display 54 by the input unit 12. Good. As a specific example, when the user selects a cat by following a tree-like category menu of animals-mammals-cats, the region specifying unit 16 selects the cat image region of the original image as a region of interest. May be specified. Note that, for example, depending on the recognition level of the image recognition unit 40, only the middle level “mammals” of the category menu is recognized, and when the type of mammal is not recognized separately, the region specifying unit 16 selects the original image. Of these, the image area of mammals may be specified as the attention area.

  In each of the above embodiments, the case where the original image on which the superimposition image is superimposed is a still image has been described as an example, but the original image may be a moving image. When the original image is a moving image, the image for superimposition is superimposed on an area that does not impair the convenience of the user by performing the same processing as in the above embodiment for each frame constituting the moving image. Can do. In the case of a moving image, the superimposition image can be continuously superimposed on the superimposition region by tracking the superimposition region with a well-known technique (tracking the movement of the region).

  Furthermore, although the image processing program has been described above as being stored in the first storage unit 60, the image processing program is recorded on a portable recording medium such as a CD-ROM, DVD-ROM, or USB memory. It is also possible to provide it in a certain form.

  The image processing apparatus 10 can also be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit) or the like, and each functional unit of the image processing apparatus 10 is realized by, for example, an electronic circuit or the like. Is also possible.

  All documents, patent applications and technical standards mentioned in this specification are to the same extent as if each individual document, patent application and technical standard were specifically and individually stated to be incorporated by reference. Incorporated by reference in the book.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 12 Input part 14 Image selection part 16 Area | region specific part 18 Superimposition image selection part 20 Superimposition area | region determination part 22 Superimposition part 24 Output part 26 Original image memory | storage part 28 Recognition result memory | storage part 30 Superimposition image memory | storage part 32 Superimposition Image storage unit 40 Image recognition unit 42 Object recognition unit 44 Face recognition unit 46 Character recognition unit 50 Computer 52 CPU
54 Display 56 Keyboard 58 Communication IF
60 First storage unit
6 6 Memory 68 Second storage section 70 Bus

Claims (10)

An image selection unit that selects an image to be superimposed from a plurality of images in accordance with an image selection instruction;
An area specifying unit for specifying an attention area noted by the user from among the images to be superimposed selected by the image selection unit based on the image selection instruction or the input information about the attention object;
Among the images to be superimposed selected by the image selection unit, at least the shape and size of the non-target region excluding the region of interest specified by the region specifying unit, and the image to be superimposed selected by the image selection unit Based on the shape and size of the image for superimposition stored in advance in the storage unit as an image to be superimposed on, a superimposition region that determines a superimposition region in which the superimposition image can be superimposed among the non-attention regions A decision unit;
A superimposing unit that superimposes the superimposing image on a superimposing region determined by the superimposing region determining unit among the superimposition target images;
I have a,
The superimposing region determination unit has the same maximum aspect ratio as that of the superimposing image based on the shape and size of the non-attention region and the shape and size of the superimposing image and does not deviate from the non-attention region. An area is obtained, and the obtained area is determined as the overlapping area where the image for superimposition can be superimposed at a magnification that maximizes the area of the image for superimposition,
The superimposing unit superimposes the superimposing image on the determined superimposing region at a magnification that maximizes the area . An image selection unit that selects an image to be superimposed from a plurality of images in accordance with an image selection instruction;
An area specifying unit for specifying a non-attention area not noticed by a user from among the images to be superimposed selected by the image selection unit based on the image selection instruction or the information about the input target of interest;
Among the images to be superimposed selected by the image selection unit, the shape and size of the non-attention area specified by the region specifying unit, and an image to be superimposed on the image to be superimposed selected by the image selection unit in advance Based on the shape and size of the superimposition image stored in the storage unit, a superimposition region determination unit that determines a superimposition region in which the superimposition image can be superimposed among the non-attention regions;
A superimposing unit that superimposes the superimposing image on a superimposing region determined by the superimposing region determining unit among the superimposition target images;
I have a,
The superimposing region determination unit has the same maximum aspect ratio as that of the superimposing image based on the shape and size of the non-attention region and the shape and size of the superimposing image and does not deviate from the non-attention region. An area is obtained, and the obtained area is determined as the overlapping area where the image for superimposition can be superimposed at a magnification that maximizes the area of the image for superimposition,
The superimposing unit superimposes the superimposing image on the determined superimposing region at a magnification that maximizes the area . The superimposing region determination unit prioritizes superimposing images when a plurality of superimposing images each having a predetermined priority are stored in the storage unit in advance as images to be superimposed on the superimposition target image. Determining a superposition region for each of the plurality of superposition images so that a region having a larger area is allocated as the superposition image as the degree increases.
The superimposing unit superimposes each of the plurality of superimposing images on each of the determined plurality of overlapping regions.
The image processing apparatus according to claim 1. The superimposing region determination unit, when the non-attention region is configured to include a plurality of partial regions having different priorities, selects a region where the superimposing image can be superimposed for each partial region as a superimposing region candidate And selecting a superimposition region candidate selected according to a predetermined condition from the superimposition region candidates or according to an instruction as the superimposition region
The image processing apparatus according to claim 1. The superimposing region determining unit is configured to superimpose a region in which the area of the superimposing image is maximized by superimposing the superimposing image by changing at least one of an inclination and an aspect ratio of the superimposing image and a magnification of the superimposing image. Determined as an area,
The superimposing unit superimposes on the determined superimposing region in a state where at least one of the inclination and aspect ratio of the superimposing image and the magnification of the superimposing image are changed to a value that is the maximum area.
The image processing apparatus according to claim 1. The superimposing region determining unit determines the superimposing region so that at least one of a tilt and an aspect ratio of the superimposing image satisfies a preset condition.
The image processing apparatus according to claim 5. An image selection step of selecting an image to be superimposed from a plurality of images in response to an image selection instruction;
An area specifying step for specifying an attention area noted by a user from among the images to be superimposed selected by the image selection step based on the image selection instruction or input information about the attention object;
Of the images to be superimposed selected by the image selection step, at least the shape and size of the non-target region excluding the region of interest specified by the region specifying step, and the image to be superimposed selected by the image selection step Based on the shape and size of the image for superimposition stored in advance in the storage unit as an image to be superimposed on, a superimposition region that determines a superimposition region in which the superimposition image can be superimposed among the non-attention regions A decision step;
A superimposing step of superimposing the superimposing image on the superimposing region determined by the superimposing region determining step in the superimposition target image;
Have
The superimposing region determining step is based on the shape and size of the non-target region and the shape and size of the superimposing image, and has the maximum area that has the same aspect ratio as the superimposing image and does not deviate from the non-target region. An area is obtained, and the obtained area is determined as the overlapping area where the image for superimposition can be superimposed at a magnification that maximizes the area of the image for superimposition,
The superimposing step is an image processing method in which the superimposing image is superimposed on the determined superimposing region at a magnification that maximizes the area. An image selection step of selecting an image to be superimposed from a plurality of images in response to an image selection instruction;
An area specifying step for specifying a non-attention area not noticed by the user from among the images to be superimposed selected by the image selection step based on the image selection instruction or the information about the inputted attention object;
Among the images to be superimposed selected by the image selection step, the shape and size of the non-attention area specified by the region specifying step and the image to be superimposed on the image to be superimposed selected by the image selection step A superimposing region determining step for determining a superimposing region in which the superimposing image can be superimposed out of the non-attention regions based on the shape and size of the superimposing image stored in the storage unit;
A superimposing step of superimposing the superimposing image on the superimposing region determined by the superimposing region determining step in the superimposition target image;
Have
The superimposing region determining step is based on the shape and size of the non-target region and the shape and size of the superimposing image, and has the maximum area that has the same aspect ratio as the superimposing image and does not deviate from the non-target region. An area is obtained, and the obtained area is determined as the overlapping area where the image for superimposition can be superimposed at a magnification that maximizes the area of the image for superimposition,
The superimposing step is an image processing method in which the superimposing image is superimposed on the determined superimposing region at a magnification that maximizes the area. On the computer,
An image selection step of selecting an image to be superimposed from a plurality of images in response to an image selection instruction;
An area specifying step for specifying an attention area noted by a user from among the images to be superimposed selected by the image selection step based on the image selection instruction or input information about the attention object;
Of the images to be superimposed selected by the image selection step, at least the shape and size of the non-target region excluding the region of interest specified by the region specifying step, and the image to be superimposed selected by the image selection step Based on the shape and size of the image for superimposition stored in advance in the storage unit as an image to be superimposed on, a superimposition region that determines a superimposition region in which the superimposition image can be superimposed among the non-attention regions A decision step;
A superimposing step of superimposing the superimposing image on the superimposing region determined by the superimposing region determining step in the superimposition target image;
Have
The superimposing region determining step is based on the shape and size of the non-target region and the shape and size of the superimposing image, and has the maximum area that has the same aspect ratio as the superimposing image and does not deviate from the non-target region. An area is obtained, and the obtained area is determined as the overlapping area where the image for superimposition can be superimposed at a magnification that maximizes the area of the image for superimposition,
The superimposing step is an image processing program for executing a process of superimposing the superimposing image on the determined superimposing region at a magnification that maximizes the area. On the computer,
An image selection step of selecting an image to be superimposed from a plurality of images in response to an image selection instruction;
An area specifying step for specifying a non-attention area not noticed by the user from among the images to be superimposed selected by the image selection step based on the image selection instruction or the information about the inputted attention object;
Among the images to be superimposed selected by the image selection step, the shape and size of the non-attention area specified by the region specifying step and the image to be superimposed on the image to be superimposed selected by the image selection step A superimposing region determining step for determining a superimposing region in which the superimposing image can be superimposed out of the non-attention regions based on the shape and size of the superimposing image stored in the storage unit;
A superimposing step of superimposing the superimposing image on the superimposing region determined by the superimposing region determining step in the superimposition target image;
Have
The superimposing region determining step is based on the shape and size of the non-target region and the shape and size of the superimposing image, and has the maximum area that has the same aspect ratio as the superimposing image and does not deviate from the non-target region. An area is obtained, and the obtained area is determined as the overlapping area where the image for superimposition can be superimposed at a magnification that maximizes the area of the image for superimposition,
The superimposing step is an image processing program for executing a process of superimposing the superimposing image on the determined superimposing region at a magnification that maximizes the area.