JP2023055725A - Image display system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image that a user wants by an intuitive operation.

SOLUTION: The image display system includes: a first display controller for displaying a reference image in a display surface; and a second display controller for selectably displaying a plurality of candidate images each having image information different from the reference image in a region near the display region of the reference image of the display surface. The image data of the candidate images is included in a search region on a predetermined space determined according to the image data of the reference image.

SELECTED DRAWING: Figure 8

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to an image display system and program.

In recent years, techniques for automatically generating images using AI (Artificial Intelligence) have been developed. For example, Non-Patent Document 1 below discloses a technique for automatically generating an illustration of a person using an algorithm called GAN (Generative Adversarial Networks).

Non-Patent Document 2 discloses a technology for automatically generating images specialized for anime characters, and an interface for generating anime characters according to user's preferences.

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio, "Generative Adversarial Nets" In NIPS 2014. Yanghua Jin, Jiakai Zhang, Minjun Li, Yingtao Tian, Huachun Zhu, Zhihao Fang, "Towards the Automatic Anime Characters Creation with Generative Adversarial Networks" eprint arXiv, https://arxiv.org/abs/1708.05509

However, in the technique disclosed in Non-Patent Document 2, since abstract parameters are used as input values for automatic image generation, a large number of trials are required for the user to automatically generate an image desired by the user. , a heavy burden on the user.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image display system and a program that enable a user to obtain an image desired by an intuitive operation.

According to the present invention, the first display control unit displays a reference image on a display surface, and a plurality of candidate images each having image information different from that of the reference image are displayed around the display area of the reference image on the display surface. and a second display control unit for selectably displaying, wherein image data of each of the plurality of candidate images is included in a search area in a predetermined space determined based on image data of the reference image. An image display system is provided.

Further, according to the present invention, the computer is configured to display a reference image on the display surface; second display control means for selectably displaying in the vicinity of the display area of the predetermined space, wherein the image data of each of the plurality of candidate images is determined based on the image data of the reference image; A program is provided, contained in the search area above.

Other problems disclosed by the present application and solutions thereof will be clarified by the section of the embodiment of the invention and the drawings.

According to the present invention, an image desired by the user can be obtained by an intuitive operation.

1 is a diagram showing an overall configuration example of an image display system according to an embodiment of the present invention; FIG. It is a figure which shows the hardware structural example of the image processing server which concerns on the same embodiment. It is a figure which shows the hardware structural example of the user terminal which concerns on the same embodiment. It is a figure which shows the functional structural example of the control part which concerns on the same embodiment. It is a figure which shows an example of display control by the 1st display control part which concerns on the same embodiment. It is a figure which shows an example of the predetermined space and search area which concern on the same embodiment. It is a figure which shows an example of the area|region setting process by the area|region setting part which concerns on the same embodiment. It is a figure which shows an example of display control by the 2nd display control part which concerns on the same embodiment. It is a figure which shows an example of the acquisition process by the operation acquisition part which concerns on the same embodiment. It is a figure which shows an example of the predetermined space and search area which concern on the same embodiment. It is a figure which shows an example of the display control process by the 1st display control part which concerns on the same embodiment. It is a figure which shows an example of the predetermined space and search area which concern on the same embodiment. It is a figure which shows an example of the adjustment process by the adjustment part which concerns on the same embodiment. FIG. 10 is a diagram showing an example of processing related to adjustment of a selection target group by an adjustment unit according to the embodiment; FIG. 10 is a diagram showing an example of processing related to adjustment of a selection target group by an adjustment unit according to the embodiment; FIG. 11 is a diagram showing an example of search area setting processing by an area setting unit according to a modification of the embodiment; FIG. 11 is a diagram showing an example of search area setting processing by an area setting unit according to a modification of the embodiment; FIG. 11 is a diagram showing an example of search area setting processing by an area setting unit according to a modification of the embodiment; It is a flowchart which shows the flow of the process performed by the image display system of the same embodiment.

The contents of the embodiments of the present invention are listed and explained. An image processing apparatus and program according to an embodiment of the present invention have the following configurations.

[Item 1]
a first display control unit for displaying a reference image on a display surface;
a second display control unit for selectively displaying a plurality of candidate images each having image information different from that of the reference image around a display area of the reference image on the display surface;
with
The image display system, wherein the image data of each of the plurality of candidate images is included in a search area on a predetermined space determined based on the image data of the reference image.
[Item 2]
The first display control unit, when a selection of one of the plurality of candidate images displayed on the display surface is input, causes the selected one image to be displayed as the reference image. 2. The image display system according to 1.
[Item 3]
3. The image display system according to item 2, wherein the first display control unit displays the selected one image as the reference image in a region where the reference image was displayed.
[Item 4]
Item 3, wherein the selection input for one of the plurality of candidate images is an input based on movement of the operating body on the display surface toward an area in which the reference image of the one image is displayed. The image display system described in .
[Item 5]
5. The image display according to any one of items 1 to 4, wherein the search area is set based on a distance between image data of the reference image and image data of an image previously selected as the reference image. system.
[Item 6]
6. The image display system according to item 5, wherein the search area is set based on a distance between image data of the reference image and image data of a plurality of images previously selected as the reference image.
[Item 7]
7. The image display system according to any one of items 1 to 6, wherein the search area is set based on a distribution in the predetermined space of image data of a plurality of images previously selected as the reference image. .
[Item 8]
8. The image display system according to item 7, wherein the search area is set based on the transition in the predetermined space of the image data of the plurality of images previously selected as the reference image.
[Item 9]
9. The image display system according to any one of items 1 to 8, wherein the search area is set based on an operation on a mapping screen of image data of an image group in the predetermined space displayed on the display surface.
[Item 10]
10. The image display system according to any one of items 1 to 9, wherein the predetermined space is a space obtained by subjecting the coordinate space of the image data of the image to dimension reduction processing or dimension number adjustment processing.
[Item 11]
11. The image display system according to any one of items 1 to 10, wherein the image data is obtained by vectorizing the image.
[Item 12]
The second display control unit controls the display mode of each of the plurality of candidate images on the display surface based on the distance in the predetermined space between the reference image and each of the plurality of candidate images. The image display system according to any one of items 1-11.
[Item 13]
The second display control unit determines a display position of each of the plurality of candidate images on the display surface based on the predetermined spatial distance between the reference image and each of the plurality of candidate images. 13. The image display system according to item 12.
[Item 14]
The second display control unit controls the distance between each of the plurality of candidate images and the reference image on the display surface based on the predetermined spatial distance between the reference image and each of the plurality of candidate images. 14. The image display system according to item 13, wherein the image display system determines:
[Item 15]
The second display control unit controls the display size of each of the plurality of candidate images on the display surface based on the predetermined spatial distance between the reference image and each of the plurality of candidate images. , the image display system according to any one of items 12 to 14.
[Item 16]
Any one of items 1 to 15, wherein the second display control unit displays the candidate image selected from the selection target group having a size set according to the processing status of the image display system in the search area. 1. The image display system according to claim 1.
[Item 17]
17. The image display system of item 16, wherein the size of the selection population relative to the search area increases or decreases as the image display system progresses.
[Item 18]
The second display control unit displays the plurality of candidate images having image data included in a new search area based on an input to an area change object for changing the size of the search area displayed on the display surface. , the image display system according to any one of items 1 to 17.
[Item 19]
the computer,
a first display control means for displaying a reference image on a display surface;
a second display control means for selectively displaying a plurality of candidate images each having image information different from that of the reference image around the display area of the reference image on the display surface;
function as
The program according to claim 1, wherein image data of each of the plurality of candidate images is included in a search area on a predetermined space determined based on image data of the reference image.

An embodiment of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a diagram showing an example of the overall configuration of an image display system according to one embodiment of the present invention. The image display system 1 of this embodiment includes an image processing server 10 . The image processing server 10 is communicably connected to one or more user terminals 20 via a communication network NW. The communication network NW is, for example, the Internet, and is constructed by a public telephone line network, a mobile telephone line network, a wireless communication path, Ethernet (registered trademark), and the like.

The image display system 1 of the present embodiment is intended to provide one or more users with a service of automatically generating an image desired by the user based on an image selected according to the user's operation. is. In the image display system 1 of the present embodiment, for example, the image processing server 10 transmits a plurality of images to the user terminal 20, the user operates the user terminal 20 to select at least one image from the plurality of images, and the image Processing server 10 automatically generates at least one image based on such selection. The image processing server 10 transmits automatically generated images to the user terminal 20, and the user operates the user terminal 20 again to select one image. By repeating such processing, the user can obtain an image that best matches his or her orientation by simply continuing to intuitively select an image that the user desires. Note that the image display system 1 stores images prepared or generated in advance in a predetermined storage or the like, and provides a service of presenting an image close to the user's request to the user according to the user's operation. may In such a case, for example, it is possible to reduce the calculation cost of the image processing server 10 and reduce the latency.

The image processing server 10 is a computer for automatically generating an image desired by the user based on an image selected according to the user's operation on the user terminal 20 . The image processing server 10 may be, for example, a personal computer, a workstation, or a virtual computer by cloud computing. The image processing server 10 is assumed to execute these functions as a so-called web application. Although the image processing server 10 is illustrated as a single server in FIG. can execute processing in this system. Also, here, the increase or decrease in the number of image processing servers that perform distributed processing can be dynamically changed according to the load of each processing and/or the size and number of images to be processed. Also, the image processing server 10 may process image data online.

The user terminal 20 is terminal equipment such as a mobile phone, a smart phone, a game machine, a tablet computer, or a personal computer.

The user terminal 20 transmits to the image processing server 10 information about the image selected by the user from among the images displayed on the display unit 26, and receives from the image processing server 10 an image generated based on the transmitted information. For example, the user terminal 20 may be installed with application software, a web browser, etc. corresponding to the image display system 1 according to the present embodiment.

FIG. 2 is a diagram showing a hardware configuration example of the image processing server 10 according to this embodiment. The image processing server 10 includes at least a control unit 11, a memory 12, a storage 13, a transmission/reception unit 14, an input/output unit 15, an image control unit 16, an image memory 17, and the like. Connected.

The control unit 11 is an arithmetic unit that controls the overall operation of the image processing server 1, controls transmission and reception of data between elements, executes applications, and performs information processing necessary for authentication processing. The control unit 11 is, for example, a CPU (Central Processing Unit), and executes a program or the like stored in the storage 13 and developed in the memory 11 to carry out each information process. The control unit 11 also has a function of controlling a search range for candidate images, which will be described later.

The memory 12 includes a main memory composed of a volatile memory device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory composed of a non-volatile memory device such as a flash memory or a HDD (Hard Disc Drive). . The memory 12 is used as a work area or the like for the control unit 11, and stores a BIOS (Basic Input/Output System) executed when the image processing server 10 is started, various setting information, and the like.

The storage 13 stores various programs such as application programs. A database (not shown) storing data used for each process may be constructed in the storage 13 .

The transmission/reception unit 14 connects the image processing server 10 to the communication network NW. The transmitting/receiving unit 14 may include a short-range communication interface such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). The transmission/reception unit 14 receives, for example, information about an image selected from the user terminal 20 and transmits image data automatically generated by the image processing server 10 to the user terminal 20 . The input/output unit 15 is information input equipment such as a keyboard, mouse, controller, etc., and output equipment such as a display.

The image control unit 16 has a function of automatically generating an image. The image control unit 16 is, for example, a GPU (Graphics Processing Unit), and can simultaneously perform parallel processing of instructions from a host CPU using a large number of cores. Also, the image control unit 16 may simultaneously perform parallel processing using an accelerator or an accelerator cluster. Examples of accelerators include, but are not limited to, CPUs, GPUs, FPGAs, ASICs, uniquely defined neural network-dedicated computing cores, inference-dedicated optical computing devices, and the like. For automatic image generation processing, known machine learning algorithms such as Boltzmann machine, Encoder-Decoder, GAN, VAE (Variational Autoencoder), and StyleTransfer are used.

The image memory 17 is used to store automatically generated image data. The image memory 17 is, for example, a VRAM (Video Random Access Memory). A bus 18 is commonly connected to the above elements and transmits, for example, address signals, data signals and various control signals.

FIG. 3 is a diagram showing a hardware configuration example of the user terminal 20 according to this embodiment. The user terminal 20 includes at least a control unit 21 , a memory 22 , a storage 23 , a transmission/reception unit 24 , an input/output unit 25 , a display unit 26 and the like, which are electrically connected to each other through a bus 27 . Components of the control unit 21, the memory 22, the storage 23, the transmission/reception unit 24, and the input/output unit 25 have the same hardware functions as those of the components of the image processing server 10 described above, so description thereof will be omitted.

The display unit 26 has a function of displaying various information. The display unit 26 may be implemented by, for example, a display. Further, the display unit 26 may be realized by integrating the functions of the input/output unit 25, such as a touch panel. In this embodiment, the display unit 26 is described as being realized by a touch panel (that is, having a function of accepting user operations).

FIG. 4 is a diagram showing a functional configuration example of the control unit 11 according to this embodiment. The control unit 11 includes an image acquisition unit 101 , a first display control unit 102 , an area setting unit 103 , a second display control unit 104 , an operation acquisition unit 105 and an adjustment unit 106 .

Note that the image acquisition unit 101, the first display control unit 102, the area setting unit 103, the second display control unit 104, the operation acquisition unit 105, and the adjustment unit 106 are the control unit 11 or the image control unit 16 provided in the image processing server 10. is implemented by reading a program stored in the storage 13 into the memory 12 or the image memory 17 and executing it. Also, the processing executed by each unit can be processed in parallel by a plurality of devices in the image processing server 10 and/or other servers depending on the processing load and hardware resources.

The image acquisition unit 101 has a function of acquiring image data. For example, the image acquisition unit 101 acquires image data stored in advance in the storage 13 or image data automatically generated by the image control unit 16 (which may be stored in the image memory 17). Further, the image acquisition unit 101 may acquire image data from external hardware such as the user terminal 20 through the input/output unit 15 or the like. Note that the image data referred to here is not limited to data that defines the image itself, and may be data that defines the image obtained by, for example, compressing the image data in advance or extracting the feature amount. .

The first display control unit 102 has a function of displaying the reference image on the display surface. A reference image according to the present embodiment means an image having image data that is an input parameter for automatic image generation processing by the image control unit 16 . Such a reference image is displayed on the display surface of the display unit 26 of the user terminal 20. FIG.

FIG. 5 is a diagram showing an example of display control by the first display control unit 102 according to this embodiment. As illustrated, a screen 200 of a web application is displayed on the display surface 26 a of the user terminal 20 . A reference image 201 is displayed in the central portion of the screen 200 . Around the reference image 201, a group of candidate images 202a-202n are displayed. Here, the candidate image is a part or all of the images that are automatically generated based on the reference image 201 and that exist within a search area on a predetermined space, which will be described later. Predetermined space means, for example, a multi-dimensional vector space that defines image data.

Also, on the screen 200, thumbnails 203a to 203c of images selected as reference images in the past are displayed in chronological order. Also, a thumbnail 204 of the current reference image 201 is displayed adjacent to the thumbnail 203c. Also, a slider bar 205 for adjusting the size of the search area, which will be described later, is displayed at the top of the screen 200 . Note that the slider bar 205 does not necessarily have to be displayed on the screen 200 .

The area setting unit 103 has a function of setting a search area. A search area means an acquisition target area of a candidate image in a predetermined space. The image acquisition unit 101 acquires image data of candidate images included in the search area set by the area setting unit 103 .

FIG. 6 is a diagram showing an example of the predetermined space and search area according to this embodiment. As shown in FIG. 6, in a predetermined space 500, image data of an image group generated in advance or generated by the image control unit 16 is schematically mapped. This screen is called a mapping screen. Although the number of dimensions of the predetermined space 500 is not particularly limited, the predetermined space 500 is represented two-dimensionally in FIGS. 6 to 8 for the purpose of explanation. Also in the following description, the predetermined space 500 is represented two-dimensionally.

FIG. 7 is a diagram showing an example of area setting processing by the area setting unit 103 according to this embodiment. As shown in FIG. 7, the region setting unit 103 sets a region within a distance R1 from the coordinates of the image data 501 of the reference image as the search region 502 . The images included in the search area 502 set here have a high degree of similarity in their image data, and are images that are likely to be relatively similar to the reference image. In addition, the distance here is, for example, Euclidean distance, Manhattan distance, Mahalanobis generalized distance, Earth Mover distance, Kullback-Leibler information amount, etc., which is defined in a multidimensional space, and the type is not particularly limited. .

The second display control unit 104 has a function of displaying candidate images included in the search area 502 . For example, the image acquisition unit 101 acquires, as candidate images, one or more images included in the search area 502 shown in FIG. It is displayed on the display surface 26a. At this time, the second display control unit 104 displays the reference image in a selectable manner around the display area of the reference image on the display surface.

FIG. 8 is a diagram showing an example of display control by the second display control unit 104 according to this embodiment. As illustrated, candidate images 202a to 202n are displayed around a reference image 201 on a screen 200. FIG. Such candidate images 202a to 202n can be displayed so as to be selectable by user's operation.

The display mode of these candidate images 202a to 202n is not particularly limited. For example, the second display control unit 104 may control the display mode of the candidate image 202 based on the distance between the reference image 201 and each of the candidate images 202 in a predetermined space.

The display mode referred to here may be, for example, the display position. Specifically, the second display control unit 104 adjusts the positions of the candidate image 202 and the reference image 201 on the screen 200 according to the distance between the reference image 201 and the candidate image 202 in the predetermined space. may be determined. That is, if the distance between the reference image 201 and the candidate image 202 is long, the second display control unit 104 displays the candidate image 202 at a position away from the reference image 201 on the screen 200, and the reference image 201 and the candidate image 202 are displayed. If the distance to is short, the candidate image 202 may be displayed on the screen 200 at a position close to the reference image 201 . Accordingly, the user can intuitively grasp the degree of similarity between the reference image 201 and the candidate image 202 simply by looking at the screen 200 .

Also, the display mode may be the size on the screen 200, for example. As shown in FIG. 8, for example, the second display control unit 104 controls the display size of the candidate image 202 according to the distance between the reference image 201 and the candidate image 202 in a predetermined space. may Specifically, the second display control unit 104 increases the display size of the candidate image 202 if the distance between the reference image 201 and each of the candidate images 202 in the predetermined space is short, and increases the display size of the candidate image 202 if the distance is long. display size may be reduced. Accordingly, the user can intuitively grasp the degree of similarity between the reference image 201 and the candidate image 202 simply by looking at the screen 200 .

Also, as shown in FIG. 8, the display mode of the candidate image 202 may be adjusted by combining the display position and the display size. In the example shown in FIG. 8, the candidate images 202 are arranged in a grid with the reference image 201 at the center. The candidate images 202 may be displayed in a manner in which the display positions of the groups of the reference image 201 and the candidate images 202 are separated.

The operation acquisition unit 105 has a function of acquiring information related to the candidate image 202 selected based on the operation on the user terminal 20 . For example, when the candidate image 202 is selected by the user's operation on the display surface 26 a of the user terminal 20 , the user terminal 20 transmits information regarding the selected candidate image 202 to the image processing server 10 . At this time, the operation acquisition unit 105 acquires the information via the transmission/reception unit 14 .

FIG. 9 is a diagram illustrating an example of acquisition processing by the operation acquisition unit 105 according to this embodiment. As illustrated, it is assumed that a candidate image 202m is selected on the screen 200 by an operating body 206 such as a user's hand. In this case, the user terminal 20 transmits information related to the candidate image 202m selected by the operating tool 206 to the image processing server 10, and the operation acquisition unit 105 acquires the information.

Note that the operation acquisition unit 105 may acquire information related to the candidate image 202m specified by an operation such as tapping or clicking a portion of the screen 200 where the candidate image 202m is displayed, or may acquire information related to the candidate image 202m as a reference. The information may be acquired by an operation (for example, a slide or flick operation) of moving the operating body 206 toward the area where the image 201 is displayed. This makes it intuitive for the user to select the candidate image 202m as the next reference image.

FIG. 10 is a diagram showing an example of the predetermined space and search area according to this embodiment. As illustrated, in a predetermined space 500, image data 503 of a candidate image 202m related to information acquired by the operation acquisition unit 105 is identified. At this time, the candidate image 202m becomes the next reference image 201. FIG. The image acquisition unit 101 acquires the candidate image 202m as the reference image 201. FIG.

FIG. 11 is a diagram showing an example of display control processing by the first display control unit 102 according to this embodiment. As illustrated, the first display control unit 102 displays the candidate image 202m selected in FIG. 9 as the reference image 201. FIG. At this time, as shown in FIG. 11, the candidate image 202m is displayed as a new reference image at the position where the reference image 201 was previously displayed. As a result, the next processing based on the new reference image can be performed. Note that the first display control unit 102 may display the thumbnail 206 of the candidate image newly selected as the reference image 201 adjacent to the thumbnail 204 of the previous reference image.

FIG. 12 is a diagram showing an example of the predetermined space and search area according to this embodiment. As shown in the figure, when the previous candidate image 202m is selected as the new reference image 201, the area setting unit 103 sets the area within the distance R2 centering on the coordinates of the image data 503 of the new reference image 201. is set as the search area 504 . In the next process, images having image data included in this search area 504 are targeted as candidate images.

Note that the distance R2 from the center of the search range 504 may be the same as or different from the distance R1 from the center of the previous search range 502 . For example, distance R2 may be less than distance R1. Since an image desired by the user is more likely to be generated as the process progresses, a candidate image with higher accuracy can be obtained more easily by reducing the distance. Also, the size of the distance may be changed in sequence according to the progress of the processing, or may be freely selected using a slider bar 205 as shown in FIG. A modification of such a search range setting method will be described later.

By sequentially repeating the processing by the functional unit, the user selects a candidate image, and the candidate image is further narrowed down using the candidate image as a new reference image. In this manner, the image desired by the user can be easily searched by simply repeating the selection of the candidate image by the user. Although not shown in FIG. 11 and the like, when the selection by the user is completed (that is, when the image last selected as the reference image is the image desired by the user), the image is displayed on the screen 200. A button or the like for ending the search may be displayed.

The adjusting unit 106 has a function of adjusting the dimension of the predetermined space and the format of the image data. The adjustment unit 106 is optionally provided in the image processing server 10 . When the dimensions of the predetermined space defining the image data have a certain size, the computational complexity for searching candidate images increases. In this case, the latency until the candidate images are displayed becomes large, so that the user experience (UX) deteriorates and the specs such as memory and CPU required for such calculations increase. Also, due to the so-called "curse of dimensionality", the distance between image data and the realization value of the image related to these image data (appearance of the image, in general, the shorter the distance between the image data, the closer the realization value, that is, similar images) may not be maintained. In order to avoid this negative possibility, the adjustment unit 106 has a function of adjusting the dimension of the predetermined space and the format of the image data.

For example, the adjustment unit 106 may adjust the dimension of the predetermined space that defines the image data. Dimensional adjustment means, for example, adjusting the number of dimensions of a vector or reducing the number of dimensions. FIG. 13 is a diagram showing an example of adjustment processing by the adjustment unit 106 according to this embodiment. Referring to FIG. 13, before adjustment processing by adjustment unit 106, image data 511 and image data 512 are defined in m-dimensional space 510, and image data 521 and image data 522 are defined in n-dimensional space. 520. Here, the image data 511 and the image data 521 have corresponding image realization values, the image data 512 and the image data 522 have corresponding image realization values, and the mutual realization values are close to each other. are similar to each other. Also, the number of dimensions is set to m>n.

As shown in FIG. 13, the distance between the image data 511 and the image data 512 in the m-dimensional space 510 is large. likely to be judged. Therefore, the adjustment unit 106 performs processing related to dimensionality reduction and dimensional adjustment so as to obtain an appropriate dimensional space (in this case, an n-dimensional space 520), thereby obtaining an n-dimensional space between the image data 512 and the image data 522. 520 can be brought closer to the appropriate distance. As a result, it is possible to acquire a more appropriate image in acquiring the candidate image. As a technique for dimensionality reduction and dimensionality adjustment, for example, principal component analysis, LDA (Linear Discriminant Analysis), t-SNE (t-distributed Stochastic Neighbor Embedding), etc. can be used. The above-described processing can be performed in the space after the adjustment processing. Such processing can also be applied to image generation processing. For example, when learning a learning model used in image generation processing, the above processing can be considered in learning.

Further, the adjustment unit 106 may vectorize the image data acquired by the image acquisition unit 101 using various techniques. For example, the adjustment unit 106 may reduce the dimensionality of high-dimensional image data. Also, if the image data is not vector data, the adjustment unit 106 may appropriately vectorize the image data. Vectorization of image data is not particularly limited, for example, principal component analysis, LDA (Linear Discriminant Analysis), t-SNE (t-distributed Stochastic Neighbor Embedding) and other statistical analysis means are used. Alternatively, a classifier such as Encoder-Decoder, AutoEncoder, StyleTransfer, or VGG may be used for feature extraction, and a method of vectorization may be used by learning locations to be vectorized at the same time as image generation during model learning.

Further, the adjustment unit 106 may adjust the size of the selection target group in the search area set by the area setting unit 103 according to the processing status of the image display system 1 . Here, the selection target group means a group in which the targets to be displayed on the screen 200 are limited in advance according to the processing status among the group of candidate images included in the search area. The second display control unit 104 may display the candidate images included in this selection target group on the display surface 26a.

14 and 15 are diagrams showing an example of processing related to adjustment of the selection target group by the adjustment unit 106 according to this embodiment. FIG. 14 relates to the previous processing, and FIG. 15 relates to the subsequent processing. As shown in FIG. 14, in the previous process, a search area 532 is set in a predetermined space 530 based on image data 531 relating to a reference image. On the other hand, as shown in FIG. 15, in subsequent processing, a search area 535 is set in a predetermined space 530 based on image data 534 relating to a new reference image. At this time, the search area 532 is wider than the search area 535 .

In the above process, while the search area is widened in order to increase the diversity of the candidate images, the precision does not have to be high. Rather, if you try to maintain the accuracy, the search area is wide, so the processing load increases. Therefore, the adjustment unit 106 adjusts the selection target group that is the target of the candidate image inside the search area. The size of the selection target population is set relatively small. In the example shown in FIG. 14, for the sake of explanation, an area in the vicinity of predetermined coordinates is set as the selection target group. may be set. By narrowing down the size of the selection target population in this way, the initial processing can be performed at high speed.

On the other hand, in the subsequent processing shown in FIG. 15, the selection by the user is in progress. Therefore, the adjustment unit 106 may set the size of the selection target group to be relatively large, and may adjust the entire search area as the selection target group. This makes it possible to select an image with a high degree of certainty for the user's preference. Conversely, the adjustment unit 106 may reduce the size of the selection target group as the process progresses.

Each function of the software configuration of the control unit 11 has been described above.

Next, a modification of the area setting process by the area setting unit 103 will be described. A region setting unit 103 sets a search region based on the image data of the reference image 201 . At that time, the position and size of the search area are not limited to the setting method described above.

FIG. 16 is a diagram showing an example of search area setting processing by the area setting unit 103 according to the modification of the present embodiment. As shown in the figure, a group of images exists in a predetermined space 540, and images related to image data 541, 542, and 543 are selected as reference images by past processing by the image display system 1. FIG. In this case, the area setting section 103 may set the search area based on the image data 541-543. For example, the region setting unit 103 may set the search region 545 based on the position of the geometric center 544 such as the center of gravity or the center of the image data 541-543.

FIG. 17 is a diagram showing an example of search area setting processing by the area setting unit 103 according to a modification of the present embodiment. As shown in the figure, a group of images exists in a predetermined space 550, and images related to image data 551 to 557 are selected as reference images by past processing by the image display system 1. FIG. In this case, the area setting section 103 may set the search area based on the distribution of the image data 551-557. For example, the area setting unit 103 may perform statistical processing on the image data 551 to 557 and set the search area 558 based on statistical values such as the average value and variance.

FIG. 18 is a diagram showing an example of search area setting processing by the area setting unit 103 according to a modification of the present embodiment. As shown in the figure, there are images in a predetermined space 560, and images related to image data 561, 562, 563, and 564 are selected as reference images by past processing by the image display system 1. FIG. In a series of repeated processes, image data 561, 562, 563, and 564 are selected as reference images in this order.

In this case, the region setting unit 103 may set the search region based on the transition of selection of the image data 561 to 564 as the reference image. As shown in the figure, for example, when the image related to the image data 564 is selected as the reference image, the region setting unit 103 sets the region centered on the weighted center of gravity of the transition of the image data 561 as the search region 565 . may be set as As a result, the region where the user's desired image is thought to exist can be derived from past transitions, so search efficiency can be improved. The region setting unit 103 may set the search region 566 by changing the weighting method (for example, by increasing the weight of the image data related to the reference image selected in the immediately preceding process).

Alternatively, the area setting unit 103 may manually set and adjust the search area by operating the slider bar 205 displayed on the screen 200 shown in FIG. In addition, the area setting unit 103 displays on the display surface 26a a screen of an image group mapped in a predetermined space 500 as shown in FIG. 6, and sets a search area based on the user's operation on the screen. You may

A modification of the search area setting process by the area setting unit 103 has been described above.

FIG. 19 is a flow chart showing the flow of processing executed by the image display system 1 of this embodiment. First, the image acquisition unit 101 acquires image data of a reference image (SQ101). Next, the area setting unit 103 sets a search area in a predetermined space based on the image data of the reference image (SQ102). Next, the image acquisition unit 101 acquires image data of candidate images included in the search area (SQ103).

Next, the first display control unit 102 performs processing for displaying the reference image on the display surface (SQ104). Further, the second display control unit 104 performs processing for displaying the candidate image on the display surface (SQ105). Next, in response to the user's operation on the display surface 26a of the user terminal 20 (SQ106), if one of the candidate images is selected (SQ106/YES), the operation acquisition unit 105 acquires information related to the selected candidate image. (SQ107). On the other hand, if no candidate image is selected in response to the user's operation on the display surface 26a (for example, a button to complete the process is selected) (SQ106/NO), the process by the image display system 1 ends.

Embodiments of the present invention include, for example, an electronic device, a system, a method performed by the electronic device or system as described above, a program for operating the electronic device, and a non-transitory tangible medium in which the program is recorded. It can contain a medium.

Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the technical scope of the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present invention can conceive of various modifications or modifications within the scope of the technical idea described in the claims. is naturally within the technical scope of the present invention.

For example, in the above embodiment, the image display system 1 is composed of the image processing server 10 and the user terminal 20, but the present invention is not limited to such an example. For example, in another embodiment, the image display system 1 may be configured with only the user terminal 20 . In this case, the functions of the image processing server 10 may be implemented by various hardware of the user terminal 20 .

Further, each step in the above embodiment does not necessarily have to be processed in chronological order according to the order described in the flowchart. For example, each step in the processing of the above-described embodiment may be processed in an order different from the order described in the flowchart, or may be processed in parallel.

Also, the effects described herein are merely illustrative or exemplary, and are not limiting. In other words, the technology according to the present invention can produce other effects that are obvious to those skilled in the art from the description of this specification in addition to or instead of the above effects.

1 image display system 10 image processing server 11 control unit 15 image control unit 16 image memory 20 user terminal 26 display unit 26a display surface 101 image acquisition unit 102 first display control unit 103 area setting unit 104 second display control unit 105 operation Acquisition unit 106 Adjustment unit

Claims (4)

A content display system for automatically generating user-desired content,
a first display control unit that displays the reference content on a display surface;
a second display control unit for selectively displaying candidate contents each having content information different from the reference content in an area of the display surface different from the display area of the reference content;
with
The content data for each of said candidate content comprises:
is generated based on the content data of the reference content and has a predetermined degree of similarity with the reference content;
When a selection of one of the candidate contents displayed on the display surface is input, the first display control unit sets the selected candidate content as a new reference content to a display area of the reference content. A content display system that displays The second display control unit displays each new candidate content related to content data having a predetermined degree of similarity with the new reference content, which is determined based on content data of the selected new reference content. 2. The content display system according to claim 1, wherein the content display system allows 2. The content display system according to claim 1, wherein said content data is obtained by vectorizing said content. A content generation system for automatically generating content desired by a user,
a display control unit configured to display reference content and candidate content that is different from the reference content and has a predetermined degree of similarity with the reference content;
determining the selected one content as the reference content when a selection of one content among the candidate content is input;
an image control unit for displaying the one content as a new reference content on the display surface;
A content generation system comprising:

Images