JP7324475B1 - Information processing device, information processing method and information processing program - Google Patents

Abstract

Kind Code: A1 To provide a technique for easily editing a comic based on an original image, which can provide a user with attractiveness such as a sense of dynamism and realism. An information processing device according to the present disclosure is an information processing device that supports editing of comics. This information processing device acquires a partial image which is a part of an original image of a comic book and includes frame border information and/or text information and/or character information, and performs predetermined processing on the partial image. and a control unit for executing the above, wherein the control unit comprises an input layer for receiving input of predetermined input image data; / or a neural network model having an intermediate layer for extracting a feature quantity representing character information and an output layer for outputting a recognition result based on the feature quantity, wherein frame border information and/or text information and/or character A partial image is acquired by inputting data of an original image into a pre-learning model constructed by performing learning using image data containing information. [Selection drawing] Fig. 3

Description

The present invention relates to an information processing device, an information processing method, and an information processing program that support editing of comics.

2. Description of the Related Art Conventionally, cartoons have often been provided by being printed on paper such as books and magazines. On the other hand, with the recent spread of smartphones, tablet terminals, and the like, there are increasing opportunities to read comics as digital content using these electronic devices.

Here, even when reading comics as digital content, comic pictures are still images. Therefore, there is no motion in the image, and it may be difficult for the user to obtain a sense of dynamism and realism. Therefore, there is known a technique for generating a moving cartoon by transcribing a plurality of new images from the original cartoon image.

For example, Japanese Patent Laid-Open No. 2002-100000 discloses that a part of an image is cut out as a partial image from an input original image of a comic book, a plurality of new images are generated based on the partial image, and image data is generated so that the new images can be reproduced in chronological order. An apparatus is disclosed. This makes it possible to create moving image data using the original image.

JP 2012-18540 A

By adding a predetermined motion or the like to conventional comics as still images, it is possible to provide users with more attractive digital content.

Here, when trying to produce such motion comics (moving comics), the data creator must create more image data than when producing conventional still comics, which increases the man-hours required to create the comics. However, there have been problems such as a prolonged production period and an increase in production costs. On the other hand, it seems that the above problem can be alleviated by generating a plurality of new images using the original image, as in the technique described in Patent Document 1, for example. However, in this case, the data creator needs to use a predetermined input unit to sequentially input regions to be cut out when cutting out partial images from the original image, which again increases the man-hours required to create the comic. As described above, there is still room for improvement in the technology for easily editing digital content that is more attractive to users based on the original cartoon image.

An object of the present disclosure is to provide a technique that enables easy editing of comics based on original drawings that can provide attractiveness such as a sense of dynamism and realism to the user.

An information processing device according to the present disclosure is an information processing device that supports editing of comics. The information processing apparatus obtains a partial image which is a part of the original image of the comic book and includes frame border information and/or text information and/or character information; a control unit that acquires an editing process command that is a command for a predetermined editing process to be executed on the . The control unit includes an input layer for receiving input of predetermined input image data, an intermediate layer for extracting a feature amount representing frame border information and/or text information and/or character information from the input image data, and and an output layer that outputs identification results based on feature quantities, and a neural network model constructed by learning using image data containing frame border information and/or text information and/or character information. The partial image is acquired by inputting the data of the original image into the pre-learned model.

In the information processing apparatus described above, a data creator of a motion comic (moving comic) can easily extract a partial image by inputting the data of the original image of the comic into the pre-learning model. Therefore, there is no need to manually cut out a partial image from the original image, and the man-hours for creating motion comics (moving comics) can be greatly reduced. As described above, according to the above, it is possible to easily edit comics that can provide readers with attractiveness such as a sense of dynamism and realism based on the original drawings.

Here, in the above information processing apparatus, the editing process may be a process of reproducing the partial images in chronological order and/or a process of displaying a part of the partial images while they are being operated. In addition, the control unit may further acquire, as the partial image, an effect image including information on depiction of movement of an object and/or depiction of light and/or comic symbols and/or onomatopoeia. In this case, the editing process may be a process of displaying the effect image while operating it. According to this, the dynamism, realism, etc. of motion comics (moving cartoons) can be further enhanced. In other words, it is possible to further enhance the performance effect of motion comics (moving comics).

Further, in the information processing apparatus of the present disclosure, the control unit automatically generates a virtual cartoon in which frame frames, text, and images related to characters are randomly arranged, which is teacher data for making the pre-learning model learn. The pre-learning model may be trained using image data generated and containing frame border information and/or text information and/or character information in the virtual cartoon. According to this, by having a pre-learning model learn random virtual cartoons as teacher data, the work cost for machine learning can be greatly reduced. In this case, the control unit generates a frame frame of random size, arranges the frame shape of the frame frame at an arbitrary position of a predetermined background image generated in advance, and places the frame shape inside the frame shape. setting a background image as a first image within the frame frame, and setting an image obtained by randomly superimposing a predetermined character image generated in advance on the first image as a second image within the frame frame; The virtual cartoon is automatically generated by setting an image obtained by randomly superimposing a predetermined text image generated in advance at a position not overlapping the character image in the second image as a third image within the frame frame. can be generated with

Also, the present disclosure can be understood from the aspect of an information processing method by a computer. That is, the information processing method of the present disclosure is an information processing method that supports editing of comics, and a computer processes a partial image that is a part of an original image of a comic, frame border information and/or text information, and /or executing a first obtaining step of obtaining a partial image including character information and a second obtaining step of obtaining an editing process command, which is a command for a predetermined editing process to be executed on the partial image. . In the second acquisition step, an input layer for receiving input of predetermined input image data, and an intermediate layer for extracting a feature amount representing frame border information and/or text information and/or character information from the input image data. , and an output layer that outputs identification results based on the feature amount, and a neural network model that performs learning using image data containing frame border information and/or text information and/or character information By inputting the data of the original image into the constructed pre-learning model, the partial image is acquired.

Also, the present disclosure can be understood from the aspect of an information processing program. That is, the information processing program of the present disclosure is an information processing program that supports editing of comics, and a computer stores a partial image that is a part of an original image of a comic, including frame border information and/or text information and /or to execute a first acquisition step of acquiring a partial image including character information and a second acquisition step of acquiring an editing process command, which is a command for a predetermined editing process to be executed on the partial image. . In the second acquisition step, an input layer for receiving input of predetermined input image data, and an intermediate layer for extracting a feature amount representing frame border information and/or text information and/or character information from the input image data. , and an output layer that outputs identification results based on the feature amount, and a neural network model that performs learning using image data containing frame border information and/or text information and/or character information By inputting the data of the original image into the constructed pre-learning model, acquisition of the partial image is executed.

Advantageous Effects of Invention According to the present disclosure, it is possible to easily edit a comic that can provide a user with attractiveness such as a sense of dynamism and presence, based on the original drawing.

1 is a diagram showing a schematic configuration of a comic editing support system according to a first embodiment; FIG. FIG. 2 is a diagram showing in more detail the constituent elements of a server included in the comic editing support system in the first embodiment, and also showing the constituent elements of a user terminal that communicates with the server; FIG. 3 is a diagram illustrating the flow of operations of the comic editing support system according to the first embodiment; FIG. 4 is a diagram for explaining a discrimination result obtained from an input to a pre-learning model in the first embodiment, and a neural network that constitutes the pre-learning model; FIG. 5 is a diagram for explaining frame border information extracted as a partial image by a pre-learning model; FIG. 4 is a diagram for explaining text information extracted as a partial image by a pre-learning model; FIG. 4 is a diagram for explaining character information extracted as a partial image by a pre-learning model; FIG. 10 is a diagram illustrating a screen displayed in an interface used by a user to input correction information; FIG. 10 is a diagram illustrating a partial image corrected based on correction information; FIG. 11 illustrates an effect image that can be further acquired as a partial image by the server; FIG. 4 is a diagram for explaining a virtual cartoon generated by a learning unit; FIG. FIG. 4 is a diagram illustrating a virtual cartoon in which background images, character images, and text images are randomly arranged;

Embodiments of the present disclosure will be described below based on the drawings. The configurations of the following embodiments are examples, and the present disclosure is not limited to the configurations of the embodiments.

<First embodiment>
An overview of the cartoon editing support system according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram showing a schematic configuration of a comic editing support system according to the present embodiment. The editing support system 100 according to this embodiment includes a network 200 , a server 300 and a user terminal 400 . The editing support system of the present disclosure is a system for supporting editing of comics, and the editing support for comics is executed by server 300 .

Network 200 is, for example, an IP network. As long as the network 200 is an IP network, it may be wireless, wired, or a combination of wireless and wired. (not shown) to communicate with the server 300 via a LAN or WAN. Also, the network 200 is not limited to these examples, and may be, for example, a public switched telephone network, an optical line, an ADSL line, a satellite communication network, or the like.

Server 300 is connected to user terminal 400 via network 200 . Although one server 300 and four user terminals 400 are shown in FIG. 1 for the sake of simplicity, the present invention is not limited to these.

The server 300 may be any electronic device as long as it has a processing capability for arithmetic processing and processing such as data acquisition, generation, and updating. It may be a device. That is, the server 300 can be configured as a computer having a processor such as a CPU or GPU, a main storage device such as a RAM or ROM, an auxiliary storage device such as an EPROM, a hard disk drive, or a removable medium. Note that the removable medium may be, for example, a USB memory or a disk recording medium such as a CD or DVD. The auxiliary storage device stores an operating system (OS), various programs, various tables, and the like.

In addition, the server 300 is not provided with dedicated software, hardware, OS, etc. for the editing support system 100 according to the present embodiment, but can be implemented as SaaS (Software as a Service), Paas (Platform as a Service), IaaS (Platform as a Service) by a cloud server. Infrastructure as a Service) may be used as appropriate.

The user terminal 400 may be an electronic device such as a mobile terminal owned by a user who uses the editing support system 100. For example, the user terminal 400 may be a mobile terminal, a tablet terminal, a smartphone, a wearable terminal, a personal computer, or other terminal equipment. good too.

Next, based on FIG. 2, a detailed description of mainly the components of the server 300 will be given. FIG. 2 is a diagram showing in more detail the constituent elements of the server 300 included in the editing support system 100 and the constituent elements of the user terminal 400 that communicates with the server 300 in the first embodiment.

The server 300 has a communication unit 301, a storage unit 302, and a control unit 303 as functional units. By controlling each functional unit and the like, it is possible to realize each function that meets a predetermined purpose in each functional unit. However, some or all of the functions may be realized by hardware circuits such as ASIC and FPGA.

Here, the communication unit 301 is a communication interface for connecting the server 300 to the network 200 . The communication unit 301 includes, for example, a network interface board and a wireless communication circuit for wireless communication. The server 300 is communicably connected to the user terminal 400 and other external devices via the communication unit 301 .

The storage unit 302 includes a main storage device and an auxiliary storage device. The main storage device is a memory in which programs executed by the control unit 303 and data used by the control programs are developed. The auxiliary storage device is a device that stores programs executed by the control unit 303 and data used by the control programs. Note that the server 300 acquires data transmitted from the user terminal 400 or the like via the communication unit 301, and the storage unit 302 stores in advance an original image, which will be described later. The storage unit 302 also stores teacher data and pre-learning models for obtaining partial images, which will be described later.

The control unit 303 is a functional unit that controls the server 300 . The control unit 303 can be realized by an arithmetic processing device such as a CPU. The control unit 303 further includes three functional units: an acquisition unit 3031 , an edit processing unit 3032 and a learning unit 3033 . Each functional unit may be realized by executing a stored program by a CPU. Note that the learning unit 3033 may be realized by executing a stored program with a GPU, since the amount of calculation associated with machine learning is large. By using the GPU for arithmetic processing associated with machine learning in this manner, high-speed processing can be achieved. In addition, in order to perform faster processing, multiple computers equipped with such GPUs are used to construct a computer cluster, and parallel processing is performed by the multiple computers included in this computer cluster. good too.

Acquisition unit 3031 acquires a partial image that is a part of the original image of the comic. Here, the original image of the comic is image data of the original image of the comic, and the user using the editing support system 100 can transmit the original image to the server 300 in advance using the user terminal 400 . For example, the user can upload the original image to the server 300 via an interface provided by a predetermined application pre-installed on the user terminal 400 or an interface provided by a predetermined website. Then, the server 300 causes the storage unit 302 to store the original image transmitted from the user terminal 400 . Then, based on the original image stored in the storage unit 302, the acquisition unit 3031 extracts the frame border information and/or the text information and/or the character information in the original image. Get a partial image.

Here, the user terminal 400 in this embodiment has a communication unit 401, an input/output unit 402, and a storage unit 403 as functional units. The communication unit 401 is a communication interface for connecting the user terminal 400 to the network 200, and includes, for example, a network interface board and a wireless communication circuit for wireless communication. The input/output unit 402 is a functional unit for displaying information or the like transmitted from the outside via the communication unit 401 and inputting the information when transmitting the information to the outside via the communication unit 401. is. The storage unit 403 includes a main storage device and an auxiliary storage device, similar to the storage unit 302 of the server 300 .

The input/output unit 402 further has a display unit 4021 , an operation input unit 4022 , and an image/audio input/output unit 4023 . The display unit 4021 has a function of displaying various kinds of information, and is realized by, for example, an LCD (Liquid Crystal Display) display, an LED (Light Emitting Diode) display, an OLED (Organic Light Emitting Diode) display, or the like. The operation input unit 4022 has a function of receiving an operation input from the user, and is specifically realized by soft keys or hard keys such as a touch panel. The image/sound input/output unit 4023 has a function of receiving input of images such as still images and moving images. It is realized by a camera using an image sensor such as oxide-semiconductor (CMOS). Also, the image/sound input/output unit 4023 has a function of receiving input/output of sound, and is specifically realized by a microphone or a speaker.

Then, the user can transmit the original image to the server 300 using the user terminal 400 configured as described above.

The editing processing unit 3032 acquires an editing processing command that is a command for a predetermined editing processing to be executed on the partial image. Here, the editing process in this embodiment is a process of reproducing partial images in chronological order and/or a process of displaying a part of the partial images while they are being operated. A user using the editing support system 100 sends the above editing processing command to the server 300 via an interface provided by a predetermined application pre-installed on the user terminal 400 or an interface provided by a predetermined website. can be sent. Then, the editing processing unit 3032 acquires the transmitted editing processing command and executes editing processing for reproducing the partial images in time series.

The learning unit 3033 is a functional unit that builds a pre-learning model used in the processing by the acquisition unit 3031, and the details thereof will be described later.

Note that the control unit 303 functions as a control unit according to the present disclosure by executing the processing of the acquisition unit 3031, the editing processing unit 3032, and the learning unit 3033.

Here, the operation flow of the editing support system 100 in this embodiment will be described. FIG. 3 is a diagram illustrating the flow of operations of the editing support system 100 according to this embodiment. FIG. 3 illustrates the flow of operations between the server 300 and the user terminal 400 in the editing support system 100 according to this embodiment, and the processing executed by the server 300 and the user terminal 400. FIG.

In this embodiment, first, an original image is input to the user terminal 400 of a user who creates a moving comic by editing the original image using the editing support system 100 (S101). As described above, the user uploads the original image to the server 300 via, for example, an interface provided by a predetermined application pre-installed on the user terminal 400 or an interface provided by a predetermined website. can be done.

The server 300 acquires the original image data transmitted from the user terminal 400 (S102). Then, the server 300 stores the acquired original image in the storage unit 302 .

Then, the server 300 acquires a partial image based on the original image. This will be explained below.

The server 300 executes processing for calling the pre-learning model (S103). Here, the pre-learning model is a machine learning model used to extract a partial image based on the original image, and includes frame border information and/or text information and/or character information by the learning unit 3033. It is constructed in advance by performing learning using image data.

Here, FIG. 4 is a diagram for explaining the identification result obtained from the input to the pre-learning model in this embodiment and the neural network that constitutes the pre-learning model. In this embodiment, a neural network model generated by deep learning is used as a pre-learning model. The pre-learning model 30 in this embodiment includes an input layer 31 that receives input of input image data, and a feature value representing frame border information and/or text information and/or character information from the image data input to the input layer 31. and an output layer 33 for outputting the identification result based on the feature amount. In the example of FIG. 4, the pre-learning model 30 has one intermediate layer 32, the output of the input layer 31 is input to the intermediate layer 32, and the output of the intermediate layer 32 is input to the output layer 33. It is However, the number of intermediate layers 32 may not be limited to one layer, and the pre-learning model 30 may have two or more intermediate layers 32 .

Also according to FIG. 4, each layer 31-33 comprises one or more neurons. For example, the number of neurons in the input layer 31 can be set according to input image data. Also, the number of neurons in the output layer 33 can be set according to the partial image that is the identification result.

Then, neurons in adjacent layers are appropriately connected, and a weight (connection weight) is set for each connection based on the result of machine learning. In the example of FIG. 4, each neuron is connected to all neurons in adjacent layers, but the connection of neurons need not be limited to such an example and can be set as appropriate.

Such a pre-learning model 30 performs supervised learning using teacher data that is a combination of, for example, a plurality of original cartoon images and labels of frame border information and/or text information and/or character information. It is constructed by Specifically, a combination of a feature amount and a label is given to a neural network, and weights of connections between neurons are tuned so that the output of the neural network is the same as the label. In this way, a pre-learned model is obtained recursively for learning the features of the training data and estimating the result from the input.

Returning to FIG. 3, the server 300 extracts a partial image by inputting data of the original image into the pre-learning model (S104).

Here, FIG. 5 is a diagram for explaining frame border information extracted as a partial image by the pre-learning model. FIG. 5(a) shows the original image, and FIG. 5(b) shows the extracted frame border information. Frame frames having four corners (frame frames A to E illustrated in FIG. 5B) are extracted from the original image by the pre-learning model. Then, the server 300 extracts the x-coordinates and y-coordinates of the upper left, upper right, lower right, and lower left of these frame borders (for example, A1, A2, A3, and A4 of the frame border A) as frame border information.

Also, FIG. 6 is a diagram for explaining text information extracted as a partial image by the pre-learning model. FIG. 6(a) shows the original image, and FIG. 6(b) shows the extracted text information. The pre-learning model extracts text regions (texts A to H illustrated in FIG. 6B) in balloons from the original image. Then, the server 300 extracts the x-coordinate and y-coordinate of the upper left of these text areas (for example, A11 of text A) as text information, and also extracts the width and height of the text area (for example, w1, w1 and w1 of text A). h1) is extracted.

FIG. 7 is a diagram for explaining character information extracted as partial images by the pre-learning model. FIG. 7(a) shows the original image, and FIG. 7(b) shows the extracted character information. Characters (characters A and B illustrated in FIG. 7B) are extracted as white images from the original image by the pre-learning model. At this time, the pre-learning model generates grayscale data in which the character is a white image and the rest is a black image. Then, the server 300 cuts out the white region from the above grayscale data by transmitting the black region, and extracts it as character information. In the example shown in FIG. 7B, as will be described later, it is assumed that the pre-learning model has not been able to extract the character in the middle frame frame.

Then, returning to FIG. 3, the server 300 transmits the extracted partial image to the user terminal 400, and the user terminal 400 acquires the information (S105).

Then, the user using the editing support system 100 can check the transmitted partial image via the user terminal 400 and input correction information for correcting the partial image as necessary (S106). .

Here, FIG. 8 is a diagram illustrating a screen displayed on the interface used for the user to input correction information. Note that the interface illustrated in FIG. 8 is used to correct character information. The screen SC1 illustrated in FIG. 8 is displayed on the display unit 4021 of the user's user terminal 400, and the screen SC1 shows an extracted partial image SC11, an input field SC12 for correction information, and a send button SC13. Then, the user inputs correction information in the correction information input field SC12 using the operation input unit 4022 (touch panel) (in the example shown in FIG. The area SC121 where the character exists is surrounded by the touch panel by the user.), and the correction information can be sent to the server 300 by pressing the send button SC13.

Then, returning to FIG. 3, the server 300 acquires the correction information transmitted from the user terminal 400 (S107). Then, server 300 can correct the partial image based on the correction information. 9A and 9B are diagrams illustrating partial images that have been modified based on the modification information. FIG. 9A shows the original image, and FIG. 9B shows the modified partial image. ing. In this embodiment, by correcting the partial image shown in FIG. 7(b), the characters that could not be extracted in FIG. 7(b) are added to the partial image in FIG. 9(b). there is Then, in this manner, the server 300 completes acquisition of partial images (S108).

In the flow shown in FIG. 3, next, edit information regarding edit processing to be executed on the partial image is input to the user's user terminal 400 (S109). As described above, the user uploads the above editing information to the server 300 via, for example, an interface provided by a predetermined application pre-installed on the user terminal 400 or an interface provided by a predetermined website. can do. Specifically, in the above interface, for example, as an editing process command to be executed on a partial image, the user can perform a process of reproducing the partial image in chronological order, a process of displaying a part of the partial image while operating it, or Editing information can be uploaded to the server 300 by selecting both of these processes.

Then, the server 300 acquires the edit processing command transmitted from the user terminal 400 (S110), and executes the edit processing (S111). As a result, motion comics (moving cartoons) will be realized. The editing process executed by the server 300 is transmitted to the user terminal 400, and the user can confirm the editing process via the user terminal 400 that has acquired this information (S112).

According to the processing described above, the motion comic (moving comic) data creator can easily extract the partial image by inputting the data of the original image of the comic into the pre-learning model. Therefore, there is no need to manually cut out a partial image from the original image, and the man-hours for creating motion comics (moving comics) can be greatly reduced. As described above, according to the above, it is possible to easily edit comics that can provide readers with attractiveness such as a sense of dynamism and realism based on the original drawings.

In the above motion comic (moving comic), partial images extracted from the original image of the comic are reproduced in chronological order, and/or while some of the partial images are operated. The display process is executed as the editing process, but the editing process in this embodiment is not limited to this.

In the present embodiment, in the above-described motion comic (moving comic), a process of displaying an effect image while moving may be executed as an editing process.

Here, the above effect image is an image containing information on depiction of object movement and/or depiction of light and/or comic symbols and/or onomatopoeia, and can be acquired as a partial image by server 300. . In this case, in the pre-learning model 30, learning is performed using image data containing information about depiction of movement of objects and / or depiction of light and / or comic symbols and / or onomatopoeia. A layer (hidden layer) 32 extracts features that represent information about descriptions of movement of objects and/or descriptions of lights and/or comics and/or onomatopoeia.

FIG. 10 is a diagram illustrating an effect image that can be further acquired as a partial image by the server 300. As shown in FIG. In the present embodiment, an image depicting smoke can be acquired as the effect image containing information on depicting the movement of the object. In addition, as shown in FIG. 10, images of comic symbols and onomatopoeia can also be obtained.

In this way, by further acquiring a performance image as a partial image and displaying the performance image while moving, the motion comic (moving comic) can have a more lively feeling and realism. In other words, it is possible to further enhance the performance effect of motion comics (moving comics).

Further, in this embodiment, the server 300 may automatically generate teacher data for the pre-learning model 30 to learn. Specifically, the learning unit 3033 of the control unit 303 of the server 300 automatically generates a virtual cartoon in which frame borders, text, and images of characters are randomly arranged as the teacher data. Then, the learning unit 3033 causes the pre-learning model 30 to learn using the image data including the frame border information and/or the text information and/or the character information in the virtual cartoon.

Specifically, the learning unit 3033 generates a frame frame of random size, arranges the frame shape of the frame frame at an arbitrary position of a predetermined background image generated in advance, and places the frame shape inside the frame shape. The background image is set to the first image within the frame frame. Then, an image obtained by randomly superimposing a predetermined character image generated in advance on the first image is set as the second image within the frame frame. Further, an image obtained by randomly superimposing a predetermined text image generated in advance at a position that does not overlap the character image in the second image is set as the third image in the frame frame, thereby automatically generating the virtual cartoon. Generate with This will be described with reference to FIG.

FIG. 11 is a diagram for explaining the virtual comics generated by the learning unit 3033. As shown in FIG. First, as shown in FIG. 11A, the learning unit 3033 places a randomly generated frame border at an arbitrary position of an arbitrary background image generated in advance (this is a position where the frame border fits within the background image). It is a random position.). Note that the background image may be resized to a size in which the randomly generated frame border does not protrude. Then, the learning unit 3033 cuts out the background image inside the frame shape of the arranged frame border, and sets the cut background image in the frame border as the first image. In generating the frame border, the learning unit 3033 divides the page into rows determined by random numbers from 1 to 3 at random heights, and divides each row in the column direction by random numbers from 1 to 3. Divide by a random width into a number determined by . From the frame borders thus generated, the learning unit 3033 can generate frame borders by extracting frame borders having heights and widths equal to or greater than a certain level and dividing them at random.

Next, the learning unit 3033 randomly superimposes a predetermined character image generated in advance on the first image, as shown in FIG. 11(b). Note that the character image may be resized so that the character image to be superimposed and displayed fits within the frame frame. Further, when the character image to be superimposed does not fit within the frame frame, the portion protruding from the frame frame may be deleted. Then, in this manner, an image in which the background image and the character image are randomly arranged within the frame frame is set as the second image within the frame frame. Note that the character image described above is, for example, an image randomly extracted from a database in which a plurality of arbitrary character images are stored, and can be randomly extracted regardless of the background depicted as the first image.

Next, the learning unit 3033 randomly superimposes a predetermined text image generated in advance on the second image, as shown in FIG. 11(c). At this time, the text image can be randomly arranged at a position that does not overlap the character image in the second image. Note that the above text image is, for example, arbitrary text drawn with a speech balloon or an onomatopoeia. Then, an image in which the background image, the character image, and the text image are randomly arranged within the frame frame is set as the third image within the frame frame.

FIG. 12 is a diagram illustrating a virtual cartoon in which background images, character images, and text images are randomly arranged. Random virtual cartoons as shown in FIG. 12 are generated by the learning unit 3033 and used as teacher data for the pre-learning model 30 to learn, thereby significantly reducing the work cost for machine learning. In other words, by automatically generating a large amount of virtual cartoons by the learning unit 3033, there is no need to manually create a large amount of teacher data in machine learning.

According to the above-described editing support system 100, it is possible to easily edit comics that can provide readers with attractiveness such as a sense of dynamism and realism based on original drawings.

<Other Modifications>
The above-described embodiment is merely an example, and the present disclosure can be modified as appropriate without departing from the scope of the present disclosure. For example, the processes and means described in the present disclosure can be freely combined and implemented as long as there is no technical contradiction.

Also, the processing described as being performed by one device may be shared and performed by a plurality of devices. For example, the editing processing unit 3032 may be formed in an arithmetic processing device separate from the server 300 . At this time, the other processing unit is preferably configured to cooperate with the server 300 . Also, the processes described as being performed by different devices may be performed by one device. In a computer system, it is possible to flexibly change the hardware configuration (server configuration) to implement each function.

The present disclosure can also be implemented by supplying a computer program implementing the functions described in the above embodiments to a computer, and reading and executing the program by one or more processors of the computer. Such a computer program may be provided to the computer by a non-transitory computer-readable storage medium connectable to the system bus of the computer, or may be provided to the computer via a network. Non-transitory computer-readable storage media include, for example, magnetic disks (floppy (registered trademark) disks, hard disk drives (HDD), etc.), optical disks (CD-ROMs, DVD disks, Blu-ray disks, etc.), any type of disk, Including read only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, optical cards, any type of medium suitable for storing electronic instructions.

DESCRIPTION OF SYMBOLS 100... Editing support system 200... Network 300... Server 301... Communication part 302... Storage part 303... Control part 400... User terminal

Claims (3)

A partial image that is a part of the original image of the manga and that contains frame border information and/or text information and/or character information, constructed by learning using predetermined image data Acquiring by inputting the data of the original image into a pre-learning model;
Acquiring an editing process command that is a command for a predetermined editing process to be executed on the partial image;
with a control unit that executes
The control unit
Automatically generate a virtual cartoon in which frame borders, text, and images related to characters are randomly arranged, and which is teacher data for making the pre-learning model learn, and using the image data contained in the virtual cartoon, training the pre-trained model;
When automatically generating the virtual cartoon,
A frame frame having a random size is generated, the frame shape of the frame frame is arranged at an arbitrary position of a predetermined background image generated in advance, and the background image inside the frame shape is the first image within the frame frame. set to image,
setting an image obtained by randomly superimposing a predetermined character image generated in advance on the first image as a second image within the frame frame;
setting an image in which a predetermined text image generated in advance is randomly superimposed and displayed in a position not overlapping the character image in the second image as a third image within the frame frame;
Information processing equipment. the computer
A partial image that is a part of the original image of the manga and that contains frame border information and/or text information and/or character information, constructed by learning using predetermined image data a first obtaining step obtained by inputting data of the original image into a pre-learning model;
a second acquiring step of acquiring an editing process command that is a command for a predetermined editing process to be executed on the partial image;
The computer
Automatically generate a virtual cartoon in which frame borders, text, and images related to characters are randomly arranged, and which is teacher data for making the pre-learning model learn, and using the image data contained in the virtual cartoon, performing training on the pre-trained model;
When automatically generating the virtual cartoon,
A frame frame having a random size is generated, the frame shape of the frame frame is arranged at an arbitrary position of a predetermined background image generated in advance, and the background image inside the frame shape is the first image within the frame frame. set to image,
setting an image obtained by randomly superimposing a predetermined character image generated in advance on the first image as a second image within the frame frame;
setting an image obtained by randomly superimposing a pre-generated predetermined text image on the second image at a position that does not overlap the character image as a third image within the frame frame;
Information processing methods. to the computer,
A partial image that is a part of the original image of the manga and that contains frame border information and/or text information and/or character information, constructed by learning using predetermined image data a first obtaining step obtained by inputting data of the original image into a pre-learning model;
a second acquiring step of acquiring an editing process command that is a command for a predetermined editing process to be executed on the partial image;
to said computer;
Automatically generate a virtual cartoon in which frame borders, text, and images related to characters are randomly arranged, and which is teacher data for making the pre-learning model learn, and using the image data contained in the virtual cartoon, causing the pre-trained model to perform learning;
When automatically generating the virtual cartoon,
A frame frame having a random size is generated, the frame shape of the frame frame is arranged at an arbitrary position of a predetermined background image generated in advance, and the background image inside the frame shape is the first image within the frame frame. set to image,
setting an image obtained by randomly superimposing a predetermined character image generated in advance on the first image as a second image within the frame frame;
setting an image obtained by randomly superimposing a predetermined text image generated in advance at a position not overlapping the character image in the second image as a third image within the frame frame;
Information processing program.

Images