JP7282444B1 - Program, information processing device, information processing system, information processing method - Google Patents

Abstract

A program, an information processing device, an information processing system, and an information processing method are provided that enable a user to browse content comfortably. A program for displaying content divided into frames and determining the order of frames to be read comprises a selection step for receiving selection of content, a display step for displaying the selected content, and a display step. a scrolling step of scrolling the content displayed in the frame order according to the order of the frames, an operation step of receiving an operation from a user viewing the content displayed on the computer, and a calculating step of calculating a scroll parameter related to scrolling in the scrolling step. , is executed. The scroll step scrolls the content in the order of frames based on the scroll parameter calculated in the calculation step in response to the user's operation received in the operation step. [Selection drawing] Fig. 7

Description

The present disclosure relates to a program, an information processing device, an information processing system, and an information processing method.

2. Description of the Related Art Techniques for viewing electronic content such as manga on mobile terminals such as smartphones are known.
Patent document 1 describes content that is divided into frames and has a predetermined order of frames to be read, and that allows the user to view content in an enlarged display state. Techniques for reducing system development costs while enabling browsing have been disclosed.
Japanese Patent Application Laid-Open No. 2002-200000 discloses a technique for displaying a page-turning image in such a manner that, when a page-turning operation is performed, a portion that was noted during past browsing can be easily found.

Retable 2017/130318 JP 2012-252620 A

Mobile terminals such as smartphones have a small screen, and there is a problem that users cannot browse content comfortably.
Accordingly, the present disclosure has been made to solve the above problems, and an object thereof is to provide a technology that allows a user to browse content comfortably.

A program comprising a processor, a storage unit, and a display screen, and for causing a computer to display content divided into frames and in which the sequence of frames to be read is determined, wherein the program instructs the processor to select content. a selection step of receiving; a display step of displaying the content selected in the selection step on a display screen; a scrolling step of scrolling the content displayed in the display step along the order of frames; and a calculating step of calculating a scroll parameter related to scrolling in the scrolling step. A program for scrolling content according to the order of frames based on the calculated scroll parameters.

According to the present disclosure, the user can browse content comfortably.

2 is a block diagram showing a functional configuration of system 1; FIG. 3 is a block diagram showing the functional configuration of the server 10; FIG. 2 is a block diagram showing a functional configuration of a user terminal 20; FIG. FIG. 10 is a diagram showing the data structure of a user table 1012; FIG. FIG. 10 is a diagram showing the data structure of a content table 1013; FIG. It is a flowchart which shows operation|movement of browsing processing. 3 is a screen example of a content image displayed on the user terminal 20. FIG. 2 is a block diagram showing the basic hardware configuration of computer 90. FIG.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all the drawings for explaining the embodiments, common constituent elements are denoted by the same reference numerals, and repeated explanations are omitted. It should be noted that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Also, not all the components shown in the embodiments are essential components of the present disclosure. Each figure is a schematic diagram and is not necessarily strictly illustrated.

<Configuration of System 1>
A system 1 according to the present disclosure is an information processing system that provides a user with a content viewing service for viewing content such as manga and comics in which frames are divided and the order of frames to be read is determined.
The system 1 includes information processing devices such as a server 10 and a user terminal 20 which are connected via a network N. FIG.
FIG. 1 is a block diagram showing the functional configuration of system 1. As shown in FIG.
FIG. 2 is a block diagram showing the functional configuration of the server 10. As shown in FIG.
FIG. 3 is a block diagram showing the functional configuration of the user terminal 20. As shown in FIG.

Each information processing device is composed of a computer having an arithmetic device and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the server 10 and the user terminal 20, descriptions overlapping with the basic hardware configuration of the computer and the basic functional configuration of the computer, which will be described later, will be omitted.

<Configuration of Server 10>
The server 10 is an information processing device that provides content such as manga and comics to users.
The server 10 has a storage unit 101 and a control unit 104 .

<Configuration of Storage Unit 101 of Server 10>
Storage unit 101 of server 10 includes application program 1011 , learning model 1021 , user table 1012 , and content table 1013 .

The application program 1011 is a program for causing the control unit 104 of the server 10 to function as each functional unit.
Application programs 1011 include applications such as web browser applications.

The learning model 1021 is an inference model that outputs (infers) a scroll parameter using a content image as input data.
The input data may include information about user attributes of users viewing content.
The learning process of the learning model 1021 will be described later.
The learning model 1021 model is, for example, one type of machine learning, artificial intelligence, deep learning model, and the like. The learning model 1021 includes regression analysis models and arbitrary calculation models created manually.
The learning model 1021 model does not need to be a single learning model, and may be realized by switching a plurality of independent learning models.
As an example of the learning model 1021, a deep learning model using a deep neural network in deep learning will be described. The learning model 1021 does not necessarily have to be a deep learning model, and may be any machine learning or artificial intelligence model.
By applying the learning model 1021 with the content image as input data, a scroll parameter for realizing a scroll operation suitable for the user is estimated.

The user table 1012 is a table that stores and manages information on member users (hereinafter referred to as users) who use the service. By registering to use the service, the user's information is stored in a new record in the user table 1012 . This enables the user to use the service according to the present disclosure.
The user table 1012 is a table having user ID, user name, and user attribute columns with user ID as a primary key.
FIG. 4 is a diagram showing the data structure of the user table 1012. As shown in FIG.

User ID is an item that stores user identification information for identifying a user. User identification information is an item in which a unique value is set for each user.
The user name is an item that stores the name of the user. Any character string such as a nickname may be set as the user name instead of the full name.
The user attribute is an item that stores information related to user attributes such as the user's age, gender, favorite content genre, and the like.

The content table 1013 is a table for storing and managing information about content (content information).
The content table 1013 is a table having columns of content ID, content name, content attribute, and content data.
FIG. 5 is a diagram showing the data structure of the content table 1013. As shown in FIG.

The content ID is an item that stores content identification information for identifying content. Content identification information is an item in which a unique value is set for each content.
The content name is an item that stores the name of the content, such as the work name and title. Any character string can be set for the content name.
The content attribute is an item that stores information related to content attributes such as content genre, expected reader's gender, and age group.
Content data is an item that stores content such as manga and comics. Specifically, the content data can be AZW/MOBI, Topaz, CEBX, ePub, HTML5/CSS3, MCBook, Mobipocket, PDF, XMDF, . This is an item for storing a binary file in which content in a data format such as a book is stored.
The content data in this disclosure is content such as manga, comics, etc., which are divided into frames and the order of the frames to be read is determined. Content repositioned horizontally. In this case, the user can read the content by scrolling the content vertically or horizontally. The content rearranged vertically or horizontally for each frame may be generated in advance based on the content divided into frames for each page, or may be automatically generated vertically or horizontally for each frame when reading the content data. You can rearrange them in any direction.
In addition, there are many users who feel that scrolling in the vertical direction is particularly cumbersome on mobile terminals such as smartphones. Content data with scrollable content is preferred.
The content data in the present disclosure may include additional information called effect information added by the author, editor, etc. of the content data. For example, the presentation information includes information such as enlargement/reduction of spread frames, order display of spread frames, enlarged display of target frames and target parts, animation display, sound effects, and the like. Note that the control unit 204 of the user terminal 20 can display, on the display 2081 of the user terminal 20, an image or moving image obtained by adding a predetermined effect to the content data based on the effect information. The performance information does not have to be included in the content data in advance, and is automatically generated using an AI model such as arbitrary machine learning or deep learning based on the image information included in the external server or content data. It can be configured. For example, when the user touches the touch device 2065 superimposed on the display 2081 of the user terminal 20, the control unit 204 of the user terminal 20 generates an image or moving image obtained by adding a predetermined effect to the content data based on the effect information. etc. are displayed on the display 2081 of the user terminal 20 and presented to the user. The user terminal 20 may set an area on the touch device 2065 for displaying an image, moving image, or the like to which a predetermined effect is added. In addition, the user terminal 20 may display an icon for displaying an image, moving image, or the like with a predetermined effect on the display 2081 and may accept a touch to the touch device 2065 .

<Configuration of Control Unit 104 of Server 10>
The control unit 104 of the server 10 has a user registration control unit 1041 and a browsing unit 1042 . Control unit 104 implements each functional unit by executing application program 1011 stored in storage unit 101 .

The user registration control unit 1041 performs processing for storing information of users who wish to use the service according to the present disclosure in the user table 1012 .
The information stored in the user table 1012 is transmitted to the server 10 by the user opening a web page or the like operated by the service provider from any information processing terminal, entering information in a predetermined input form, and transmitting the information to the server 10 . The user registration control unit 1041 stores the received information in a new record of the user table 1012, and user registration is completed. As a result, the users stored in the user table 1012 can use the service.
Prior to registration of user information in the user table 1012 by the user registration control unit 1041, the service provider may perform a predetermined examination to limit whether or not the user can use the service.
The user ID may be any character string or number that can identify the user, any character string or number desired by the user, or any character string or number automatically set by the user registration control unit 1041. .

<Configuration of user terminal 20>
The user terminal 20 is an information processing device operated by a user who uses the service. The user terminal 20 may be, for example, a mobile terminal such as a smart phone or tablet, a stationary PC (Personal Computer), or a laptop PC. Moreover, it may be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch type terminal.
The user terminal 20 includes a storage unit 201 , a control unit 204 , an input device 206 and an output device 208 .

<Configuration of Storage Unit 201 of User Terminal 20>
The storage unit 201 of the user terminal 20 has a user ID 2011 and an application program 2012 .

A user ID 2011 is a user's account ID. The user transmits the user ID 2011 from the user terminal 20 to the server 10 . The server 10 identifies the user based on the user ID 2011 and provides the user with the service according to the present disclosure. The user ID 2011 includes information such as a session ID temporarily assigned by the server 10 to identify the user using the user terminal 20 .

The application program 2012 may be stored in the storage unit 201 in advance, or may be downloaded from a web server or the like operated by the service provider via the communication IF.
Application programs 2012 include applications such as web browser applications.
Application program 2012 includes an interpreted programming language such as JavaScript (registered trademark) that runs on a web browser application stored in user terminal 20 .

<Configuration of the control unit 204 of the user terminal 20>
The control unit 204 of the user terminal 20 has an input control unit 2041 and an output control unit 2042 . Control unit 204 implements each functional unit by executing application program 2012 stored in storage unit 201 .

<Configuration of Input Device 206 of User Terminal 20>
The input device 206 of the user terminal 20 has a camera 2061 , a microphone 2062 , a position information sensor 2063 , a motion sensor 2064 and a touch device 2065 .

<Configuration of the output device 208 of the user terminal 20>
The output device 208 of the user terminal 20 has a display 2081 and a speaker 2082 .

<Operation of System 1>
Each process of the system 1 will be described below.
FIG. 6 is a flowchart showing the operation of browsing processing.
FIG. 7 is a screen example of a content image displayed on the user terminal 20. As shown in FIG.

<Browsing process>
Browsing processing is processing in which the user browses the comics on the terminal.

<Overview of browsing process>
The browsing process is a series of processes of presenting a selectable list of contents to the user, accepting selection of contents from the user, presenting the selected contents to the user, and accepting an operation to browse the selected contents from the user. be.

<Browsing process details>
Details of the viewing process will be described below.

In step S<b>101 , the user executes the application program 2012 stored in the storage unit 201 of the user terminal 20 by operating the input device 206 of the user terminal 20 . The control unit 204 of the user terminal 20 transmits a request for a content list to the server 10 . Upon receiving the request, the browsing unit 1042 of the server 10 transmits content information including the content ID, content name, content attribute, and content data of the content table 1013 to the user terminal 20 .
The control unit 204 of the user terminal 20 displays the received content information on the display 2081 of the user terminal 20 as a list.

In step S102, the browsing unit 1042 of the server 10 executes a selection step of receiving content selection.
Specifically, by operating the input device 206 of the user terminal 20 , the user selects one or a plurality of pieces of content information that the user desires to view from the list of content information displayed on the display 2081 of the user terminal 20 . In addition, in the present disclosure, for the sake of simplicity, a configuration for selecting one predetermined content information is disclosed.
The control unit 204 of the user terminal 20 transmits the content ID corresponding to the content information selected by the user to the server 10 . The browsing unit 1042 of the server 10 receives and accepts the content ID selected by the user.

In step S103, the browsing unit 1042 of the server 10 executes a display step of displaying the content selected in the selection step on the display screen.
Specifically, the browsing unit 1042 of the server 10 searches the item of the content ID in the content table 1013 based on the received content ID, and acquires the content data. The browsing unit 1042 of the server 10 transmits the acquired content data to the user terminal 20 . The control unit 204 of the user terminal 20 displays the received content data on the display 2081 of the user terminal 20. FIG.

FIG. 7 is a screen example of a content image displayed on the user terminal 20. As shown in FIG. A screen D10 displayed on the display 2081 of the user terminal 20 includes a content display area D101, a dialog display area D102, and an operation area D103.
A part of the content data (content image) selected by the user in step S102 is displayed in the content display area D101. In the present disclosure, content data is manga, comics, etc., but content such as manga, comics, etc. has a large amount of content (large number of pages, etc.), so the content cannot be displayed in the content display area D101 at once. . Therefore, only a part of the content data is displayed in the content display area D101, and the user can read the content data by scrolling. In the present disclosure, the user can read the content data by scrolling it vertically on the user terminal 20 . Note that the present disclosure can also be applied to a configuration in which content is scrolled in the horizontal direction, or a configuration in which content can be scrolled in any other direction.
In the dialogue display area D102, the dialogue included in the content image displayed in the content display area D101 is displayed as text information.
When the user terminal 20 is a mobile terminal such as a smart phone, the physical size of the content display area D101 is small, and the characters of dialogue included in the content image are small, which may make it difficult for the user to read. Even in such a case, the user can read the dialogue displayed in the dialogue display area D102, so that the content can be browsed comfortably.
The text information of the dialogue displayed in the dialogue display area D102 is content data of a predetermined area before and after the content image displayed in the content display area D101, and is included in the content data not displayed in the content display area D101. You may also include textual information about the dialogue that will be displayed. The user can confirm the text information of the dialogue included in the content data that is not displayed in the content display area D101 before and after the content image. Content data can be read comfortably.
One or more user interface components such as buttons for operating the application program 2012 are displayed in the operation area D103. By operating the buttons or the like arranged in the operation area D103, the user changes the content data selected in step S102 to other content data or scrolls the content data displayed in the content display area D101. You can read on.
For example, the user touches the display 2081 of the user terminal 20 with a finger or a part of the body, uses the contact position as a reference position, and moves the contact part up and down or left and right from the reference position while the body is in contact. The content data may be scrolled in the direction of movement or in a direction opposite to the direction of movement. In this case, depending on the distance between the reference position and the contact portion, the scroll speed may be increased or decreased as the distance increases.

In step S104, the control unit 204 of the user terminal 20 executes a scroll step for scrolling the content displayed in the display step along the order of frames.
The control unit 204 of the user terminal 20 executes an operation step of receiving an operation from a user browsing content displayed on the computer. The scrolling step executes a step of scrolling the content along the order of frames according to the user's operation received in the operation step.
Specifically, the user can scroll the content data displayed in the content display area D101 by operating the input device 206 of the user terminal 20 . Alternatively, the user may scroll the content data displayed in the content display area D101 without operating the input device 206 of the user terminal 20. FIG.
The content data displayed in the content display area D101 is scrolled not by the control unit 204 of the user terminal 20 but by the browsing unit 1042 of the server 10. By transmitting the generated content image to the user terminal 20, Scrolling of content data on the display 2081 of the user terminal 20 may be realized.
In the present disclosure, the user terminal 20 can be scrolled vertically or horizontally. Note that the user terminal 20 may be scrollable in any direction.

In step S104, the operation steps include a first operation process in which the user touches the display screen and a second operation process in which the user leaves the display screen. The scrolling step includes a step of starting scrolling of the content along the order of the frames according to the first operation process, and a step of stopping the scrolling according to the second operation process.
Specifically, when the user touches the touch device 2065 provided over the display 2081 of the user terminal 20 with a part of the body such as a finger, the control unit 204 of the user terminal 20 starts scrolling the content data. do. While the user touches the touch device 2065 overlaid on the display 2081 of the user terminal 20, the content data is continuously scrolled. As a result, the user can comfortably view the content data while scrolling the content data without performing an input operation. At this time, the control unit 204 of the user terminal 20 executes scrolling according to the scroll parameter. When the user releases a part of the body such as a finger from the touch device 2065 provided over the display 2081 of the user terminal 20, the control unit 204 of the user terminal 20 stops scrolling the content data. As a result, the user can easily stop scrolling at the position of the content data that the user wants to browse over time.

In step S104, the browsing unit 1042 of the server 10 executes a calculation step of calculating a scroll parameter in which information about scrolling in the scrolling step is defined based on the content image of the content displayed on the display screen in the displaying step. The calculating step executes a step of calculating the scroll parameter by applying the content image displayed on the display screen as input data to the deep learning model.
The calculating step is a step of calculating scrolling parameters including a scrolling speed, a scrolling time, a stop time and a scrolling mode, wherein the scrolling step is based on the scrolling speed, the scrolling time, the stopping time and the scrolling mode calculated in the calculating step, Execute the step of scrolling the content along the order of the frames.
Specifically, the control unit 204 of the user terminal 20 transmits to the server 10 a request including part of the content data (content image) displayed in the content display area D101. The browsing unit 1042 of the server 10 calculates and acquires scroll parameters as output data by applying the received content image as input data to the learning model 1021 .
Note that the learning model 1021 may be stored in the user terminal 20 and the control unit 204 of the user terminal 20 may calculate and acquire scroll parameters without sending a request to the server 10 .
The input data to be applied to the learning model 1021 may include content data of predetermined regions before and after the content image. When the content image displayed in the content display area D101 is updated by scrolling, the scroll parameter is updated by calculating a new scroll parameter based on the updated content image.

In step S104, the calculating step performs a step of calculating scroll parameters based on user attributes related to attributes of users viewing content displayed on the computer. The calculating step performs a step of calculating scroll parameters based on user attributes including the user's gender, age, and favorite content genre.
Specifically, the browsing unit 1042 of the server 10 may include the user attribute of the user of the user terminal 20 in the input data applied to the learning model 1021 . As a result, the user can comfortably browse the content based on the scroll parameters calculated according to the user's attributes such as the user's age, sex, and favorite content genre.

In step S104, the calculating step executes a step of calculating a scroll parameter based on the content attributes regarding the content attributes received in the selecting step. The calculating step performs the step of calculating scroll parameters based on the content attributes received in the selecting step, including the genre of the content, the gender of the expected reader of the content, and the age group of the expected reader of the content.
Specifically, the browsing unit 1042 of the server 10 may include the content attribute of the content data to be displayed in the input data applied to the learning model 1021 . This allows the user to browse content comfortably based on scroll parameters calculated according to content attributes such as the genre of the content, the sex of the assumed reader of the content, and the age group of the assumed reader of the content. be able to.

In step S104, the calculating step executes a step of calculating the scroll parameter based on the frame feature quantity, which is the feature quantity relating to one or more frames included in the content image displayed on the display screen. The calculation step calculates scroll parameters based on frame feature quantities including frame division, images in frames, and the number of people in frames included in one or more frames included in the content image displayed on the display screen. Execute the step.
Specifically, the browsing unit 1042 of the server 10 includes, in the input data to be applied to the learning model 1021, the frame division, the image in the frame, and the number of persons in the frame included in one or more frames included in the content image. A frame feature amount may be included. Thereby, the user can browse the content comfortably based on the scroll parameter calculated according to the frame feature amount related to the frame included in the content image.

In step S104, the calculating step executes a step of calculating a scroll parameter based on a dialogue feature amount that is a feature amount relating to one or more dialogues included in the content image displayed on the display screen. The calculating step is a step of calculating the scroll parameter based on the dialogue feature amount including the dialogue content, the number of dialogues, the number of dialogue characters, and the dialogue image size included in one or more dialogues included in the content image displayed on the display screen. to run.
Specifically, the browsing unit 1042 of the server 10 selects the input data to be applied to the learning model 1021 and includes the contents of dialogue (text content of the dialogue), the number of dialogues, the number of dialogue characters, and A dialogue feature including the dialogue image size may be included. Thereby, the user can browse the content comfortably based on the scroll parameter calculated according to the dialogue feature amount related to the dialogue included in the content image.

In step S104, the browsing unit 1042 of the server 10 displays one or more lines included in the content image displayed on the display screen together with the content image on the display screen in a line area different from the content image. to run. The dialogue display step executes a step of arranging one or more dialogues included in the content image in a predetermined order and displaying them in the dialogue area.
Specifically, the control unit 204 of the user terminal 20 arranges a plurality of lines D1021, D1022, D1023, and D1024 included in the content image displayed in the content display area D101 as text information in order from the top. , is displayed in the dialogue display area D102 of the display 2081 of the user terminal 20. Multiple lines may be arranged in order from right or left.
A plurality of lines included in the content information may be stored and displayed as text information for each frame included in the content data in association with the frame. A plurality of dialogues included in the content information are recognized as text information based on the image data for each frame included in the content data and based on optical character recognition technology, etc., and associated with the frames. may be stored.
It should be noted that a plurality of lines may be displayed in the line area by arranging them in order according to the utterance order of the characters included in the content image. The order of a plurality of lines may be set and stored in advance by a content information provider or the like for each content data. The order of a plurality of lines may be automatically obtained and stored by applying content data to an AI model, deep learning model, or the like as input data.

In step S104, the dialogue display step includes a step of displaying one or more dialogues of the first character included in the content image near one end of the dialogue region, and displaying one or more dialogues of the second character included in the content image. and displaying the serifs close to the other end of the serif region.
Specifically, the control unit 204 of the user terminal 20 displays the dialogues D1021 and D1023 of the first character (CHRA) included in the content image so as to be close to the left end of the dialogue display area D102. On the other hand, the control unit 204 of the user terminal 20 displays the dialogue D1022 of the second character (CHRB) included in the content image, moving it to the right end of the dialogue display area D102. The words may be displayed near the upper end or the lower end of the words display area D102.
As a result, the dialogue for each character included in the content image is displayed in different areas of the dialogue area, so that the user can intuitively read the dialogue for each character.

In step S104, the dialogue display step includes a step of displaying one or more dialogues of the main character included in the content image near one end of the dialogue region, and a step of displaying one or more dialogues of other characters contained in the content image. is displayed close to the other end of the serif region.
Specifically, when the first character (CHRA) is the main character, and the second character (CHRB) and the third character (CHRC) are characters other than the main character, the control unit 204 of the user terminal 20 controls the content The dialogues D1021 and D1023 of the first character (CHRA) included in the image are displayed so as to be close to the left end of the dialogue display area D102. On the other hand, the control unit 204 of the user terminal 20 displays the dialogues D1022 and D1024 of the other characters (CHRB, CHRC) included in the content image so as to be close to the right end of the dialogue display area D102.
As a result, the dialogue of the main character included in the content image is displayed at one end of the dialogue area, and the dialogue of the other characters is displayed at the other end, so that the user can intuitively read the dialogue of each character. Whether or not the character is the main character may be preset by the content provider and stored in association with the character information. Whether or not the character is the main character may be determined automatically by applying content data as input data to an AI model, deep learning model, or the like, and stored in association with character information. In one piece of content data, one or more characters may be set as the main character. In one piece of content data, one or more characters may be set as the main character for each chapter, section, chapter, or the like.

In step S104, the dialogue display step executes a step of displaying one or more dialogues of the first character included in the content image in a display mode different from the one or more dialogues of the second character included in the content image. .
Specifically, the control unit 204 of the user terminal 20 displays the lines D1021 and D1023 of the first character (CHRA) included in the content image in the first color. On the other hand, the control unit 204 of the user terminal 20 displays the dialogue D1022 of the second character (CHRB) included in the content image in a second color different from the first color. The control unit 204 of the user terminal 20 displays the dialogue D1024 of the third character (CHRC) included in the content image in a third color different from the first and second colors.
Note that the control unit 204 of the user terminal 20 may display the lines of the first character, the second character, and the third character in association with different shaped display frames, icons, and the like. Further, as shown in FIG. 7, by associating the lines D1021 and D1023 with the name CHRA of the first character, the name CHRB of the second character with the line D1022, and the line D1024 of the third character with the line D1024, You may display by a different display mode.

In step S104, the browsing unit 1042 of the server 10 executes a reading step of reading aloud one or more lines displayed in the line area.
In the reading step, one or more lines of a plurality of characters included in the content image are read aloud by different voices associated with each character.
Specifically, the control unit 204 of the user terminal 20 outputs the text information contained in the dialogues D1021, D1022, D1023, and D1024 displayed in the dialogue display area D102 as voice from the speaker 2082, and reads out the dialogue as voice. The read-out voice may be configured such that different voices are read out for each character associated with each line.
As a result, the user can listen to the dialogue read aloud by the voice associated with each character as the content is browsed. The user can comfortably browse the content.
The voices read aloud for each of the multiple lines are obtained by recording the voice of a person or the like in advance for each of the multiple lines included in the content information, storing it in association with the lines, and reproducing the recorded lines. may be read aloud by voice.

In step S104, the browsing unit 1042 of the server 10 executes an operation step of receiving an operation on one or more lines displayed in the line area in the line display step from the user who views the content displayed on the computer.
The scrolling step executes a step of scrolling the content along the order of frames so that the content image corresponding to one or more lines for which the user's operation is accepted in the operation step is displayed on the display screen.
Specifically, by operating the input device 206 of the user terminal 20, the user touches the lines D1021, D1022, D1023, and D1024 displayed in the line display area D102 with a finger or a part of the body to make a selection operation. I do. The control unit 204 of the user terminal 20 scrolls the content data to the position corresponding to the line for which the selection operation was performed.
For example, when the user performs a selection operation on the dialogue D1024, the content data is scrolled so that the dialogue in the content image corresponding to the dialogue D1024 is positioned at the top of the content display area D101. When the content data is scrolled, the dialog in the content image corresponding to the dialog D1021 moves out of the content display area D101, thereby erasing the dialog D1021 from the dialog display area D102. Then, when the content display area D101 includes new lines in the content image by scrolling the content data, the text information of the lines corresponding to the new lines is displayed in the lines display area D102. .
Thereby, the user can display a content image corresponding to the operated line on the display screen by performing an operation such as clicking or tapping on the line. The user can comfortably browse the content.

<Learning processing>
Learning processing of the learning model 1021 will be described below.

<Learning processing of learning model 1021>
The learning process of the learning model 1021 is a process of learning the learning parameters of the deep neural network included in the learning model 1021 by deep learning.

<Overview of Learning Processing of Learning Model 1021>
In the learning process of the learning model 1021, the content image, user attribute, content attribute, frame feature amount, and dialogue feature amount are input data (input vector), and the scroll parameter is output data (teacher data). This is the process of learning the learning parameters of the deep neural network included in the by deep learning.
Any one of the user attribute, the content attribute, the frame feature amount, and the dialog feature amount may be omitted from the input data of the learning model 1021 .

<Details of the learning process of the learning model 1021>
The learning unit 1051 of the server 10 uses content images, user attributes, content attributes, frame feature amounts, dialog feature amounts, etc. as input data (input vectors), and scroll parameters as output data (teacher data). create. For example, learning data is constructed by conducting questionnaire surveys in advance regarding various content images, various scroll parameters, and various users' comfort levels regarding content browsing.
The learning unit 1051 of the server 10 creates data sets such as training data, test data, and verification data for learning the deep neural network of the learning model 1021 based on the learning data.
The learning unit 1051 of the server 10 learns the learning parameters of the deep neural network included in the learning model 1021 by deep learning based on the created data set. Specifically, for each content image, the learning parameters of the deep neural network are learned such that the scroll parameters are inferred so as to maximize the user's comfort in browsing the content.

<Basic computer hardware configuration>
FIG. 8 is a block diagram showing the basic hardware configuration of the computer 90. As shown in FIG. The computer 90 includes at least a processor 901, a main storage device 902, an auxiliary storage device 903, and a communication IF 991 (interface). These are electrically connected to each other by a communication bus 921 .

The processor 901 is hardware for executing an instruction set described in a program. The processor 901 is composed of an arithmetic unit, registers, peripheral circuits, and the like.

The main storage device 902 is for temporarily storing programs and data processed by the programs. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

Auxiliary storage device 903 is a storage device for storing data and programs. For example, flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, and the like.

The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using a wired or wireless communication standard.
The network is composed of various mobile communication systems constructed by the Internet, LAN, wireless base stations, and the like. For example, networks include 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), wireless networks (for example, Wi-Fi (registered trademark)) that can be connected to the Internet through predetermined access points, and the like. When connecting wirelessly, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), Bluetooth (registered trademark), and the like. In the case of wired connection, the network includes direct connection using a USB (Universal Serial Bus) cable or the like.

It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration to a plurality of computers 90 and connecting them to each other via a network. Thus, the computer 90 is a concept that includes not only the computer 90 housed in a single housing or case, but also a virtualized computer system.

<Basic Functional Configuration of Computer 90>
A functional configuration of the computer realized by the basic hardware configuration of the computer 90 (FIG. 8) will be described. The computer includes at least functional units of a control section, a storage section, and a communication section.

Note that the functional units included in the computer 90 can also be implemented by distributing all or part of each functional unit to a plurality of computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

The control unit is implemented by the processor 901 reading out various programs stored in the auxiliary storage device 903, developing them in the main storage device 902, and executing processing according to the programs. The control unit can implement functional units that perform various information processing according to the type of program. Thereby, the computer is implemented as an information processing device that performs information processing.

A storage unit is realized by the main storage device 902 and the auxiliary storage device 903 . The storage unit stores data, various programs, and various databases. Also, the processor 901 can secure a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 according to a program. In addition, the control unit can cause the processor 901 to execute addition, update, and deletion processing of data stored in the storage unit according to various programs.

A database refers to a relational database, and is used to manage tabular tables structurally defined by rows and columns, and data sets called masters in association with each other. In a database, a table is called a table, a master is called a column, and a row is called a record. In a relational database, relationships between tables and masters can be set and associated.
Normally, each table and each master has a primary key column for uniquely identifying a record, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, and update records in specific tables and masters stored in the storage unit according to various programs.

Note that the database and master in the present disclosure may include any data structure (list, dictionary, associative array, object, etc.) in which information is structurally defined. The data structure also includes data that can be regarded as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

A communication unit is implemented by the communication IF 991 . The communication unit implements a function of communicating with another computer 90 via a network. The communication section can receive information transmitted from another computer 90 and input it to the control section. The control unit can cause the processor 901 to execute information processing on the received information according to various programs. Also, the communication section can transmit information output from the control section to another computer 90 .

<Appendix>
The items described in the above embodiments will be added below.

(Appendix 1)
A program comprising a processor, a storage unit, and a display screen, and for causing a computer to display content divided into frames and in which the sequence of frames to be read is determined, wherein the program instructs the processor to select content. a selection step (S102) for receiving; a display step (S103) for displaying the content selected in the selection step on the display screen; and a scroll step (S104) for scrolling the content displayed in the display step in the order of frames. ), an operation step (S104) of receiving an operation from a user viewing content displayed on a computer, and a calculation step (S104) of calculating a scroll parameter for scrolling in the scroll step, and a scroll step (S104). is a step of scrolling the content along the order of frames based on the scroll parameter calculated in the calculation step in response to the user's operation received in the operation step.
Thereby, the content can be scrolled according to the operation by the user. The user can comfortably browse the content.

(Appendix 2)
The calculating step (S104) is a step of calculating scrolling parameters including the scrolling speed, the scrolling time, the stopping time and the scrolling mode, and the scrolling step includes calculating the scrolling speed, the scrolling time, the stopping time and the scrolling mode calculated in the calculating step. The program according to appendix 1, which is a step of scrolling the content along the order of frames based on.
This allows the user to comfortably browse the content.

(Appendix 3)
3. The program according to appendix 1 or 2, wherein the calculating step (S104) is a step of calculating the scroll parameter based on the content image of the content displayed on the display screen in the displaying step.
This allows the user to comfortably browse the content.

(Appendix 4)
The program according to appendix 3, wherein the calculating step (S104) is a step of calculating the scroll parameter by applying the content image displayed on the display screen as input data to a deep learning model.
This allows the user to comfortably browse the content.

(Appendix 5)
5. The program according to any one of Appendices 1 to 4, wherein the calculating step (S104) is a step of calculating the scroll parameter based on user attributes related to attributes of a user viewing content displayed on the computer.
As a result, the user can comfortably browse the content based on the scroll parameters that take into account his/her own user attributes.

(Appendix 6)
6. The program according to appendix 5, wherein the calculating step (S104) is a step of calculating the scroll parameter based on user attributes including the user's gender, age, and favorite content genre.
As a result, the user can comfortably browse the content based on the scroll parameters that take into account his/her own user attributes.

(Appendix 7)
7. The program according to any one of appendices 1 to 6, wherein the calculating step (S104) is a step of calculating the scroll parameter based on the content attributes regarding the content attributes received in the selecting step.
As a result, the user can comfortably browse the content based on the scroll parameters that consider the content attribute of the selected content.

(Appendix 8)
The calculation step (S104) is a step of calculating scroll parameters based on the content attributes received in the selection step, including the genre of the content, the gender of the assumed reader of the content, and the age group of the assumed reader of the content. The program according to Appendix 7.
As a result, the user can comfortably browse the content based on the scroll parameters that consider the content attribute of the selected content.

(Appendix 9)
9. Any one of Appendices 1 to 8, wherein the calculating step (S104) is a step of calculating a scroll parameter based on a frame feature quantity which is a feature quantity relating to one or more frames included in the content image displayed on the display screen. program as described.
As a result, the user can browse the content comfortably based on the scroll parameters that take into consideration the frame feature amount of the frame of the selected content.

(Appendix 10)
In the calculation step (S104), scroll parameters are calculated based on frame feature amounts including frame division, images in frames, and the number of people in frames, which are included in one or more frames included in the content image displayed on the display screen. 9. The program according to Appendix 9, which is the step of calculating
As a result, the user can browse the content comfortably based on the scroll parameters that take into consideration the frame feature amount of the frame of the selected content.

(Appendix 11)
11. Any one of Appendices 1 to 10, wherein the calculating step (S104) is a step of calculating a scroll parameter based on a dialogue feature amount that is a feature amount relating to one or more dialogues included in the content image displayed on the display screen. program as described.
As a result, the user can browse the content comfortably based on the scroll parameter that takes into account the dialogue feature amount related to the dialogue of the selected content.

(Appendix 12)
In the calculating step (S104), the scroll parameter is calculated based on the dialogue feature amount including the dialogue contents, the number of dialogues, the number of dialogue characters, and the dialogue image size included in one or more dialogues included in the content image displayed on the display screen. 12. The program according to Appendix 11, which is the calculating step.
As a result, the user can browse the content comfortably based on the scroll parameter that takes into account the dialogue feature amount related to the dialogue of the selected content.

(Appendix 13)
The operation step (S104) includes a first operation process in which the user touches the display screen, and a second operation process in which the user leaves the display screen. 13. The program according to any one of appendices 1 to 12, comprising a step of starting scrolling along the order of frames, and a step of stopping scrolling in response to the second operation processing.
Thereby, the content can be scrolled according to the operation by the user. The user can comfortably browse the content.

(Appendix 14)
The program causes the processor to display one or more lines included in the content image displayed on the display screen together with the content image in a line area different from the content image included in the display screen (S104); 14. The program according to any one of Appendixes 1 to 13, comprising:
As a result, the user can comfortably browse the content by confirming the sentences displayed in the dialogue area of the dialogue included in the content image.

(Appendix 15)
15. The program according to appendix 14, wherein the dialogue display step (S104) is a step of arranging one or more dialogues included in the content image in a predetermined order and displaying them in a dialogue area.
As a result, the user can comfortably browse the content by confirming the sentences displayed in the dialogue area of the dialogue included in the content image.

(Appendix 16)
The dialogue display step (S104) includes displaying one or more dialogues of the first character included in the content image so as to be close to one end of the dialogue area, and displaying one or more dialogues of the second character contained in the content image. 16. The program according to appendix 14 or 15, comprising the step of displaying near the other end of the serif region.
As a result, the dialogue for each character included in the content image is displayed in different areas of the dialogue area, so that the user can intuitively read the dialogue for each character.

(Appendix 17)
The dialogue display step (S104) is a step of displaying one or more dialogues of the first character included in the content image in a display mode different from the one or more dialogues of the second character contained in the content image. 16. The program according to 16.
As a result, the lines of each character included in the content image are displayed in different display modes (with colors, shapes, and character names) in the lines area, so that the user can intuitively read the lines of each character. be able to.

(Appendix 18)
The dialogue display step (S104) includes a step of displaying one or more dialogues of the main character included in the content image near one end of the dialogue area, and displaying one or more dialogues of other characters contained in the content image. 18. The program according to any one of Appendices 14 to 17, comprising the steps of: .
As a result, the dialogue of the main character included in the content image is displayed at one end of the dialogue area, and the dialogue of the other characters is displayed at the other end, so that the user can intuitively read the dialogue of each character.

(Appendix 19)
The program causes the processor to execute a read-aloud step (S104) of reading one or more lines displayed in the dialogue area by voice, and the reading-aloud step (S104) is performed by reading a plurality of characters included in the content image. 19. The program according to any one of appendices 14 to 18, wherein the one or more lines are read aloud by different voices associated with each character.
As a result, the user can listen to the dialogue read aloud by the voice associated with each character as the content is browsed. The user can comfortably browse the content.

(Appendix 20)
The program causes the processor to perform an operation step (S104) of receiving an operation on one or more lines displayed in the line area in the line display step from a user browsing the content displayed on the computer, and a scrolling step. (S104) is a step of scrolling the content along the order of the frames so that the content image corresponding to one or more lines for which the user's operation is accepted in the operation step is displayed on the display screen. 20. The program according to any one of 14-19.
Thereby, the user can display a content image corresponding to the operated line on the display screen by performing an operation such as clicking or tapping on the line. The user can comfortably browse the content.

(Appendix 21)
An information processing apparatus comprising a processor and a storage unit, wherein the processor executes the program according to any one of Appendixes 1 to 20.
This allows the user to comfortably browse the content.

(Appendix 22)
An information processing system including an information processing device comprising a processor and a storage unit, wherein the processor executes the program according to any one of appendices 1 to 20.
This allows the user to comfortably browse the content.

(Appendix 23)
An information processing method executed by a computer comprising a processor and a storage unit, the information processing method comprising causing the computer to execute the program according to any one of Appendices 1 to 20.
This allows the user to comfortably browse the content.

1 system, 10 server, 101 storage unit, 104 control unit, 106 input device, 108 output device, 20 user terminal, 201 storage unit, 204 control unit, 206 input device, 208 output device

Claims (22)

A program for displaying, on a computer, content comprising a processor, a storage unit, and a display screen, and in which the order of the frames to be read is determined and the content is divided into frames,
The program causes the processor to:
a selection step of receiving a selection of said content;
a display step of displaying the content selected in the selection step on the display screen;
a scrolling step of scrolling the content displayed in the displaying step in the order of the frames;
an operation step of receiving an operation from a user viewing the content displayed on the computer;
a dialog display step of displaying one or more lines included in the content image displayed on the display screen, together with the content image, in a dialog region different from the content image included in the display screen;
an operation step of receiving an operation on the one or more lines displayed in the line area in the line display step from a user browsing the content displayed on the computer;
and
The scrolling step scrolls the content in the order of the frames so that the content image corresponding to the one or more lines for which the operation from the user is received in the operation step is displayed on the display screen. is the step to scroll,
program. The program causes the processor to:
a calculating step of calculating a scroll parameter for said scrolling in said scrolling step;
and
The scroll step is a step of scrolling the content according to the order of the frames based on the scroll parameter calculated in the calculation step in response to the operation from the user received in the operation step.
A program according to claim 1. the calculating step is calculating the scroll parameters including scroll speed, scroll time and pause time;
The scrolling step is a step of scrolling the content along the order of the frames based on the scrolling speed, the scrolling time, and the stop time calculated in the calculating step.
3. A program according to claim 2 . The calculating step is a step of calculating the scroll parameter based on the content image of the content displayed on the display screen in the displaying step.
3. A program according to claim 2 . The calculating step is a step of calculating the scroll parameter based on user attributes related to attributes of a user viewing the content displayed on the computer.
3. A program according to claim 2 . The calculating step is a step of calculating the scroll parameter based on the user attributes including the user's gender, age, and favorite content genre.
6. A program according to claim 5 . The calculating step is a step of calculating the scroll parameter based on the content attribute regarding the attribute of the content received in the selecting step.
3. A program according to claim 2 . The calculating step calculates the scroll parameter based on the content attributes received in the selecting step, including the genre of the content, the gender of the expected reader of the content, and the age group of the expected reader of the content. is the step
8. A program according to claim 7 . The calculating step is a step of calculating the scroll parameter based on a frame feature quantity, which is a feature quantity relating to one or more frames included in the content image displayed on the display screen.
3. A program according to claim 2 . The calculating step is based on the frame feature amount including the frame division, images in the frame, and the number of people in the frame included in one or more frames included in the content image displayed on the display screen. calculating the scroll parameters,
10. A program according to claim 9 . The calculating step is a step of calculating the scroll parameter based on a dialogue feature amount that is a feature amount relating to one or more dialogues included in the content image displayed on the display screen.
3. A program according to claim 2 . In the calculating step, the scrolling is performed based on the dialogue feature amount including the dialogue content, the number of dialogues, the number of dialogue characters, and the dialogue image size included in one or more dialogues included in the content image displayed on the display screen. A step of calculating parameters,
12. A program according to claim 11 . The operating step includes:
a first operation process in which the user touches the display screen;
a second operation process in which the user leaves the display screen;
including
The scrolling step includes:
a step of starting to scroll the content along the order of the frames in accordance with a first operation process;
stopping the scrolling in response to a second operation process;
including,
3. A program according to claim 1 or 2 . The program causes the processor to:
a dialog display step of displaying one or more lines included in the content image displayed on the display screen, together with the content image, in a dialog region different from the content image included in the display screen;
including,
3. A program according to claim 1 or 2 . The dialogue display step is a step of arranging one or more dialogues included in the content image in a predetermined order and displaying them in the dialogue area.
15. A program according to claim 14 . The dialogue display step includes:
displaying the one or more lines of the first character included in the content image so as to be close to one end of the line area;
a step of displaying the one or more lines of the second character included in the content image so as to be close to the other end of the line area;
including,
15. A program according to claim 14 . The dialogue display step displays the one or more dialogues of the first character included in the content image in a display mode different from the one or more dialogues of the second character included in the content image. is
17. A program according to claim 16 . The dialogue display step includes:
a step of displaying the one or more lines of the main character included in the content image so as to be close to one end of the line area;
a step of displaying the one or more lines of another character included in the content image so as to be close to the other end of the line area;
including,
15. A program according to claim 14 . The program causes the processor to:
a reading step of reading aloud the one or more lines displayed in the line area;
and
The reading step is a step of reading out the one or more lines of a plurality of characters included in the content image with different voices associated with each character.
15. A program according to claim 14. An information processing device comprising a processor and a storage unit,
The processor executes the program according to claim 1 or 2 ,
Information processing equipment. An information processing system including an information processing device comprising a processor and a storage unit,
The processor executes the program according to claim 1 or 2 ,
Information processing system. An information processing method executed by a computer comprising a processor and a storage unit,
causing the computer to execute the program according to claim 1 or 2 ,
Information processing methods.

Images