JP2018158046A - Operation control system, and character screening system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a character operation system or the like that can simplify a player's operation, can improve an existence feeling of a character and a life feeling in a performance of the character, and can execute an event for which the audience can have a high sense of satisfaction.SOLUTION: A system control device 20 is configured to present an operation command selected according to audience information for showing information regarding the audience as an operation command candidate from among a plurality of operation commands for controlling an operation of a game character to be imaged during screening of content, to identify the presented operation command candidate by the operator's operation instruction, to receive the identified operation command candidate as the operation command, and to control an operation of a performance character to be displayed, based on the received operation command.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an operation control system, a character screening system, a program, and the like.

  Conventionally, in an event venue that accommodates a plurality of spectators, a non-existent two-dimensional or three-dimensional character such as an idol is screened for the spectators and the character is operated as if it exists ( Hereinafter, it is also referred to as “event character operation system”).

  On the other hand, in a moving image editing apparatus that edits CG animation and moving images, an apparatus that eliminates complicated image editing by an operator is known (for example, Patent Document 1). Then, by incorporating such a moving image editing device into the event character operation system, it is considered to provide a moving image including a character to be screened in real time according to the situation of the audience and the venue. Yes.

JP 2001-195600 A

  However, even if the video editing device described in Patent Document 1 is simply incorporated into the event character operation system, the operation of the operator can be simplified, but depending on the atmosphere of the venue and the state of the audience, It is difficult to control the operation.

  The present invention has been made to solve the above-described problems, and its purpose is to make an image of a character by moving it according to the atmosphere of the event venue and the state of the spectator while simplifying the operation by the operator. It is to provide a character operation system or the like that can enhance a sense of existence of a character and a live feeling in the performance of the character, excite an event, and execute an event with a high level of satisfaction for the audience.

(1) In order to solve the above problems, the present invention provides:
An operation control system for controlling operation of an imaged target character to be viewed by a spectator,
Image generating means for setting a virtual camera in the object space in which the target character is arranged, and generating an image of the target character in the object space with the virtual camera as a viewpoint;
Audience display control means for displaying an image of the target character on the audience display device visually recognized by the audience;
An operation command selected by the operator according to spectator information indicating information related to the spectator from among a plurality of operation commands for controlling the movement of the target character in the object space is set as an operation command candidate to the operator. Presenting means to present;
Receiving means for receiving an operation command instructed by the operator from among the presented operation command candidates;
Action control means for controlling the action of the displayed target character based on the accepted operation command;
It has the composition provided with.

  With this configuration, the present invention provides information indicating the audience atmosphere and the current state of the audience, or an operation command selected by the operator according to audience information of each audience such as audience attributes and billing history, or An operation command extracted according to the audience information and selected by the operator can be presented to the operator as an operation command candidate.

  For example, according to the present invention, when it is determined that the audience watching the target character such as an event venue is not excited based on the audience information, MC (line or microphone) is used as an action for the target character to increase the audience. The command of performing the performance) can be selected, or the operation command for performing the intense dance by the target character can be selected. The operation of the target character according to the state) can be selected by the operator.

  In addition, for example, the present invention controls the movement of the target character for women when the audience viewing the target character is determined by the audience information, or predetermined goods in the past behavior Operation command to control the action of the target character in accordance with the behavior history and attributes of the spectator who is viewing by executing a special performance of the target character to the spectator. Can be selected by the operator.

  Therefore, the present invention makes it possible to move the target character in accordance with the audience atmosphere and the audience state while simplifying the operation by the operator. While improving the live feeling in performance, it is possible to enliven the audience and achieve a fresh performance.

  As a result, the present invention can control the action (ie, performance) of the target character that is highly satisfied by the audience.

  Note that the spectator display device is a device that is provided on a stage and displays a target character to be viewed for performing performances such as singing, dancing, theater, lectures, etc., for example, a liquid crystal display device, a projector, etc. Or a stereoscopic imaging device such as a hologram.

  The “operation command selected according to audience information” is, for example, an operation command that can be selected according to the operator's intention while referring to audience information, and, for example, a camera The operation command selected (extracted) according to the audience information detected by the detection means such as the operation command selected by the operator.

For example, audience information includes
(1) Audience attribute information such as gender, age, residential area or position (seat) at the time of viewing (2) Audience preference information such as favorite acting characters,
(3) Information of terminal devices held by the audience such as smartphones and ID tags (for example, information indicating the position (seat) at the time of viewing such as information for identifying the audience and electronic ticket information)
(4) Information stored in association with each spectator such as an ID or in association with a terminal device held by the spectator, and the target character being viewed or the target character is performing Information (game play information) regarding play status and history (including billing history) of a predetermined game such as a game related to the event,
(5) Information stored using the terminal device held by the spectator, including action history information indicating participation in an event in which the target character or the target character is performing, and related to the event or performance Billing information and billing history information from the designated mail-order sites and retail stores (including event venues),
(6) Biological information (for example, facial images, voice information (including voice print information and frequency characteristic information indicating frequency characteristics) of each audience), posture information, heat information, and a predetermined device (smart phone, Information similar to biological information such as light information using psyllium)) is included.

  Further, the operation command may include a command for controlling the operation of the imaged target character in the object space for executing a plurality of commands integrally.

  For example, the plurality of operation commands may be a plurality of types of operation commands (jumping, running, one-turn operation commands, etc.) with different movements of the target character, or the same type having different effects when executing the operation commands. May be an operation command (such as an operation command that generates a different line).

  In addition to the above, “presenting operation command candidates” may be a case where an operation command is displayed on a tablet having a display device or touch panel function, or an operation command is assigned to a physical switch or keyboard. It may be a case where it is shown.

  “Target character motion control” includes not only controlling motion according to an operation command but also operating according to an operation command at a predetermined timing while operating according to a predetermined algorithm.

  In particular, when controlling the movement of the target character using a predetermined algorithm, it shows that the movement is controlled by an algorithm that defines performance (including words) during performance such as singing and dancing. When the operation command is received while the motion control is being performed, instead of processing based on the algorithm or together with the processing of the algorithm, motion control of the target character is included according to the operation command.

(2) The present invention also provides:
A monitor display control means for displaying the target character on a monitor display device visually recognized by the operator;
The presenting means is
The monitor display device is configured to present the operation command candidates together with the target character to the operator.

  With this configuration, according to the present invention, operation command candidates can be displayed on a character to be operated that can be visually recognized by the operator, so that the operation of the operator can be simplified.

  In particular, when the monitor display device is configured by a touch panel, the present invention can present operation command candidates together with the target character to be motion-controlled. Operation input can be simplified.

  The monitor display device indicates an operator monitor provided separately from the spectator display device, and includes, for example, a tablet terminal device, a smartphone, and a liquid crystal device. And it is preferable that the monitor display apparatus is comprised by the touchscreen as mentioned above.

(3) The present invention also provides:
The presenting means is
The spectator information is acquired, and the acquired spectator information is presented to the operator together with the operation command candidates.

  With this configuration, the present invention allows the operator to recognize spectator information when selecting operation command candidates, so that an operation command in which an action that matches the spectator with respect to the target character is defined by the operator. It is possible to make it easier to select.

  Therefore, the present invention can present the audience information such as the behavior history and attributes of the audience, excite the audience, and provide a fresh action, so the behavior of the target character with a high satisfaction in the audience It is possible to easily select an operation command for which (that is, performance) is defined.

(4) The present invention also provides:
Management means for managing storage means for storing personal information of each audience as the audience information for each audience,
The presenting means is
The stored audience information is acquired, and the acquired audience information is presented to the operator together with the operation command candidates.

  With this configuration, the present invention allows the operator to recognize the spectator's personal information when selecting the operation command candidate. It can be made easier to select by a person.

  For example, if it is recognized that there are many women in the audience watching the target character based on the presented personal information, the operator selects an operation command for controlling the movement of the target character for women. When it is recognized that there are spectators who have purchased certain goods in the past actions (there are many spectators), the operator performs a special presentation of the target character It becomes easy to select the operation command to be performed.

  In other words, the present invention can present the audience personal information such as the behavior history and attributes of the audience, excite the audience, and provide fresh behavior, so that the target character with high satisfaction in the audience It is possible to easily select an operation command for which the operation (that is, performance) is defined.

  The personal information includes, for example, behavior history information indicating past behavior history of each spectator, gender, age, residential area, favorite target character, position (seat) at the time of viewing, or smartphone. And attribute information such as information related to terminal devices such as ID tag (RFID).

  In particular, the information related to the terminal device includes, for example, information indicating the position (seat) at the time of viewing such as information for identifying the individual of the spectator and ticket information, and purchase history information such as game history information and products.

  In addition, the action history information includes, for example, the number of times of participation in related events such as events in which the performance of the target character is performed, the date and time, games related to the event in which the target character and the target character are performing, etc. Information on the play status or history of a given game, purchase information on goods (goods) or services from a given mail-order site or retail store (including the event venue) related to the event or target character, etc. Accounting information (including accounting history information) generated thereby is included.

(5) The present invention also provides:
It further comprises registration means for registering and accumulating information of operation commands accepted when the motion control of the target character is executed in the storage means as usage history information,
The presenting means is
According to the spectator information and the registered use history information, an operation command selected by the operator is presented to the operator as the operation command candidate.

  With this configuration, the present invention responds to usage history information such as, for example, usage frequency, the type of operation command used immediately before, and information indicating the presence / absence of usage within a predetermined period (such as during a series of performance periods). Thus, the operation command selected by the operator can be presented to the operator as an operation candidate command.

  For example, according to the present invention, when selecting an operation command, an operation command that is frequently used, an operation command related to the immediately preceding operation command, an operation command that defines an action of a target character that most of the audience has not yet viewed, Operation commands that are not yet used during execution of a series of performances, or operation commands that should be recognized by the operator as operation command candidates that can be preferentially selected compared to other operation commands, are presented to the operator. be able to.

  In addition, for example, the present invention does not present unselectable operation command candidates such as an operation command that has already been executed or an operation command that has executed a special presentation already provided to a specific audience in a series of performances. can do.

  Furthermore, for example, according to the present invention, if an already registered operation command is analyzed, an operation command that prescribes the action of the target character that always swells, or an operation command that prescribes the action of the target character that functions effectively in any scene, etc. In addition, an operation command in accordance with the tendency of past operation commands can be presented as an operation command candidate.

  Therefore, according to the present invention, it is possible to operate the character according to the atmosphere of the audience and the state of the audience while further simplifying the operation by the operator using the past usage history, and to the target character of each audience. Without detracting from the interest in viewing, it is possible to excite the audience and provide fresh behavior.

(6) The present invention also provides:
The usage history information includes spectator information used when the operation command and the operation command are received,
The presenting means is
Presenting an operation command selected by the operator as the operation command candidate to the operator according to a given analysis result in the operation command based on the usage history information and the audience information; It has a configuration.

  With this configuration, the present invention analyzes an operation command that prescribes the action of a target character that is always excited by analyzing it with customer information using an operation command that has already been registered, or the action of a target character that functions effectively in any scene. An operation command that conforms to the tendency of past operation commands, such as an operation command that prescribes, can be presented as an operation command candidate.

(7) The present invention also provides:
It further comprises detection means for detecting the state of each audience as the audience information,
The presenting means is
An operation command selected by the operator according to the detected audience information is presented to the operator as an operation command candidate.

  With this configuration, the present invention detects, for example, the audience's reaction to the audience characteristics or the behavior of the audience target character such as the audience's face image, sound, emitted light, or information on the terminal device owned by the audience. Therefore, the characteristics of the spectator who is visiting the target character (for example, there are many spectators with a specific light or many women), or the state when the spectator is viewing the target character. It can be detected as audience information.

  Therefore, according to the present invention, it is possible to narrow down and present operation command candidates according to whether or not the place where the target character is viewed based on the audience information is exciting, or according to the state and attributes of the audience. It is possible to improve the sense of existence of the character to be imaged and the live feeling in the performance of the character, to increase the audience, and to perform a performance with high satisfaction for the audience while simplifying the operation in the above.

  In addition, if the detected spectator information is accumulated, the present invention creates a database of the motion of the target character and the state of the spectator, and uses it for subsequent motions in the target character (ie, the same performance at different times and different locations). can do.

  Therefore, according to the present invention, if a large amount of spectator information can be stored in a database, the audience can be further increased and a performance with a high degree of satisfaction can be performed by the spectator.

  The detection means includes, for example, a camera or infrared sensor for detecting the light emitted by a device such as recognition of the audience's face or a device such as a psyllium or a smartphone, a microphone for collecting the sound of the audience, the entire audience Thermography that detects the amount of heat of each person, seating sensor that detects the posture of each spectator (sitting or standing, etc.), a detection unit comprising a marker unit and a camera, or information on a terminal device owned by the spectator ( For example, a wireless receiver that detects a viewing (seat) position of each spectator and information for specifying an individual) is included.

  And as spectator information indicating the state of each spectator, for example, spectator's face information, ecological information such as sound information (frequency information and volume information), or spectator state such as calorie information, posture information and light information, etc. Information similar to recognizable biological information is included.

(8) The present invention also provides:
The accepting means is
Along with the operation command, accept a spectator designation instruction for designating a specific spectator who enjoys the motion of the target character,
The operation control means is
It has the structure which controls the operation | movement of the said target character displayed toward the audience specified by the received audience designation | designated instruction | indication.

  With this configuration, the present invention can control the movement of the target character such as a special effect toward the specific audience at the viewing position of the specific audience. The live feeling in the performance of the character can be improved, the audience can be enlivened, and the audience can perform a performance with a high degree of satisfaction.

Note that the motion of the target character enjoyed by a specific audience includes throwing kisses, hand gestures, name and feature lines for the specific audience, and the like.
In addition, for example, the designation of viewing may be performed using a display device that displays the position in the venue of each viewing, using an operator such as a touch panel or a joystick, or data on the face image of the audience is stored. It is also possible to automatically specify using a detection means such as a camera on the assumption that it is.

(9) The present invention also provides:
The presenting means is
An operation command determined according to the movement state of the target character is included in the operation command candidates and presented to the operator.

  With this configuration, for example, the present invention always operates at a predetermined timing such as an operation command that is always expected to be used, such as the first action or the last action of the target character, or the singing of a specific song according to the past use history. Since the operation command to be used can be included in the operation command candidate and presented, a suitable operation command can be extracted as the operation command candidate.

  The “motion state of the target character” indicates a state in which each action of the target character, such as performance start timing and end timing, is started or is being executed.

(10) The present invention also provides:
The presenting means is
The presentation according to the elapsed time from the start timing of image generation in the target character, the number of times operation commands received during the image generation, or a combination of the types of operation commands received during the image generation The type of operation command to be changed is changed.

  With this configuration, the present invention, for example, during the movement of the target character by the algorithm for controlling a series of movements of the target character performing performances such as singing, dancing, or theater or lecture, It is possible to present an operation command suitable for the operator.

For example, the present invention
(1) At the start timing of image generation (that is, content screening) in the target character, an operation command for controlling an action for raising the audience is presented,
(2) At the second half of the screening of the content, present an operation command for controlling the action to maintain and maintain the exciting audience.
(3) For quiet scenes such as between songs, present command commands specific to that scene, such as “Serif”.
be able to.

  Therefore, the present invention can improve the sense of existence of the character to be imaged and the live feeling of the performance of the character, excite the audience, and perform performance with high satisfaction for the audience.

(11) The present invention also provides:
While following a given algorithm and imaging the motion of the target character placed in the object space,
For each operation command, a reception completion allowable time indicating a time at which reception of the operation command is permitted from the start timing of image generation in the target character or a plurality of given sections are determined, and the image generation is executed. Section information indicating a section in which the operation command is allowed to be received and an execution time during which the target character's motion control is executed based on each operation command are defined,
The presenting means is
The type of the operation command to be presented is determined according to the elapsed time from the image generation start timing in the target character or the section to which the operation command is presented and the execution time of each operation command. Or it has the structure which presents the operation command which changed the content of the motion control of the said target character.

  With this configuration, the elapsed time from the start timing of image generation, or, for example, if the operation control is a live performance in music, the first act or A melody, B melody or chorus, It is possible to identify which part of each part (section), such as Act 2, and to present the operation command, and depending on the situation, the operation control of the target character in the operation command such as shortening the time or the action content The contents can also be changed.

  Therefore, the present invention is suitable as a motion of the target character at that time during the motion of the target character by an algorithm for controlling a series of motions of the target character performing performances such as singing, dancing, theater or lecture. The operation command can be presented to the operator.

(12) The present invention also provides:
The operation control means is
When the operation command is received during the image generation of the target character, the movement control of the displayed target character based on the received operation command is started from a predetermined timing. Yes.

  With this configuration, the present invention can control the movement of the target character in the operation command while waiting for the movement timing of the movement of the target character during image generation (that is, during the content screening) of the target character. On the other hand, seamless operation control without a sense of incongruity can be performed.

(13) The present invention also provides:
The presenting means is
While presenting the operation command candidate, the operation start time at which the motion control of the target character is started or the time related to the operation start time is presented based on the operation command received at the timing.

  With this configuration, the present invention can prevent the operator from forgetting to input and can improve the operability of the operator.

  The “time related to the operation start time” includes, for example, the remaining time from the current timing to the operation start time.

(14) The present invention also provides:
The operation control means is
While following a given algorithm, when the motion of the target character placed in the object space is imaged, and the operation command is received, the algorithm after receiving the operation command, It changes according to the received operation command, and has the structure which controls the operation | movement of the said target character displayed according to the said changed algorithm.

  With this configuration, when the operation command is received, the present invention controls the movement of the target character accordingly, so that it is possible to provide an image of the target character that is controlled by the operator and every time the operation command is received. In addition, the movement of the target character can be changed.

  Further, the present invention can change the algorithm for controlling the motion of the target character even when the performance time of the target character is changed by the motion control by the operation command. You can develop content stories such as performance without losing.

  Therefore, the present invention makes it possible to move the target character in accordance with the audience atmosphere and the audience state while simplifying the operation by the operator. While improving the live feeling in performance, it is possible to enliven the audience and achieve a fresh performance.

(15) Further, in order to solve the above problems, the present invention provides:
A program for operating and controlling the movement of an imaged target character to be viewed by a spectator,
Image generating means for setting a virtual camera in the object space in which the target character is arranged, and generating an image of the target character in the object space with the virtual camera as a viewpoint;
Audience display control means for displaying an image of the target character on the audience display device visually recognized by the audience;
An operation command selected by the operator according to spectator information indicating information related to the spectator from among a plurality of operation commands for controlling the movement of the target character in the object space is set as an operation command candidate to the operator. Presenting means to present;
An accepting means for accepting an operation command instructed by the operator from among the presented operation command candidates; and
Action control means for controlling the action of the displayed target character based on the accepted operation command;
As a computer.

  With this configuration, the present invention provides information indicating the audience atmosphere and the current state of the audience, or an operation command selected by the operator according to audience information of each audience such as audience attributes and billing history, or An operation command extracted according to the audience information and selected by the operator can be presented to the operator as an operation command candidate.

  Therefore, the present invention makes it possible to move the target character in accordance with the audience atmosphere and the audience state while simplifying the operation by the operator. While improving the live feeling in performance, it is possible to enliven the audience and achieve a fresh performance.

  As a result, the present invention can control the action (ie, performance) of the target character that is highly satisfied by the audience.

(16) Further, in order to solve the above problems, the present invention provides:
An operation control system for operating and controlling the motion of the imaged target character to be viewed by the audience;
An audience display device visually recognized by the audience;
With
The operation control system is
Image generating means for setting a virtual camera in the object space in which the target character is arranged, and generating an image of the target character in the object space with the virtual camera as a viewpoint;
Spectator display control means for displaying an image of the target character on the spectator display device;
An operation command selected by the operator according to spectator information indicating information related to the spectator from among a plurality of operation commands for controlling the movement of the target character in the object space is set as an operation command candidate to the operator. Presenting means to present;
Receiving means for receiving an operation command instructed by the operator from among the presented operation command candidates;
Action control means for controlling the action of the displayed target character based on the accepted operation command;
It has the composition provided with.

  With this configuration, the present invention provides information indicating the audience atmosphere and the current state of the audience, or an operation command selected by the operator according to audience information of each audience such as audience attributes and billing history, or An operation command extracted according to the audience information and selected by the operator can be presented to the operator as an operation command candidate.

  Therefore, the present invention makes it possible to move the target character in accordance with the audience atmosphere and the audience state while simplifying the operation by the operator. While improving the live feeling in performance, it is possible to enliven the audience and achieve a fresh performance.

  As a result, the present invention can control the action (ie, performance) of the target character that is highly satisfied by the audience.

It is a block diagram which shows the schematic structure in one Embodiment of the content screening system which concerns on this invention. It is a figure for demonstrating the relationship between the content screening system and large sized display apparatus in the event venue of one Embodiment, and the relationship between an acting character and a spectator. It is an example of the functional block diagram which shows the structure of the server apparatus in one Embodiment. It is an example of the functional block diagram which shows the structure of the system control apparatus in one Embodiment. It is FIG. (1) for demonstrating the operation command candidate presentation process of the content of this embodiment. It is FIG. (2) for demonstrating the operation command candidate presentation process of the content of one embodiment. It is FIG. (3) for demonstrating the operation command candidate presentation process of the content of this embodiment. FIG. 10 is a diagram (No. 1) for describing an operation control process based on a content operation command according to an embodiment; FIG. 10 is a diagram (No. 1) for describing an operation control process based on a content operation command according to an embodiment; It is a flowchart (the 1) which shows operation | movement of the operation control process based on the operation command containing the operation command candidate presentation process performed by the system control apparatus of one Embodiment. It is a flowchart (the 1) which shows operation | movement of the operation control process based on the operation command containing the operation command candidate presentation process performed by the system control apparatus of one Embodiment.

  Hereinafter, embodiments will be described. The embodiments described below do not unduly limit the contents of the present invention described in the claims. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention. Further, in the following embodiment, an image in an object space in which a character to be viewed (hereinafter also referred to as “target character”) is arranged for a spectator accommodated in a given venue in real space. Is an embodiment in which the operation control system, the content screening system, and the program according to the present invention are applied to a content screening system for screening as content.

[1] Content Screening System First, the outline and schematic configuration of the content screening system 10 of the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of a system configuration showing the configuration of a network system including the content screening system 10 of the present embodiment, and FIG. 2 shows the relationship between the content screening system 10 and the large display device 12 at the event venue. It is a figure for demonstrating the relationship between the acting character which is an example of an object character, and a spectator.

  The content screening system 10 according to the present embodiment has, for example, a plurality of audiences accommodated in a given venue (hereinafter referred to as “screening venue”) in a real space where content screening such as an event venue is performed. The contents of the group's acting performance (specifically, singing live) that is made up of idols (so-called characters acting in the object space (hereinafter referred to as “acting characters”)) are screened for viewing It is a screening system.

  Then, the content screening system 10 of the present embodiment generates a moving image of the performance while controlling the motion of the performance character based on the algorithm defined in the content and the instruction of the operator. The present invention has a configuration that enhances the audience and provides a fresh performance as a moving image while improving the actual feeling of the acting character and the live feeling of the performance of the acting character.

  Specifically, as shown in FIG. 1, the content screening system 10 is connected to a server device 11 via a network such as the Internet, and the server device 11 uses the other content screening systems (10A, 10B, 10C). ).

  As shown in FIGS. 1 and 2, the content screening system 10 is installed in a given venue (for example, an event venue) in real space, and has a large display device 12 that displays the content so that the audience 3 can see it. The system controller 20 receives various operations of the operator 5 and causes the large display device 12 to display content.

  The large display device 12 is installed on a stage 2 provided in a venue where content such as an event venue is screened, for example, and images an object space including a performance character 8 that performs performances such as singing and dancing or performing or giving a lecture. It is a device to do.

  The large display device 12 includes a large screen formed on the stage 2 and a projector that projects an image on the screen (for example, from the back).

  In addition, the large display device 12 shows live performances of acting characters 8 such as singing music, playing musical instruments, dancing, street performances, and the like as contents.

  The large display device 12 may be constituted by a large-sized liquid crystal display device or a stereoscopic imaging device such as a hologram.

  The system control device 20 is configured by an information processing device such as a tablet information terminal device, a personal computer, a smartphone, or a game machine, and can be connected to the server device 11 via a network such as the Internet (WAN) or a LAN. Device. The communication line between the system control device 20 and the server device 11 may be wired or wireless.

In addition, the system control device 20
(1) A virtual camera is set in the object space where the acting character is arranged, and an image of the object space with the virtual camera as a viewpoint is generated.
(2) Display an image of the object space centered on the acting character on a large display device visually recognized by the audience;
(3) An operation command selected from among a plurality of operation commands for controlling the motion of the acting character to be imaged during content screening is selected according to spectator information indicating information about the spectator as an operation command candidate. , Present to the operator,
(4) The presented operation command candidate is specified by the operation instruction of the operator, the specified operation command candidate is received as an operation command,
(5) controlling the action of the acting character to be displayed based on the received operation command;
It has a configuration.

  Then, the system control device 20 has a configuration in which an image from the viewpoint of a virtual camera arranged in the object space is displayed on the large display device 12 as a content to be viewed by the audience with the acting character as the center.

  In particular, when the system control device 20 selects one content in which the type of acting character such as gender, age, number of people, and idle characters is defined, the object space is determined according to the operation of the operator while following the algorithm included in the content. The performance performance of a performance character (for example, an idol character) arranged on the screen is imaged.

  For example, as shown in FIG. 2, the system control apparatus 20 of the present embodiment is a group of acting characters composed of two groups of a first acting character (idol character) 8A and a second acting character (idol character) 8B. The large-scale display device 12 displays the live performance executed as described above as content.

  The server device 11 provides various data to each content screening system 10 using, for example, an API (Application Programming Interface) or a predetermined platform, or acquires and manages audience information from each content screening system 10. Information processing apparatus.

  In particular, the server device 11 images the performance of the acting character in the content screening system 10 and provides data (hereinafter referred to as “content data”) for outputting sound to each content screening system 10.

  In addition, the server device 11 has a storage area (a storage unit 140 described later), and using the recording area, each spectator's billing information, action history information (hereinafter referred to as “action history information”), and the like. Perform information management. However, the server device 11 may store data in a database (storage device or memory in a broad sense) connected via a network (intranet or Internet) instead of the storage area.

  In addition, the server apparatus 11 may be comprised by one (apparatus, processor), and may be comprised by the some (apparatus, processor).

  Further, in the present embodiment, the content screening system 10 is configured in the object space based on content data (specifically, control data that defines an algorithm described later) or based on an operation command received by an operator. However, the server device 11 may perform part or all of the processing of each action control of the acting character.

  In this case, for example, when all the processes are executed by the server device 11, the content screening system 10 displays the display control data for displaying the acting character whose motion is controlled in the object space transmitted from the server device 11. Are received, and display control is performed, and operation information input by the operator is transmitted to the server device 11.

[2] Server Device First, the outline and configuration of the server device 11 of this embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating functional blocks of the server device 11 according to the present embodiment. Moreover, the server apparatus 11 of this embodiment is good also as a structure which abbreviate | omitted a part of component (each part) of FIG.

  The server apparatus 11 includes a server processing unit 100 that mainly performs processing related to provision of content to be provided, an input unit 120 for use by an administrator and other inputs, a display unit 130 that performs predetermined display, and main information such as audience information. A storage unit 140 that stores various data used for content, an information storage medium 180 that stores predetermined information, and a communication unit 196 that communicates with the content screening system 10 and others.

  The input unit 120 is used by a system administrator or the like for settings related to content, other necessary settings, and data input. For example, the input unit 120 of this embodiment is configured with a mouse, a keyboard, and the like.

  The display unit 130 displays an operation screen for the system administrator. For example, the display unit 130 of the present embodiment is configured by a liquid crystal display or the like.

  The information storage medium 180 (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (CD, DVD), magneto-optical disk (MO), magnetic disk, hard disk, and magnetic tape. Or a memory (ROM) or the like.

  The communication unit 196 performs various controls for communicating with the outside (for example, a terminal, another server, or another network system), and functions as hardware such as various processors or a communication ASIC. Hardware and programs.

The storage unit 140 serves as a work area for the server processing unit 100, the communication unit 196, and the like, and its function is configured by a RAM (VRAM) or the like. Note that information stored in the storage unit 140 may be managed in a database.

  In the present embodiment, the storage unit 140 includes a main storage unit 142 used as a work area, content data 144 indicating data related to content to be provided to each content screening system 10, and audience indicating audience information regarding the audience. Information 146 and data (hereinafter referred to as “content collection data”) 148 collected and displayed for each content are stored. However, the storage unit 140 stores various types of information necessary for content calculation in addition to the above information.

  In the content data 144, for each content, data that specifies an algorithm used in each content screening system 10 in association with the content ID (hereinafter also referred to as “control data”), and the acting character indicates the performance. Various kinds of sound data such as music and lines at the time of execution, and various kinds of image data used for imaging an object space including acting characters are included.

  In particular, the algorithm includes the action character action in the object space, the position and angle of the virtual camera set in the object space, the timing of presenting the operation command, the timing controlled by the operation command, the type of operation command to be presented In addition, the flow of the content showing the output timing of various sounds such as music and lines when the acting character performs the performance is defined.

  In addition, the content data 144 stores the type, timing, and number of times of each operation command actually used by each content screening system 10 as usage history information for each content.

The audience information 146 is associated with the audience ID for each audience,
(1) Audience attribute information such as gender, age, and residential area,
(2) Audience preference information such as favorite acting characters,
(3) Information of terminal devices held by the audience such as smartphones and ID tags (for example, information specifying the audience's individuals and information indicating the position (seat) at the time of viewing such as ticket information),
(4) Information stored in association with each spectator such as an ID or in association with a terminal device held by the spectator, and the target character being viewed or the target character is performing Information (game play information) regarding play status and history (including billing history) of a predetermined game such as a game related to the event,
(5) Information stored using the terminal device held by the spectator, including action history information indicating participation in an event in which the target character or the target character is performing, and related to the event or performance Billing information and billing history information from the designated mail-order sites and retail stores (including event venues),
(6) Biological information (for example, facial images, voice information (including voice print information and frequency characteristic information indicating frequency characteristics) of each audience), posture information, heat information, and a predetermined device (smart phone, Information similar to biological information such as light information using psyllium)) is included.

  In addition, as described above, the charge history includes the charge history in the game and the charge related to the purchase of services and products, the charge date and time indicating the date and time when these charges were executed, the purchase of items and the purchase of twists. The type of charging element such as, the amount of charge indicating the amount charged, and the like are included.

  The content collection data 148 includes, for each content, data of the content shown in each content screening system 10 (date and number of times), operation command data received in accordance with an instruction from the operator when the content was shown, and Data indicating the content of motion control of each acting character is stored.

  The server processing unit 100 performs various processes using the main storage unit 142 in the storage unit 140 as a work area. The functions of the server processing unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.), ASIC (gate array, etc.), and programs.

  The server processing unit 100 performs various processes of the present embodiment based on a program (data) stored in the information storage medium 180. That is, the information storage medium 180 stores a program for causing a computer to function as each unit of the present embodiment (a program for causing a computer to execute processing of each unit).

  For example, the server processing unit 100 (processor) controls the entire server device 11 based on a program stored in the information storage medium 180, and provides content data to the content screening system 10, or Various processes such as a process of providing spectator information in conjunction with the content screening system 10 and a process of registering operation commands and spectator information collected during the screening of content transmitted from the content screening system 10 in the storage unit 140 are performed.

  Specifically, the server processing unit 100 of this embodiment includes at least a communication control unit 101, a content management unit 102, and an audience management unit 103.

  The communication control unit 101 performs processing for transmitting and receiving data to and from the content screening system 10 via a network. That is, the server apparatus 11 transmits content data to the content screening system 10 by the communication control unit 101 and receives various information such as audience information and operation commands transmitted from the content screening system 10.

  The content management unit 102 registers, in the content collection data 148, data such as the performance of the content screened in each content screening system 10 and operation commands used during the screening of each content screening system 10.

  In addition, the content management unit 102 analyzes each content collection data of the same content at a predetermined timing, optimizes an operation command to be presented to the operator, an operation command set as a default during the content screening, etc. Update the algorithm set with.

  The spectator management unit 103 performs processing related to each spectator registration procedure (registration of attribute information, preference information, etc.) and storage management of data associated with an account (audience ID).

  Specifically, the spectator management unit 103 is also referred to as account grant control for registering and managing each spectator information, spectator information of each spectator detected during content execution (hereinafter referred to as “detected spectator information”, for example, In addition to the biometric information of each spectator in the face image, it also includes operation commands and performance timing when the biometric information is detected.) And personal information such as spectator attribute information and preference information Management, virtual account setting and deposit / withdrawal control to be used as a payment source for compensation associated with the execution of the accounting element linked to the account, registration of accounting history related to the accounting element by the audience of the account, and registration of game history Etc. are executed.

  In particular, the audience management unit 103 stores and manages each piece of information detected by the content screening system 10 as a detected audience information in association with an audience information account.

The spectator management unit 103 also stores game play information and billing history information such as a game server device (not shown), a management server device that operates a mail order site, or another server (not shown) such as a credit card or bank settlement server device. In conjunction with the apparatus, each information is registered in the audience information 146 by exchanging predetermined data.

[3] System Control Device Next, the content screening system 10 of this embodiment will be described with reference to FIG. FIG. 4 is a diagram showing functional blocks of the content screening system 10 of the present embodiment. In addition, the content screening system 10 according to the present embodiment may have a configuration in which some of the components (each unit) in FIG. 4 are omitted.

  The spectator information detection unit 250 detects various biological information such as an imaging camera, an infrared sensor, a sound collecting microphone, a seating sensor, a thermography, or a marker unit included in each spectator and its detection device (camera), or information equivalent to the biological information. It has a function to do.

  The spectator information detection unit 250 detects predetermined biological information and the like according to an algorithm during the content screening, and provides the detected information to the operation command candidate presentation processing unit 214.

  For example, the audience information detection unit 250 of the present embodiment constitutes the detection means of the present invention.

  The input unit 260 is for an operator to input an operation command, and the function can be realized by a touch panel, a touch panel display, a trackball, a keyboard, a mouse, or the like.

  That is, for example, when the input unit 260 is configured by a device that performs touch operation input such as a touch panel, the two-dimensional designated position coordinates (x, y) on the screen on which the image is displayed can be detected. The detection part 262 is provided.

  For example, the input unit 260 includes a detection unit 262 that can detect a two-dimensional contact position coordinate (x, y) in the contact detection region (touch panel).

  In the present embodiment, the contact position is position information obtained from the contact detection area by an operator's contact operation (including direct contact and indirect contact with a pointing device).

  Note that the contact operation of the operator according to the present embodiment includes a direct touch operation on the monitor display unit 290 (touch panel) by a user's finger and an indirect operation performed via a device such as a pointing device. In addition to a touch operation on the monitor display unit 290 (touch panel), a monitor display unit using a light beam or a camera without directly or indirectly touching or sliding on the monitor display unit 290 (touch panel). A pseudo touch operation on 290 (touch panel) is also included.

  In the present embodiment, the monitor display unit (display) 290 for the operator is a touch panel type display in which a liquid crystal display and a touch panel for detecting the contact position of the operator are stacked. That is, in the present embodiment, the monitor display unit 290 functions as the input unit 260.

  Further, the touch operation on the monitor display unit 290 may be performed using a fingertip, or may be performed using an input device such as a touch pen.

  Furthermore, the input unit 260 may include a button, a lever, a steering, a microphone, an acceleration sensor, and the like that can input operation information (operation signal) other than the designated position.

  The storage unit 270 serves as a work area for the processing unit 200, the communication unit 296, and the like, and its function can be realized by a RAM (VRAM) or the like. The storage unit 270 of this embodiment includes a main storage unit 271 used as a work area, an image buffer 272 in which a final display image and the like are stored, audience information 273 indicating information about the audience, and algorithm And a content data storage unit 274 that stores various data necessary for executing the presentation of content such as data, operation command data, or table data.

  In the spectator information 273, predetermined spectator information such as spectator information of each spectator detected by the spectator information detection unit 250 and attribute information of spectators who have visited the screening venue is stored.

  The content data storage unit 274 stores, as usage history information, the type, timing, and number of times of each operation command used during the content screening.

  In addition, it is good also as a structure which abbreviate | omits these, and the memory | storage part 140 of the server apparatus 11 may comprise that part. Further, for example, the storage unit 270 of the present embodiment constitutes a storage unit of the present invention alone or together with the storage unit 140 of the server device 11.

  In the spectator information 273, for example, spectator information stored in the storage unit 140 of the server device 11 is stored with spectator information of the audience who is scheduled to view the viewing performance or is observing.

  The content data storage unit 274 stores various types of information necessary for showing the content, information such as operation commands used during the showing of the content, and condition information used in various determination processes. .

  The information storage medium 280 (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (CD, DVD), magneto-optical disk (MO), magnetic disk, hard disk, and magnetic tape. Alternatively, it can be realized by a memory (ROM).

  In addition, the information storage medium 280 can store a program for causing a computer to function as each unit of the present embodiment (a program for causing a computer to execute processing of each unit). Note that the processing unit 200 performs various processes of the present embodiment based on a program (data) stored in the information storage medium 280, as will be described later.

  The monitor display unit 290 outputs an image generated according to the present embodiment, and its function is a CRT, LCD, touch panel type display, HMD (head mounted display), for example, a projection device including a screen and a projector. It can be realized by.

  In particular, in the present embodiment, the monitor display unit 290 also functions as the input unit 260 where the operator performs various inputs by using the touch panel display as described above. Here, as the touch panel, for example, a resistive film type (4-wire type, 5-wire type), capacitive type, electromagnetic induction type, ultrasonic surface acoustic wave type, infrared scanning type, or the like can be used.

  The sound output unit 292 outputs the sound generated by the present embodiment, and its function can be realized by a speaker, headphones, or the like.

  The communication unit 296 performs various controls for communicating with the outside (for example, a host device or another terminal device), and functions thereof are hardware such as various processors or communication ASICs, and a program. It can be realized by.

  The system control device 20 receives, via the network, a program or data for causing the computer to function as each unit of the present embodiment stored in the information storage medium or the storage unit 270 of the server device 11. The program and data may be stored in the information storage medium 280 or the storage unit 270. The case where the system control device 20 is made to function by receiving the program or data as described above can also be included in the scope of the present invention.

  The processing unit 200 (processor) presents an operation command according to the program and algorithm data provided from the server device 11 or in conjunction with the server device 11 based on the input data or program from the input unit 260. Processing such as content screening processing, image generation processing, and sound generation processing including processing and operation command reception processing is performed.

  In particular, in the present embodiment, as the content screening processing, processing for starting content screening when content screening start conditions are satisfied, processing for arranging various objects used for content screening such as acting characters, etc. , Processing for controlling the action of the acting character in accordance with an algorithm or in accordance with an operation command of the operator, processing for ending the game when a content screening end condition is satisfied, and the like.

  The processing unit 200 performs various processes using the storage unit 270 as a work area. The functions of the processing unit 200 can be realized by hardware such as various processors (CPU, DSP, etc.), ASIC (gate array, etc.), and programs.

  In particular, the processing unit 200 of the present embodiment includes an object space setting unit 210, a touch detection processing unit 211, a content display control unit 212, a monitor display control unit 213, an operation command candidate presentation processing unit 214, and a movement processing unit. 215, a virtual camera control unit 216, a content editing processing unit 217, a content screening execution control unit 218, a content management unit 219, a drawing unit 220, and a sound processing unit 230. A part of these may be omitted.

  For example, the touch detection processing unit 211 of the present embodiment constitutes the acceptor of the present invention, and the content display control unit 212 constitutes the audience display control means of the present invention. Further, for example, the monitor display control unit 213 of the present embodiment constitutes a monitor display control unit of the present invention, and the operation command candidate presentation processing unit 214 constitutes a presentation unit of the present invention. Further, for example, the content editing processing unit 217 of the present embodiment constitutes an editing unit, and the content screening execution control unit 218 constitutes a registration unit and an operation control unit of the present invention.

  The object space setting unit 210 includes various objects (sprites, billboards, polygons, free-form surfaces, or subdivisions) representing display objects such as character objects (acting characters), moving paths, buildings, trees, pillars, walls, and maps (terrain). A process of placing and setting an object composed of a primitive surface such as a surface in an object space (that is, a game space) is performed.

  Specifically, the object space setting unit 210 determines the position and rotation angle (synonymous with orientation and direction) of the object (model object), and rotates the object to that position (X, Y) or (X, Y, Z). The object is arranged at an angle (rotational angle around the X, Y axis) or (rotational angle around the X, Y, Z axis).

  The object space includes both a so-called virtual two-dimensional space and a virtual three-dimensional space. The two-dimensional space is a space in which an object is arranged at, for example, two-dimensional coordinates (X, Y), and the three-dimensional space is a space in which an object is arranged at, for example, three-dimensional coordinates (X, Y, Z). is there.

  When the object space is a two-dimensional space, the objects are arranged based on the priority order set for each of the plurality of objects. For example, processing can be performed in which objects are arranged in order from the object (sprite) that the user wants to appear on the back side, and the objects that the user wants to appear on the near side are superimposed.

  Also, if you place an object with a large drawing size below the image and an object with a small drawing size above the image, you can make the object space corresponding to the top of the screen appear to be on the back side. It can be seen that the corresponding object space is on the near side.

  On the other hand, when the object space is a three-dimensional space, the object is arranged in the world coordinate system.

  The touch detection processing unit 211 performs a recognition process for receiving and recognizing instruction information for performing action control of the acting character such as an operation command input by the operator to the input unit 260.

  Specifically, the touch detection processing unit 211 acquires a contact position (two-dimensional contact position coordinates) in a contact detection area (touch panel) for detecting an operator's contact operation (ie, touch operation) as an instruction position.

In particular, the touch detection processing unit 211
(1) A change (hereinafter referred to as “state change”) from a state where no touch operation is input to the screen (hereinafter referred to as “contactless state”) to a state where the touch operation is input (hereinafter referred to as “contact state”). )) And a detection position where the state change is executed,
(2) Detection of an operation command candidate displayed at the detection position as an operation command, operation command editing instruction (for example, addition, change and deletion of default operation control in an acting character with respect to the operation command), or performance Detection of the viewing of a specific audience who enjoys a special performance of the character is executed.

  The content display control unit 212 executes various kinds of control for displaying on the large display device 12 an image obtained by imaging the object space centered on the acting character so that the audience can view the image.

  The monitor display control unit 213 relates to operation support such as displaying content centered on the acting character being displayed on the monitor display unit 290 and displaying operation command candidates to the operator during the content screening. Various displays are performed on the monitor display unit 290.

  The movement processing unit 215 performs a movement calculation of the object in the object space. That is, the movement processing unit 215 moves the moving object in the object space based on input data or a program (movement algorithm) or various data (motion data) input by the operator through the input unit 260, or Processing for controlling the motion (motion, animation) of the moving object is performed.

Specifically, the movement processing unit 215 according to the present embodiment performs object movement information (movement direction, movement amount, movement speed, position, rotation angle, or acceleration) and operation information (position or rotation angle of each part object). ) Is sequentially obtained for each frame. Here, the frame is a unit of time for performing object movement processing, motion processing (simulation processing), and image generation processing. In the present embodiment, the frame rate may be fixed or variable depending on the processing load.

  The virtual camera control unit 216 generates an image that is visible from a given viewpoint and appears to have depth. In this case, the virtual camera control unit 216 performs a virtual camera (viewpoint) control process for generating an image that can be seen from a given (arbitrary) viewpoint in the object space. Specifically, a process for controlling the position (X, Y, Z) or the rotation angle (rotation angle about the X, Y, Z axes) of the virtual camera (process for controlling the viewpoint position and the line-of-sight direction) is performed.

  For example, when an object (for example, a character, a ball, a car) is photographed from behind using a virtual camera, the position or rotation angle of the virtual camera (virtual camera is set so that the virtual camera follows changes in the position or rotation of the object. The direction).

  In this case, the virtual camera can be controlled based on information such as the position, rotation angle, or speed of the object obtained by the movement processing unit 215. Alternatively, the virtual camera may be controlled to rotate at a predetermined rotation angle or to move along a predetermined movement path. In this case, the virtual camera is controlled based on virtual camera data for specifying the position (movement path) or rotation angle of the virtual camera.

  When there are a plurality of virtual cameras (viewpoints), the above control process is performed for each virtual camera.

  The content editing processing unit 217 executes editing processing for editing the motion control of the target character corresponding to the operation command based on an instruction from the operator during the content screening or at other timing.

The content screening execution control unit 218
(1) executing an algorithm control process for controlling the action of the target character imaged according to an algorithm defined in the control data included in the content data;
(2) When an operation command is received during the algorithm control process, it is linked with the action of the acting character that is the target of the algorithm control process, or instead of the action of the action character specified in the algorithm control process Then, the movement of the acting character shown is controlled based on the received operation command.

  Also, the content screening execution control unit 218 provides the server device 11 with the operation commands used and the acting character actions based on the operation commands used, the detected detected audience information, and the used audience information as content collection data during the content screening. To do.

  Specifically, the content screening execution control unit 218 uses the operation command received during the screening and the detected audience information as information for analyzing the content screening result information at a predetermined timing after the content screening, or Then, it is registered in the server device 11 as usage history information used for other screenings of the same content.

  The content management unit 219 displays, as content screening result information, the type and timing of the received operation command and each spectator information detected when the acting character's action is executed based on the operation command during the content screening. And stored in the content data storage unit 274.

  In addition, the content management unit 219 uses the operation command received during the screening and the detected audience information as information for analyzing the content screening result information at a predetermined timing after the screening of the content, or in the same content It is registered in the server device 11 as usage history information used for other screenings.

  The drawing unit 220 performs drawing processing based on the results of various types of processing (each processing related to content screening) performed by the processing unit 200, thereby generating an image, and the large display device 12 and the monitor display unit (display). To 290.

  The image generated by the drawing unit 220 may be a so-called two-dimensional image or a so-called three-dimensional image. The drawing unit 220 is an image that can be viewed from a virtual camera in the object space, and Generate an image to display on top.

  Here, when generating a two-dimensional image, the drawing unit 220 draws objects in order from the set low priority object, and when the objects overlap, draws by overwriting the high priority object. .

  When generating a three-dimensional image, the drawing unit 220 of the present embodiment firstly uses vertex data (vertex position coordinates, texture coordinates, color data, normal vector, or α value) of each vertex of the object (model). Etc.) is input, and vertex processing is performed based on the vertex data included in the input object data. When performing the vertex processing, vertex generation processing (tessellation, curved surface division, polygon division) for re-dividing the polygon may be performed as necessary.

  In the vertex processing, geometric processing such as vertex movement processing, coordinate transformation (world coordinate transformation, camera coordinate transformation), clipping processing, perspective transformation, or light source processing is performed. The given vertex data is changed (updated or adjusted) for the vertex group to be configured. Then, rasterization (scan conversion) is performed based on the vertex data after the vertex processing, and the surface of the polygon (primitive) is associated with the pixel. Subsequent to rasterization, pixel processing (fragment processing) for drawing pixels constituting an image (fragments constituting a display screen) is performed.

  In pixel processing, various processes such as texture reading (texture mapping), color data setting / changing, translucent composition, anti-aliasing, etc. are performed to determine the final drawing color of the pixels that make up the image, and perspective transformation is performed. The drawing color of the object is output (drawn) to the image buffer 272 (frame buffer, buffer that can store image information in units of pixels; VRAM, rendering target). That is, in pixel processing, per-pixel processing for setting or changing image information (color, normal, luminance, α value, etc.) in units of pixels is performed.

  Thereby, an image that can be seen from the virtual camera (given viewpoint) set in the object space is generated. Note that when there are a plurality of virtual cameras (viewpoints), an image can be generated so that an image seen from each virtual camera can be displayed as a divided image on one screen.

It should be noted that the vertex processing and pixel processing performed by the drawing unit 220 are performed by hardware that can program polygon (primitive) drawing processing by a shader program written in a shading language, so-called programmable shaders (vertex shaders and pixel shaders). It may be realized. Programmable shaders can be programmed with vertex-level processing and pixel-level processing, so that the degree of freedom of rendering processing is high, and the expressive power can be greatly improved compared to fixed rendering processing by hardware. .
The drawing unit 220 performs geometry processing, texture mapping, hidden surface removal processing, α blending, and the like when drawing an object.

  In the geometry processing, processing such as coordinate conversion, clipping processing, perspective projection conversion, or light source calculation is performed on the object. Then, the object data (positional coordinates of object vertices, texture coordinates, color data (luminance data), normal vector, α value, etc.) after geometry processing (after perspective projection conversion) is stored in the storage unit 270.

  In the texture mapping, the texture (texel value) stored in the texture storage unit of the storage unit 270 is mapped to the object. Specifically, the texture (surface properties such as color (RGB) and α value) is read from the texture storage unit of the storage unit 270 using the texture coordinates set (given) to the vertex of the object, etc. Map textures that are to objects. In this case, processing for associating pixels with texels, bilinear interpolation or the like is performed as texel interpolation.

  In the present embodiment, when an object is drawn, a process for mapping a given texture may be performed. In this case, the color distribution (texel pattern) of the texture to be mapped can be dynamically changed.

  In this case, textures having different color distributions (pixel patterns) may be dynamically generated, or a plurality of textures having different color distributions are prepared in advance, and the texture to be used is dynamically switched. May be. The texture color distribution may be changed in units of objects.

  In the hidden surface removal processing, hidden surface removal processing is performed by a Z buffer method (depth comparison method, Z test) using a Z buffer (depth buffer) in which the Z value (depth information) of the drawing pixel is stored. That is, when drawing the drawing pixel corresponding to the primitive of the object, the Z value stored in the Z buffer is referred to, and the Z value of the referenced Z buffer and the Z value at the drawing pixel of the primitive are obtained. In comparison, if the Z value at the drawing pixel is a Z value (for example, a small Z value) that is on the near side when viewed from the virtual camera, the drawing pixel is drawn and the Z value in the Z buffer is updated. Update to the correct Z value.

  In α blending (α synthesis), the rendering unit 220 performs a translucent synthesis process (usually α blending, addition α blending, subtraction α blending, or the like) based on an α value (A value). The α value is information that can be stored in association with each pixel (texel, dot), for example, plus alpha information other than color information. The α value can be used as mask information, translucency (equivalent to transparency and opacity), bump information, and the like.

  In particular, the drawing unit 220 in the present embodiment may generate an image to be displayed in a display area corresponding to the contact detection area. The image displayed in the display area may be an image including an object, for example.

  The sound processing unit 230 performs sound processing based on the results of various processes performed by the processing unit 200, and generates various sounds when the acting character performs performance such as BGM, sound effects, or voices, The sound is output to the sound output unit 292.

  The timer unit 240 performs a time measurement for executing a process for measuring a timer value such as a time management related to a content screening based on the current date and time, a time limit, and an algorithm using the system clock.

[4] Technique [4.1] Outline of Present Embodiment Next, an outline of the technique (operation command presentation processing) of the present embodiment will be described.

  The content screening system 10 according to the present embodiment has a configuration that allows an operator to easily operate an acting character without requiring special skills when content is displayed.

  The content screening system 10 of the present embodiment can move the character based on a simple operation in accordance with the atmosphere of the audience viewing the acting character and the state of the audience. And the live feeling in the performance of the target character, the audience can be enlivened and a fresh performance can be realized.

In particular, the content screening system 10
(1) Control the motion of the acting character while placing the acting character in the object space according to the algorithm included in the content data acquired from the server device 11; (2) Virtually focusing on the acting character whose motion is controlled Displaying an image of the object space from the viewpoint of the camera on the large display device 12 and the monitor display unit 290;
Each process is configured to be executed as a content screening process for screening content.

Then, the content screening system 10 performs the content screening process,
(1) An operation command candidate that is presented to the operator as an operation command candidate from among a plurality of operation commands for controlling the movement of the target character in the object space, according to the audience information. Presentation process, and
(2) When one or more operation command candidates are specified from the operation command candidates presented by the operator, the specified operation command candidates are received as operation commands, and acting is performed based on the received operation commands. Processing for controlling the movement of the character (hereinafter, also referred to as “motion control processing based on operation commands”),
It has the structure which performs each process of these.

  By having such a configuration, the content screening system 10 increases the audience when the audience watching the target character such as the event venue is not excited based on the audience information. The audience watching the target character, such as making it possible to select a command to execute MC (Serif or Microphone Performance), or to select an operation command for the target character to perform intense dance The operation of the target character according to the situation (that is, the current state of the audience) can be selected by the operator.

  Also, the content screening system 10 controls the movement of the target character for women when the audience information determines that there are many women viewing the target character, or predetermined goods in the past action. Operation command to control the action of the target character in accordance with the behavior history and attributes of the spectator who is viewing by executing a special performance of the target character to the spectator. Can be selected by the operator.

As described above, the content screening system 10 according to the present embodiment can move the acting character according to the atmosphere of the audience and the state of the audience while simplifying the operation by the operator. It is possible to increase the audience and to realize a fresh performance while improving the sense of liveness and the live feeling in the performance of the acting character.

  Therefore, the content screening system 10 can control the action (that is, performance) of the target character that is highly satisfied by the audience.

[4.2] Operation Command Candidate Presentation Processing Next, content operation command candidate presentation processing according to the present embodiment will be described with reference to FIGS. 5 to 7 are diagrams for explaining the content operation command candidate presentation processing according to the present embodiment.

(Basic principle)
The operation command candidate presentation processing unit 214 includes, as the operation command candidate presentation processing, an operation command candidate for controlling the action character's motion at a predetermined timing defined in the algorithm from the start to the end of the performance of the acting character. Present.

In particular, the operation command candidate presentation processing unit 214
(1) A first operation command candidate that presents, as operation command candidates, an operation command selected by the operator's intention while referring to audience information from among a plurality of operation commands for controlling the movement of the target character in the object space Presentation process, and
(2) An operation command selected according to audience information and selected by the operator from among a plurality of operation commands for controlling the movement of the target character in the object space is operated as an operation command candidate. The second operation command candidate presentation process presented to the user is executed.

  In the present embodiment, the operation command candidate presentation processing unit 214 executes either the first operation command candidate presentation process or the second operation command candidate presentation process during the performance of the acting character based on the algorithm. To do.

(First operation command candidate presentation process)
The operation command candidate presentation processing unit 214 works in conjunction with the server device 11 to specify the performance performance of the performance character while specifying the audience who has come to the event venue in advance using an entrance ticket such as associating the ticket ID and the audience ID. The audience information of the viewing audience's attributes, preference information, game play information, and billing history information is acquired from the server device 11.

  Then, the operation command candidate presentation processing unit 214 is linked with the monitor display control unit 213, and at the timing specified by the algorithm, the acquired audience information, analysis information indicating the result of analyzing the audience characteristics by the audience information, or Both pieces of information are displayed on the monitor display unit 290 and presented to the operator.

  In particular, the operation command candidate presentation processing unit 214 is obtained in conjunction with the monitor display control unit 213 so that the operator can easily select an operation command for causing the acting character to perform an action that matches the audience. The displayed audience information is displayed in a list on the monitor display unit 290.

  In addition, the operation command candidate presentation processing unit 214 can identify a specific spectator in the event venue (specifically, a position in the venue of the specific spectator) by using a seat or the like. In conjunction with H.213, the acquired audience information is displayed in a list on the monitor display unit 290 in association with the audience (or the audience position).

  For example, as shown in FIG. 5A, the operation command candidate presentation processing unit 214 causes the operator to select an operation command for executing a suitable effect when performing an effect on the front row of the event venue. In addition, the spectator information of the spectators in the front row of the event venue is displayed in a list on the monitor display unit 290 together with the operation command candidates defined at the timing by the algorithm.

  FIG. 5A shows an example in which each operation command of commands A to D formed on the first acting character 8A is displayed on the monitor display unit 290 which is a touch panel together with the audience information as operation command candidates. Show.

  In particular, in FIG. 5A, in addition to the above, image parts for supporting the operation of the operator such as the command execution speed changeable by the touch operation and the content screening time are displayed on the monitor display unit 290. An example is shown.

  In addition, as shown in FIG. 5B, when the content screening execution control unit 218 receives an operation command for operating the acting character from the operation command candidates presented by the touch detection processing unit 211, the content screening execution control unit 218 receives the operation command. Based on this, the motion control of the target acting character is executed.

  FIG. 5B shows that there are male spectators who like the first acting character 8A in the lower front row of stage 2 and who have a billing amount of a certain amount or more. An example is shown in which the first acting character 8A has received an operation command A for performance “shake hand with lower hand” to the lower front row of stage 2. However, the action control of the acting character in the content screening execution control unit 218 will be described later.

  On the other hand, the operation command candidate presentation processing unit 214 determines the characteristics of the entire audience, the characteristics of each audience, based on the audience information of the audience who is visiting the event venue before or during the performance of the acting character. Characteristics (individual characteristics) or both may be analyzed.

For example, the operation command candidate presentation processing unit 214 has the following characteristics of the entire audience:
(A1) Audience composition ratio (age, gender difference, regional difference),
(A2) Support rate of each acting character as a favorite character (such as many spectators who like character A),
(A3) Total and average charges for the entire audience, and
(A4) The average play time in a specific game is analyzed.
Further, for example, the operation command candidate presentation processing unit 214 has the following characteristics as each audience:
(B1) A spectator who is charging more than a certain amount,
(B2) Audience who purchased a specific product or service, or
(B3) Predetermined conditions in a specific game such as possessing a specific player character or item in a specific game, being above a predetermined level, or having a predetermined point or experience value The audience they have,
Is analyzed.

  Then, the operation command candidate presentation processing unit 214 refers to the audience information, presents it as an operation command that can be selected according to the operator's intention, and presents it with an algorithm in conjunction with the monitor display control unit 213. The analysis result is displayed at a position on the monitor display unit 290 that can be visually recognized by the operator at the same time as the operation command specified to be performed.

  For example, as shown in FIG. 6A, the operation command candidate presentation processing unit 214 indicates that there are more than a certain percentage of spectators who have attended an event more than a predetermined number of times (for example, 3 times). In the case of analysis, in order for the operator to easily select an operation command for executing a special effect such as throwing a kiss or a speech toward the audience, a charging history of a certain amount or more for the audience is required. The monitor display unit 290 displays a list of the presence and the seat position of the corresponding audience along with operation command candidates.

  In such a case, for example, the content screening execution control unit 218 performs motion control of an acting character that sings a new song that has not been performed so far as a special effect when the number of visits is large.

  Further, for example, as shown in FIG. 6B, the operation command candidate presentation processing unit 214 analyzes that the female audience is present in the event venue by a certain percentage or more (for example, 70%). In order to allow the operator to easily select an operation command for performing a directing effect, the monitor display unit 290 displays a display indicating that the percentage of the female audience is constant.

  In such a case, for example, the content screening execution control unit 218 executes motion control of an acting character that emits a dialogue for women.

  On the other hand, as shown in FIG. 5 and FIG. 6, the operation command candidate presentation processing unit 214 is linked to the monitor display control unit 213 and displays an image of the acting character to be operated on the monitor display unit 290 as a touch panel. Each operation command candidate may be displayed so that an operation command can be selected on the acting character image.

  That is, in this embodiment, by having such a configuration, it is possible to intuitively execute an operation command operation instruction by the operator, thereby simplifying the operation of the operator and improving the operability thereof. Be able to.

(Second operation command candidate presentation process)
The operation command candidate presentation processing unit 214 works in conjunction with the spectator information detection unit 250 and, at a timing specified by the algorithm, biological information such as the spectator's facial expression and voice, the spectator's posture, hot air, or light emitted by the spectator (for example, Smart phones and psyllium), or information on terminal devices owned by the audience (for example, information for identifying each audience's viewing (seat) position and individuals, game history information, purchase history information of products, etc.) Information corresponding to biological information is detected.

  That is, the operation command candidate presentation processing unit 214, for example, whether or not the audience is exciting, whether or not the audience is enjoying, the gender ratio (whether there are many men or many women), or, for example, the acting character Audiences emitting red light used to respond to dialogues and games (indicating the element of jaken by the type of light, or “YES” for red light, “NO” for blue light, etc.) Analyze the state of the audience, such as whether there are many.

Camera In particular, the audience information detection unit 250 includes an imaging camera, an infrared sensor, a sound collection microphone, a seating sensor, a thermography, a marker unit included in each audience, and a detection device (camera) terminal device (for example, a smartphone or an ID tag). It has a function for detecting various types of biological information such as a wireless receiver that acquires information transmitted from the camera, or a camera that captures the entire screening venue and acting characters.

Further, the operation command candidate presentation processing unit 214 stands by each seat sensor by the sound component (frequency characteristic) of each spectator collected by the sound collecting microphone based on the face image of each spectator captured by the imaging camera. It is analyzed as the state of the audience whether it is excited by the amount of heat of each audience depending on whether it is sitting or sitting.

  Then, the operation command candidate presentation processing unit 214 selects an operation command for excitement, an operation command when there are many women, or a case where the audience is enjoying the operation command from the operation commands defined at the corresponding timing by the algorithm. An operation command or the like is selected (extracted) as an operation command candidate, and the selected operation command candidate is displayed on the monitor display unit 290 in conjunction with the monitor display control unit 213 and presented to the operator.

  For example, the operation command candidate presentation processing unit 214, as shown in FIG. 7, is a case where a “Genki!” Question (ie, a call) is executed from the acting character toward the audience according to the algorithm, When the unit 250 detects a cheering of a predetermined decibel (for example, 80 dB at the center of the stage) or more as the reaction (ie, response), the first acting character A “thanks everyone” from the plurality of operation commands. An operation command A for issuing a line, an operation command B for the first acting character A issuing a line “Still More!”, And an operation command C for starting the next song by issuing a line “First Go” A Is displayed as an operation command candidate on the monitor display unit 290 and presented to the operator.

  Note that the spectator information detection unit 250 registers the spectator information indicating the detected spectator state in the content data storage unit 274 together with operation command information as usage history information to be described later.

[4.3] Operation Control Process Based on Operation Command Next, the operation control process based on the co-operation command of the present embodiment will be described with reference to FIGS. 8 and 9. 8 and 9 are diagrams for explaining the operation control process based on the content operation command of the present embodiment.

The content screening execution control unit 218
(1) executing an algorithm control process for controlling the action of the target character imaged according to an algorithm defined in the control data included in the content data;
(2) When an operation command is received during the algorithm control process, it is linked with the action of the acting character that is the target of the algorithm control process, or instead of the action of the action character specified in the algorithm control process Then, the movement of the acting character shown is controlled based on the received operation command.

  That is, when the content screening execution control unit 218 receives an operation command candidate specified by the touch detection processing unit 211 among the presented operation command candidates as an operation command, the content screening execution control unit 218 controls the action of the acting character by an algorithm (hereinafter referred to as an operation command). , "Algorithm control process"), or in place of the action character action of the algorithm control process, an operation command for controlling the action character action based on the received operation command. Based on the operation control process.

  In particular, the content screening execution control unit 218 changes the algorithm in the action of the corresponding acting character according to the received operation command, or replaces the algorithm with the action of the acting character in the algorithm control process. The motion of the acting character based on the received operation command is controlled in conjunction with or in place of the motion of the acting character in the algorithm control process.

For example, it is assumed that a speech is generated while controlling the body movement of the acting character according to a prescribed algorithm.

  In this case, when the algorithm is changed, as shown in FIG. 8A, when the touch detection processing unit 211 accepts an operation command “command A (swelling!)” Of the command A, the content As shown in FIGS. 8B and 8C, the screening execution control unit 218 controls the movement of the corresponding acting character's body using an algorithm, and changes the lines emitted from the acting character.

  FIG. 8B shows an example in which the line is changed from “good” to “exciting!” While controlling the movement of raising one hand toward the audience by an algorithm. (C) shows an example in which a line “Genki” defined by the algorithm is emitted while controlling the movement of raising one hand toward the audience by the algorithm.

  In addition, as shown in FIG. 8C, the content screening execution control unit 218 can receive an operation command by the touch detection processing unit 211 by a predetermined timing (for example, an operation start timing based on the operation command). If not, execute the action control that generates the line “Genki” originally defined in the algorithm

  On the other hand, in the same case, when the algorithm is replaced, as shown in FIG. 9A, when the operation command of the command B is received by the touch detection processing unit 211, the content screening execution control unit 218 displays FIG. As shown in (B) and (C), the algorithm in the action of the corresponding acting character is changed according to the received operation command.

  In FIG. 9B, for example, the movement of raising one hand toward the audience by an algorithm is changed to a movement that moves with both hands raised (for example, a running movement), and the lines are changed from “Genki-” to “ An example of changing to “It is exciting!” Is shown.

  Further, FIG. 9C shows an example in which the line “Genki-” defined by the algorithm is emitted while controlling the movement of raising one hand toward the audience by the algorithm, as in FIG. 8C. Show.

[4.4] Operation Command Candidate Presentation Processing Based on Operation Command Usage History Next, operation command candidate presentation processing based on the operation command usage history of the present embodiment will be described.

(Operation command candidate presentation process based on usage history of operation commands being screened)
The content screening execution control unit 218 registers and accumulates the operation command instructed by the operator (that is, the operation command received by the operation command candidate presentation processing unit 214) in the content data storage unit 274 during the content screening. .

  Specifically, the content screening execution control unit 218 registers and accumulates various types of information such as the type, timing, and number of use of operation commands in the content data storage unit 274 as usage history information.

  In addition, when the spectator information is detected, the content screening execution control unit 218 uses the spectator information indicating the detected spectator state together with operation command information in conjunction with the spectator information detection unit 250. The history information is registered and accumulated in the content data storage unit 274.

Then, the operation command candidate presentation processing unit 214 uses the operation command selected by the operator according to the audience information and the registered usage history information as the operation command candidate when presenting the operation command candidate. The operation command candidate presentation process to be presented is executed.

  In particular, the operation command candidate presentation processing unit 214 performs a plurality of operations corresponding to the first operation command candidate presentation process based on the use history from among a plurality of operation commands for controlling the motion of the target character in the object space. A command is selected (extracted), and the selected operation command is presented as a candidate operation command selected by the operator's intention while referring to the registered audience information.

For example, when the operation command candidate presentation processing unit 214 selects an operation command,
(1) An operation command related to the previous operation command,
(2) Operation commands that have not been used yet for the presentation of the content (ie, during a series of performance performances),
(3) frequently used operation commands,
(4) Operation commands raised during the screening, and
(5) An operation command for executing special effects for some spectators who have not been executed for other spectators or for individual audiences,
Are selected as operation command candidates from a plurality of operation commands, and the selected operation command candidates are displayed on the monitor display unit 290 and presented to the operator.

  Further, as described above, the operation command candidate presentation processing unit 214 displays the audience information on the monitor display unit 290 and presents it to the operator. Is presented to the operator.

  The operation command candidate presentation processing unit 214 may simply present an operation command corresponding to (1) to (4), or may be set in advance when priority is given to the operation command. The priority of the operation command candidates selected based on the priority may be determined by adding the priorities based on other factors such as the priority being reduced or the number of selections.

  In addition, when the operation command candidate presentation processing unit 214 causes the operator to select an operation command, for example, the operation commands that have been used cannot be presented sequentially, or the priority is lowered and can be presented, or the priority is high. The operation command may be displayed on the monitor display unit 290 as an operation command candidate and presented to the operator. However, the operation command candidate presentation processing unit 214 is configured to exclude the same operation command in the same audience for the operation commands for some spectators and individual spectators.

  Further, when the operation command candidate presentation processing unit 214 causes the operator to select an operation command, the operation command candidate presentation processing unit 214 displays information such as the number of times of use and use timing on the monitor display unit 290 together with the corresponding operation command and presents it to the operator. Also good.

  On the other hand, the operation command candidate presentation processing unit 214 performs the second operation command candidate presentation processing based on the detected audience information and usage history from among a plurality of operation commands for controlling the motion of the target character in the object space. A plurality of corresponding operation commands are extracted, and the extracted operation commands are presented as operation command candidates.

  The operation command candidate presentation processing unit 214 also includes a plurality of operation commands (for example, the operation commands (1) to (4) described above) as the operation command candidate presentation process, as in the first operation command candidate presentation process. The operation command is further selected according to the audience information from the selected operation command, and the selected operation command is displayed as an operation command candidate on the monitor display unit 290 and presented to the operator.

  Similar to the first operation command candidate presentation processing, the operation command candidate presentation processing unit 214 displays information such as the number of times of use and use timing together with the corresponding operation commands when the operator selects an operation command. It may be displayed on 290 and presented to the operator.

(Operation command candidate presentation processing based on operation command usage history for each content)
When the content screening execution control unit 218 executes the content screening, the content screening execution control unit 218 acquires use history information of operation commands already registered in the server device 11 or the like for the same content (that is, the same content ID). The data is stored in the data storage unit 274.

  Then, the content screening execution control unit 218, when presenting operation command candidates, allows the operator to select an operation command that can be selected according to audience information and registered usage history information, and audience information and usage history information. An operation command candidate presentation process for presenting the operator with at least one of the operation commands selected according to the above as an operation command candidate is executed.

  In particular, the operation command candidate presentation processing unit 214 performs a plurality of operations corresponding to the first operation command candidate presentation process based on the use history from among a plurality of operation commands for controlling the motion of the target character in the object space. A command is selected, and the selected operation command is presented as an operation command candidate as an operation command that can be selected according to the operator's intention while referring to the audience information.

For example, when the operation command candidate presentation processing unit 214 selects an operation command,
(1) Operation commands frequently used at the same timing in the algorithm,
(2) Operation commands that have been raised at the same timing in the algorithm,
(3) An operation command that defines the behavior of the target character that the majority of the audience has not yet viewed.
Are selected as operation command candidates from a plurality of operation commands, and the selected operation command candidates are displayed on the monitor display unit 290 and presented to the operator.

  The operation command candidate presentation processing unit 214 may simply present the operation commands corresponding to (1) to (3), give a priority to the operation command, and set a preset priority. The presentation order of the operation command candidates selected based on the priority may be determined together with the priority based on other factors such as a value that decreases depending on the degree and the number of selections.

  In addition, when the operation command candidate presentation processing unit 214 causes the operator to select an operation command, for example, the operation commands that have been used are not presented sequentially, or the priority is lowered, and the operation that can be presented or that has a high priority is performed. The command may be displayed on the monitor display unit 290 as a selected operation command candidate and presented to the operator. However, the operation command candidate presentation processing unit 214 is configured to exclude the same operation command in the same audience for the operation commands for some spectators and individual spectators.

  Furthermore, when the operation command candidate presentation processing unit 214 causes the operator to select an operation command, the operation command candidate presentation processing unit 214 displays information such as the number of times of use during the screening and the use timing on the monitor display unit 290 together with the corresponding operation command, and notifies the operator May be presented.

On the other hand, the operation command candidate presentation processing unit 214 corresponds to the second operation command candidate presentation process based on audience information and usage history from among a plurality of operation commands for controlling the motion of the target character in the object space. A plurality of operation commands are selected, and the selected operation commands are presented as operation command candidates.

  Further, the operation command candidate presentation processing unit 214 has a plurality of operation commands (for example, the operation commands (1) to (3) described above) as the operation command candidate presentation processing, as in the first operation command candidate presentation processing. The operation command is further selected according to the audience information from the selected operation command, and the selected operation command is displayed as an operation command candidate on the monitor display unit 290 and presented to the operator.

  Similar to the first operation command candidate presentation process, the operation command candidate presentation processing unit 214 provides information such as the number of times of use and use timing together with the corresponding operation command when the operator selects an operation command being shown. It may be displayed on the monitor display unit 290 and presented to the operator.

[4.5] Other Functions and Processing Next, operation command candidate presentation processing based on the operation command usage history of the present embodiment will be described.

(Performance control of acting characters for a specific audience)
The touch detection processing unit 211 is linked with viewing position information indicating the viewing position of each spectator displayed on the monitor display unit 290, for example, and specifies the viewing position of a specific spectator who enjoys the action of the acting character together with the operation command. Audience designation instructions may be accepted.

  In this case, the content screening execution control unit 218 controls the action of the acting character based on the received operation command toward the audience viewing position specified by the accepted audience designation instruction.

  For example, the content screening execution control unit 218 moves toward the specified viewing position by the acting character's hand gesture control, line-of-sight control (control to send a line of sight), and approaches the closest position on the stage (to approach the large display device 12). And control each action of the call by a given line (eg, the name of the audience if the audience information is known).

  In addition, the monitor display control unit 213 causes the monitor display unit 290 to display the position (for example, seat) of each spectator in the exhibition hall as viewing position information so that a touch operation can be performed. Then, when the audience position displayed by the touch detection processing unit 211 is touched and accepted as an audience designation instruction, the content screening execution control unit 218 outputs an operation command received toward the accepted audience position. Control the action of the acting character based.

  Note that the audience information detection unit 250 may detect the audience information at this time and register it in the content data storage unit 274 or the server device 11 as usage history information.

  In the present embodiment, the audience viewing position is received by the touch operation of the touch detection processing unit 211. However, instead of the touch operation, the content screening execution control unit 218 uses a trackball or a keyboard. The viewing position may be received by moving the cursor.

(Special operation command)
The operation command candidate presentation processing unit 214 may include an operation command (that is, a special operation command) determined according to the action state of the acting character as an operation command candidate and present it to the operator.

For example, the operation command candidate presentation processing unit 214
(1) At the start of the presentation of the content (that is, at the start of the operation by the algorithm), an operation command (an operation command for carrying out a determined pose) for controlling the action for exciting the audience is presented.
(2) At the end of the screening of the content, an operation command (an operation command for executing a continuous call) for controlling the action to maintain and keep the audience active is presented.
(3) For a quiet scene such as between songs, an operation command specialized for the scene such as “Serif” may be presented as an operation command candidate.

  The operation command candidate presentation processing unit 214 may present only a special operation command as an operation command candidate.

  On the other hand, the operation command candidate presentation processing unit 214 is performing the screening obtained by analyzing the content screening result information collected from each content screening system 10 for each content by the server device 11 (specifically, the content management unit 102). It is also possible to acquire an optimal operation command at a given timing in the content of the content, and present the acquired optimal operation command as a special operation command included in operation command candidates.

(Presentation of operation command candidates based on elapsed time or number of receptions)
The operation command candidate presentation processing unit 214 changes the type of operation command to be presented as an operation command candidate according to the elapsed time from the start of content presentation, according to the number of received operation commands, or according to a combination of already selected operation commands. You may let them.

That is, the operation command candidate presentation processing unit 214
(1) When the screening end time approaches, an operation command that defines the operation toward the ending is displayed as an operation command candidate on the monitor display unit 290 and presented to the operator.
(2) When the number of operation command receptions increases, an operation command with a short operation time is displayed on the monitor display unit 290 and presented to the operator in order to adjust the content screening time.
(3) According to a combination of operation commands that have already been selected, an operation command that matches the contents may be displayed on the monitor display unit 290 and presented to the operator.

  In addition, for each operation command, when the action control of the acting character is being executed according to the algorithm, the acceptance permission information indicating the time and interval for which the acceptance is permitted and the action character action control are executed based on each operation command. Execution time may be defined, and the presentation of operation command candidates may be executed using these pieces of information.

  Specifically, for each operation command, the reception permission information includes reception indicating the time (elapsed time) at which the operation command is permitted to be received from the performance start timing (image generation start timing) of the acting character according to the algorithm. When the image generation is performed with a permissible completion time or a plurality of given sections (in the case of music, in the case of a prelude, A melody, chorus, or theatrical performance, act 1 or act 2) And section information indicating a section in which acceptance of the operation command is allowed.

  The operation command candidate presentation processing unit 214 then presents an operation to be presented in accordance with the elapsed time from the image generation start timing of the acting character or the section to which the operation command is presented, and the execution time of each operation command. You may present the operation command which determined the type of command or changed the content of the action control of an acting character.

  In other words, the operation command candidate presentation processing unit 214 determines the elapsed time from the start timing, or each part (section) such as a prelude, A melody, B melody, or chorus when the operation control is a live performance in music. It is possible to determine which part of the image is displayed and present the operation command, and depending on the situation, it is also possible to change the content of motion control of the target character in the operation command such as time shortening and simplification of motion content It has become.

(Multiple operation command batch selection processing)
The content screening execution control unit 218 receives a plurality of operation commands by the touch detection processing unit 211 at the same timing or during a predetermined period assumed as the same timing (for example, 10 seconds or 20 seconds). In such a case, the movement of the target character to be displayed may be controlled in accordance with a given order (for example, the order of input) of the accepted operation commands.

  That is, the operation command candidate presentation processing unit 214 outputs each operation command at each timing when the continuous operation is started during the screening, for example, when it is necessary to input a plurality of operation commands in the linked operation in the performance. In the case of presenting, selecting, and accepting, not only the operation of the operator becomes complicated, but also the operation command candidate presentation processing itself is burdened and may not be accurately processed.

  Therefore, the operation command candidate presentation processing unit 214 may execute an operation command candidate presentation process at each timing and accept a plurality of operation commands in order to select a plurality of operation commands in advance.

  Note that the operation command candidate presentation processing unit 214 may accept the operation command at each timing by the operation command candidate presentation process before the content is shown.

(Edit operation commands)
The content editing processing unit 217 has a configuration for editing the motion control of the target character corresponding to each operation command based on an instruction from the operator.

  Specifically, the content editing processing unit 217 adds, removes a part of the acting character's motion, changes the speed at which the motion is performed, or changes the speed of the operation command designated as the editing target based on the operator's instruction, or Then, the operation contents of other operation commands are integrated.

  For example, when the action of the acting character in the operation command A is only the determined pose, the content editing processing unit 217 adds a movement to take a predetermined step after the determined pose and an action to emit a predetermined line, and then determine the determined pose. , Step, and generate an operation command A that defines the action of a series of acting characters that utter words.

  Then, when the content screening execution control unit 218 receives the operation command A, the acting character controls the motion of the series of acting characters that make a determined pose, take a step, and generate a speech.

  Further, the content editing processing unit 217 may adjust the motion speed while maintaining the motion control of the acting character in the operation command A. For example, the content editing processing unit 217 may edit the speed so that a determined pose with a series of movements executed in 10 seconds is executed in 20 seconds.

(Operation command operation start display)
The operation command candidate presentation processing unit 214 prevents the operator from forgetting to input and improves the operability for the operator. The operation command received at this timing during the execution of the operation command candidate presentation process according to the algorithm The motion start time at which motion control of the acting character is started may be presented based on this.

(Random presentation of operation commands)
In the operation command candidate presentation processing unit 214, in the operation command candidate presentation process, in order to prevent unexpectedness and patterning, the operation command candidate presentation processing unit 214 includes a plurality of operation commands presented by the first operation command presentation process or the second operation command presentation process. Some operation commands may be presented as operation command candidates at random.

(Control of multiple acting characters)
When performing the motion control based on the operation command when controlling the motion of a plurality of acting characters, the content screening execution control unit 218 accepts the operation command for one acting character and based on the operation command. The motion control process for controlling the motion is executed, and the motion control defined in the algorithm is executed for the other acting characters.

  Further, in this case, the operation command candidate presentation processing unit 214 performs a presentation for causing the acting character operator who executes the motion control process based on the operation command to select, and the content screening execution control unit 218 is selected. The action control process based on the operation command is executed for the acting character.

  Further, the operation command candidate presentation processing unit 214 operates each of a plurality of operators (for example, two operators) when controlling the motions of a plurality of acting characters (for example, two acting characters). Command candidate presentation processing may be executed.

  In this case, the operation command candidate presentation processing unit 214 presents, as operation command candidates, operation commands that match the control period of the motion control process based on the operation command so that the motion control of each acting character does not fail. The process for receiving each operation command is executed by the timing.

  In this case, the content screening execution control unit 218 executes motion control of each acting character from the same predetermined timing.

(Reception processing of other operation commands during operation control based on operation commands)
The operation command candidate presentation processing unit 214 receives another operation command and controls the received operation command (hereinafter referred to as “second operation command”) during the motion control of the acting character based on the received first operation command. The operation command candidates may be presented so that the operation control based on.

  For example, when the action control of the acting character is executed based on the first operation command and the audience information indicating that the audience is not excited is presented, or the first The operation command candidate presentation processing unit 214 is selected according to the audience information in a special situation such as a case where there is a delay in the entire performance performance time when the action character movement control by the operation command is performed to the end. At least one or more operation commands are presented as operation command candidates.

Then, the content screening execution control unit 218 performs the first operation command and the first operation command by the touch detection processing unit 211 during the action control of the acting character based on the first operation command (hereinafter referred to as “first action control”). When a different second operation command is accepted, the operation is stopped when the first action control is accepted, and the action control of the acting character based on the second action command (hereinafter, “second action control”). Is called).

  In addition, when canceling the first motion control, the content screening execution control unit 218 executes the control up to a motion with a good break in the performance of the acting character, and then executes the control to stop.

  In addition, the operation command candidate presentation processing unit 214 sets an operation command different from the first operation command when the first operation command is presented as the second operation command candidate (that is, an operation command that has not been accepted). ) May be presented (the state where the first operation command is presented may be maintained), or a predetermined operation such as according to audience information or according to usage history information of the operation command. A new command may be presented.

[4.6] Modification (Modification 1)
In the above embodiment, the operation command candidate presentation processing unit 214 displays the operation command candidates on the monitor display unit 290 so as to be able to perform a touch operation. Good.

  In this case, the operation command candidate presentation processing unit 214 may present the switch and each key assigned to the monitor display unit 290, or may directly present the switch and each key.

(Modification 2)
In the above embodiment, the touch detection processing unit 211 specifies the viewing position based on a touch operation on the monitor display unit 290, but the monitor display unit 290 is operated by an operator such as a joystick or a keyboard. The viewing position may be designated by manipulating the cursor displayed on the screen or the indicated position display.

(Modification 3)
In the above embodiment, the content is shown to the audience who has visited the event venue. However, in the above embodiment, the real time video that distributes the video of the content using a network or by television broadcasting is used. It can also be applied in distribution.

  In this case, the content screening execution control unit 218 controls the action of the acting character based on the algorithm or the operation command, and the communication control control unit (not shown) uses the virtual camera in the object space including the acting character as a viewpoint. A data stream is generated from the image generated by the drawing unit 220, and the generated data stream is distributed to the server device 11 or a terminal device (not shown) via the communication unit 296 using a network or by broadcast waves.

(Modification 4)
In the above-described embodiment, as a performance object (one aspect of the character) instead of the acting character, for example, a special effect device such as a bazooka type or cannon type cracker that blows decorations such as confetti or tape, a pin spot type Lighting devices such as light and search light, stage devices such as active flower paths and seri, sound devices such as speakers, or flying devices in which stage objects fly over the audience seats, and the like.

That is, the content screening system 10
(1) Control the operation of the production object placed in the real space to be viewed by the audience,
(2) An operation command selected according to audience information indicating information related to the audience from among a plurality of operation commands for controlling the motion of the effect object is presented to the operator as an operation command candidate. ,
(3) Accepting an operation command instructed by the operator from the presented operation command candidates,
(4) controlling the operation of the effect object based on the received operation command;
You may have a structure.

  Moreover, in the said embodiment, although the operation command which prescribes | regulates the action control of an acting character is shown, the action control of the virtual camera at the time of imaging the object space containing the said acting character as one form of object character is carried out. A prescribed operation command may be presented.

  For example, the operation command candidate presentation processing unit 214 selects an operation command selected from among a plurality of operation commands for controlling the operation of the virtual camera according to audience information indicating information regarding the audience by the operator. As a candidate, it is presented to the operator, and the content screening execution control unit 218 may control the operation of the virtual camera based on the received operation command.

(Modification 5)
In the above embodiment, the content screening system 10 executes various processes in conjunction with the server device 11, but registration and analysis of usage history information such as past audience information and operation commands, and content screening are performed. Each process executed by the server device 11 such as acquisition of result information may be executed, and content screening may be executed as a stand-alone device.

(Modification 6)
In the above embodiment, the content is displayed on the large display device 12 to allow the audience to view the content. However, the HMD (wearable image display device) that allows each audience to visually recognize the image in the virtual three-dimensional space. The VR performance (virtual reality experience) acting character that is recognized as if it exists around each spectator in the virtual space may be viewed in 3D.

  In this case, for example, a terminal device having an HMD unit and a control unit is provided for each audience, and each terminal device provides images and sounds to the HMD based on various data output from the control unit. .

  In addition, each terminal device has a configuration capable of detecting the sound, line of sight, or posture (head posture) of the spectator attached as spectator information.

  Each terminal device may be provided to a plurality of spectators at the same event venue and simultaneously perform the content screening, or may execute the content screening at different timings in different spaces.

[5] Operation in the present embodiment Next, the operation of the operation control process based on the operation command including the operation command candidate presentation process executed by the system control apparatus 20 of the present embodiment will be described with reference to FIGS. 10 and 11. . 10 and 11 are flowcharts showing the operation of the operation control process based on the operation command including the operation command candidate presentation process executed by the system control apparatus 20 of the present embodiment.

(In the case of the first operation command candidate presentation process)
In this operation, it is assumed that the action of the acting character arranged in the object space is executed by an algorithm of predetermined content, and viewing information of each spectator is acquired from the server device 11 and already stored in the content data storage unit 274. It is assumed that

  Further, in this operation, the virtual camera set in the object space is controlled according to the algorithm, and images of acting characters in the object space from the virtual camera as a viewpoint are sequentially generated and displayed on the large display device 12. Shall.

  First, when the content screening execution control unit 218 detects that the operation command reception timing is based on the algorithm (step S101), the operation command candidate presentation processing unit 214 works in conjunction with the monitor display control unit 213 to store content data. The spectator information of the spectators who have visited the event venue stored in the unit 274 is acquired and displayed on the monitor display unit 290 (step S102).

  At this time, the content screening execution control unit 218 causes the timer unit 240 to start measuring the remaining time (input time limit) until the acting character starts moving based on the operation command.

  Note that the content screening execution control unit 218 may acquire spectator information of spectators who have visited the event site directly from the server device 11.

  Next, the operation command candidate presentation processing unit 214 works in conjunction with the monitor display control unit 213 to select an operation command selected according to the audience information displayed by the operator from among a plurality of operation commands predetermined by the algorithm. The operation command candidate is displayed on the monitor display unit 290 and presented to the operator (step S103).

  At this time, the operation command candidate presentation processing unit 214 is interlocked with the monitor display control unit 213, and the remaining time until the operation start time at which the action control of the acting character is started based on the operation command received at the timing (input restriction) Time) is displayed on the monitor display unit 290 together with operation command candidates and spectator information.

  Next, the content screening execution control unit 218 determines whether or not the input restriction time has become “0” (step S104), and when it is determined that the input restriction time has become “0”, it is a target. For the acting character, execution of motion control defined in advance by the algorithm is started (step S105), and the motion is terminated.

  On the other hand, when the content screening execution control unit 218 determines that the input restriction time has not yet become “0”, the touch detection processing unit 211 determines whether or not the operation command instructed by the operator is detected. If it is determined (step S106) and it is determined that the operation command is not detected, the process returns to step S104.

  Further, when it is determined that the operation command instructed to be operated by the operator is detected, the touch detection processing unit 211 receives the detected operation command (step S107).

  Next, the content screening execution control unit 218 starts the motion control of the target acting character based on the received operation command (step S108) and ends the motion.

(In the case of the second operation command candidate presentation process)
In this operation, it is assumed that the action of the acting character arranged in the object space is executed by an algorithm of predetermined content, and viewing information of each spectator is acquired from the server device 11 and already stored in the content data storage unit 274. It is assumed that

  Further, in this operation, the virtual camera set in the object space is controlled according to the algorithm, and images of acting characters in the object space from the virtual camera as a viewpoint are sequentially generated and displayed on the large display device 12. Shall.

  First, when the content screening execution control unit 218 detects that the operation command is received based on the algorithm (step S201), the spectator information detection unit 250 detects the spectator information of the spectators who have visited the event venue. (Step S202).

  At this time, even if the content screening execution control unit 218 controls the acting character to execute predetermined motion control such as output of predetermined speech to the audience, and detects the audience information of the audience based on the motion control. Good.

  At this time, the content screening execution control unit 218 causes the timer unit 240 to start measuring the remaining time (input time limit) until the acting character starts moving based on the operation command.

  Next, the operation command candidate presentation processing unit 214 analyzes the current audience state based on the detected audience information (step S203).

  Next, the operation command candidate presentation processing unit 214 works in conjunction with the monitor display control unit 213 to select an operation command as a candidate operation command from among a plurality of operation commands determined in advance by an algorithm according to the detected audience information. The information is displayed on the monitor display unit 290 and presented to the operator (step S204).

  At this time, the operation command candidate presentation processing unit 214 is interlocked with the monitor display control unit 213, and the remaining time until the operation start time at which the action control of the acting character is started based on the operation command received at the timing (input restriction) Time) is displayed on the monitor display unit 290 together with operation command candidates and spectator information.

  Next, the content screening execution control unit 218 determines whether or not the input restriction time has become “0” (step S205), and if it is determined that the input restriction time has become “0”, the content screening execution control unit 218 becomes a target. For the acting character, execution of motion control defined in advance by the algorithm is started (step S206), and the motion is terminated.

  On the other hand, when the content screening execution control unit 218 determines that the input restriction time has not yet become “0”, the touch detection processing unit 211 determines whether or not the operation command instructed by the operator is detected. If it is determined (step S207) and it is determined that the operation command is not detected, the process returns to step S205.

  If the touch detection processing unit 211 determines that the operation command instructed by the operator is detected, the touch detection processing unit 211 receives the detected operation command (step S208).

  Next, the content screening execution control unit 218 starts the motion control of the target acting character based on the received operation command (step S209) and ends the motion.

[6] Others The present invention is not limited to that described in the above embodiment, and various modifications can be made. For example, terms cited as broad or synonymous terms in the description in the specification or drawings can be replaced with broad or synonymous terms in other descriptions in the specification or drawings.

  In the present embodiment, various processes are executed in one content screening system 10, but the content screening system 10 may be configured by linking a plurality of devices.

  Further, in the present embodiment, processing related to various types of content is executed by the system control device 20 of the content screening system 10, but the above-described functions and content programs of the server processing unit 100 of the server device 11 are executed. It may be realized. Further, each process may be executed by the content screening system 10 alone without being linked to the server device 11.

  The present invention includes configurations that are substantially the same as the configurations described in the embodiments (for example, configurations that have the same functions, methods, and results, or configurations that have the same objects and effects). In addition, the invention includes a configuration in which a non-essential part of the configuration described in the embodiment is replaced. In addition, the present invention includes a configuration that exhibits the same operational effects as the configuration described in the embodiment or a configuration that can achieve the same object. In addition, the invention includes a configuration in which a known technique is added to the configuration described in the embodiment.

  As described above, the embodiments of the present invention have been described in detail. However, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. . Accordingly, all such modifications are intended to be included in the scope of the present invention.

10: Content screening system 11: Server device 12: Large display device 20: System control device 100: Server processing unit 101: Communication control unit 102: Content management unit 103: Audience management unit 120: Input unit 130: Display unit 140: Storage Unit 142: Main storage unit 144: Content data 146: Audience information 148: Content collection data 180: Information storage medium 196: Communication unit 200: Processing unit 210: Object space setting unit 211: Touch detection processing unit 212: Content display control unit 213: Monitor display control unit 214: Operation command candidate presentation processing unit 215: Movement processing unit 216: Virtual camera control unit 217: Content editing processing unit 218: Content screening execution control unit 219: Content management unit 220: Drawing unit 230: Sound Processing unit 240: Thailand Part 250: Audience information detection part 260: Input part 262: Detection part 270: Storage part 271: Main storage part 272: Image buffer 273: Audience information 274: Content data storage part 280: Information storage medium 290: Monitor display part 292 : Sound output unit 296: Communication unit

Claims (16)

An operation control system for controlling operation of an imaged target character to be viewed by a spectator,
Image generating means for setting a virtual camera in the object space in which the target character is arranged, and generating an image of the target character in the object space with the virtual camera as a viewpoint;
Audience display control means for displaying an image of the target character on the audience display device visually recognized by the audience;
An operation command selected by the operator according to spectator information indicating information related to the spectator from among a plurality of operation commands for controlling the movement of the target character in the object space is set as an operation command candidate to the operator. Presenting means to present;
Receiving means for receiving an operation command instructed by the operator from among the presented operation command candidates;
Action control means for controlling the action of the displayed target character based on the accepted operation command;
An operation control system comprising: The operation control system according to claim 1,
A monitor display control means for displaying the target character on a monitor display device visually recognized by the operator;
The presenting means is
An operation control system for presenting the operation command candidates to the operator together with the target character on the monitor display device. In the operation control system according to claim 1 or 2,
The presenting means is
An operation control system that acquires the audience information and presents the acquired audience information together with the operation command candidates to the operator. In the operation control system according to claim 3,
Management means for managing storage means for storing personal information of each audience as the audience information for each audience,
The presenting means is
An operation control system that acquires the stored audience information and presents the acquired audience information together with the operation command candidates to the operator. In the operation control system according to any one of claims 1 to 4,
It further comprises registration means for registering and accumulating information of operation commands accepted when the motion control of the target character is executed in the storage means as usage history information,
The presenting means is
An operation control system that presents an operation command selected by the operator as the operation command candidate to the operator according to the audience information and the registered use history information. The operation control system according to claim 5,
The usage history information includes spectator information used when the operation command and the operation command are received,
The presenting means is
Presenting an operation command selected by the operator as the operation command candidate to the operator according to a given analysis result in the operation command based on the usage history information and the audience information; Operation control system. In the operation control system according to any one of claims 1 to 6,
It further comprises detection means for detecting the state of each audience as the audience information,
The presenting means is
An operation control system that presents an operation command selected by the operator according to the detected audience information as an operation command candidate to the operator. In the operation control system according to any one of claims 1 to 7,
The accepting means is
Along with the operation command, accept a spectator designation instruction for designating a specific spectator who enjoys the motion of the target character,
The operation control means is
An operation control system for controlling an action of the displayed target character toward an audience specified by the accepted audience designation instruction. In the operation control system according to any one of claims 1 to 8,
The presenting means is
An operation control system that presents to the operator an operation command determined according to the movement status of the target character, included in the operation command candidates. In the operation control system according to any one of claims 1 to 9,
The presenting means is
The presentation according to the elapsed time from the start timing of image generation in the target character, the number of times operation commands received during the image generation, or a combination of the types of operation commands received during the image generation An operation control system that changes the type of operation command to be executed. In the operation control system according to any one of claims 1 to 9,
While following a given algorithm and imaging the motion of the target character placed in the object space,
For each operation command, a reception completion allowable time indicating a time at which reception of the operation command is permitted from the start timing of image generation in the target character or a plurality of given sections are determined, and the image generation is executed. Section information indicating a section in which the operation command is allowed to be received and an execution time during which the target character's motion control is executed based on each operation command are defined,
The presenting means is
The type of the operation command to be presented is determined according to the elapsed time from the image generation start timing in the target character or the section to which the operation command is presented and the execution time of each operation command. Or the operation control system which presents the operation command which changed the content of the motion control of the said object character. In the operation control system according to any one of claims 1 to 11,
The operation control means is
An operation control system for starting motion control of the displayed target character based on the received operation command from a predetermined timing when the operation command is received during image generation on the target character. In the operation control system according to any one of claims 1 to 12,
The presenting means is
An operation control system that presents a motion start time at which motion control of the target character is started or a time related to the motion start time based on an operation command received at the timing while presenting the operation command candidate. In the operation control system according to any one of claims 1 to 13,
The operation control means is
While following a given algorithm, when the motion of the target character placed in the object space is imaged, and the operation command is received, the algorithm after receiving the operation command, An operation control system that changes according to an accepted operation command and controls the movement of the displayed target character according to the changed algorithm. A program for operating and controlling the movement of an imaged target character to be viewed by a spectator,
Image generating means for setting a virtual camera in the object space in which the target character is arranged, and generating an image of the target character in the object space with the virtual camera as a viewpoint;
Audience display control means for displaying an image of the target character on the audience display device visually recognized by the audience;
An operation command selected by the operator according to spectator information indicating information related to the spectator from among a plurality of operation commands for controlling the movement of the target character in the object space is set as an operation command candidate to the operator. Presenting means to present,
An accepting means for accepting an operation command instructed by the operator from among the presented operation command candidates; and
Action control means for controlling the action of the displayed target character based on the accepted operation command;
A program characterized by causing a computer to function. An operation control system for operating and controlling the motion of the imaged target character to be viewed by the audience;
An audience display device visually recognized by the audience;
With
The operation control system is
Image generating means for setting a virtual camera in the object space in which the target character is arranged, and generating an image of the target character in the object space with the virtual camera as a viewpoint;
Spectator display control means for displaying an image of the target character on the spectator display device;
An operation command selected by the operator according to spectator information indicating information related to the spectator from among a plurality of operation commands for controlling the movement of the target character in the object space is set as an operation command candidate to the operator. Presenting means to present,
Receiving means for receiving an operation command instructed by the operator from among the presented operation command candidates;
Action control means for controlling the action of the displayed target character based on the accepted operation command;
A character screening system comprising:

Images