JP2022152443A - Information processing system, information processing method and computer program - Google Patents

Abstract

To provide an information processing system, information processing method and computer program capable of providing, in a virtual space, a content including a special value added thereto, the value that is difficult to realize in a real space.SOLUTION: Each of one or a plurality of computer processors provided in an information processing system according to the present invention includes a first display processing unit displaying an image that displays on a display screen, an avatar object of a user and a first object in a virtual space, a moving processing unit for moving the avatar object in the virtual space in response to the user's operation, a determination unit for determining whether or not a relation of the avatar object and the first object satisfies a first condition, and a second display processing unit when it is determined to satisfy the first condition by the determination unit, displaying an image of the real space on the display screen.SELECTED DRAWING: Figure 17

Description

The present invention relates to an information processing system, an information processing method, and a computer program.

In recent years, techniques for displaying a user's avatar in virtual space or real space have been actively developed.

As an example, Non-Patent Document 1 discloses holding an event in which users display their own avatars in a virtual space and enjoy content such as provided moving images.

Reissue Records Co., Ltd., “FORTNITE will hold a virtual event on August 7th!!”, [online], July 31, 2nd year of Reiwa, [searched on February 8th, 3rd year of Reiwa], Internet <https: //reissuerecords.net/live/＞

Content that could only be realized in the real space in the past is now being provided as content in the virtual space.

Therefore, an object of the present invention is to provide an information processing system, an information processing method, and a computer program capable of providing, in a virtual space, content with special added value that is difficult to realize in the real space. It is in.

An information processing system of the present disclosure is an information processing system for displaying an image of a virtual space on a display screen of a first information processing device, and includes one or more computer processors, wherein the one or more computer processors a first display processing unit for displaying an image in which the user's avatar object and the first object are displayed on the display screen in the virtual space; and movement for moving the avatar object in the virtual space according to the user's operation. a processing unit; a determination unit that determines whether the relationship between the avatar object and the first object satisfies the first condition; and a second display processing unit that displays an image of the physical space.

The second display processing unit displays a menu image for selecting one real space image from a plurality of real space images on the display screen when the determination unit determines that the first condition is satisfied. The real space image of the position to be displayed on the display screen can be determined according to the selection by the user.

The second display processing unit displays, as an image of the physical space, a first image that is a real-time image of an object placed in the physical space captured by a second information processing device that is different from the first information processing device. be able to.

The object includes an operation unit that can be remotely operated by the user, and the user can obtain a prize placed inside the object.

The target object is a crane game machine, the operation part is a crane part, and the first image can include at least a space of the crane game machine including a prize.

The target object is a coin-operated game machine, the operation part is a coin-dropping part, and the first image can be a space containing at least the medals of the coin-operated game machine.

Predetermined markers are attached to the prizes, and the second display processing unit superimposes the related objects associated with the markers on the real space image based on the markers included in the real space image. can be done.

The one or more computer processors can further include a prize storage unit that stores data of the related object in association with the user when the user successfully acquires the prize.

The first display processing section can display the related object stored in the prize storage section as an attached object of the avatar object displayed in the virtual space in accordance with the user's operation.

The first display processing section can display the related object stored in the prize storage section as a character object in the virtual space or the real space according to the user's operation.

The second display processing section can display the avatar object as the related object.

The one or more computer processors can further comprise an output unit that outputs data of the associated object to the 3D printing device if the user successfully wins the prize.

The second display processing unit further displays a second image, which is a real-time image of an object captured by a third information processing device different from the first information processing device, as an image of the physical space, The information processing device can image the object from the front of the object, and the third information processing device can image the object from the side of the object.

The one or more computer processors further comprise a switching operation receiving unit for switching between the first image and the second image, and the second display processing unit displays an image to be displayed in accordance with the operation received by the switching operation receiving unit. can be switched.

The determination unit determines whether the relationship between the avatar object and the first object satisfies a second condition, and the second display processing unit determines if the second condition is satisfied by the determination unit. You can switch the image to be displayed.

The second information processing device can image the surroundings from inside the object.

The second display processing section can change the displayed image according to the relationship between the avatar object and the first object.

The second display processing unit can display the image of the physical space by superimposing it on a part of the first object or by replacing it with a part of the first object.

The first display processing unit can display a display object that displays information related to a physical space image associated with the first object.

An information processing method of the present disclosure is an information processing method in an information processing system for displaying an image of a virtual space on a display screen of a first information processing device, wherein one or more computer processors included in the information processing system include: a first display processing step of displaying on the display screen an image in which the user's avatar object and the first object are displayed in the virtual space; and a moving processing of moving the avatar object in the virtual space according to the user's operation. a determining step of determining whether the relationship between the avatar object and the first object satisfies the first condition; and if it is determined in the determining step that the first condition is satisfied, the physical space is displayed on the display screen. and a second display processing step of displaying an image of

An information processing method according to the present disclosure is an information processing method for displaying an image of a virtual space on a display screen of a first information processing device, wherein one or more computer processors included in the first information processing device display A first display processing step of displaying an image in which the user's avatar object and the first object are displayed on the screen in the virtual space, and a movement processing step of moving the avatar object in the virtual space according to the user's operation. a determination step of determining whether the relationship between the avatar object and the first object satisfies the first condition; and a second display processing step of displaying an image of

A computer program of the present disclosure is a computer program for displaying an image of a virtual space on a display screen of a first information processing device, wherein one or more computer processors provided in the first information processing device display a a first display processing function for displaying an image in which the user's avatar object and the first object are displayed in the virtual space; a movement processing function for moving the avatar object in the virtual space according to the user's operation; a judgment function for judging whether the relationship between the avatar object and the first object satisfies the first condition; and an image of the real space on the display screen when the judgment function judges that the first condition is satisfied. It is characterized by realizing a second display processing function of displaying the .

According to the invention of the present disclosure, it is possible to provide a technical improvement that solves or alleviates at least part of the above-described problems of the prior art. Specifically, according to the invention of the present disclosure, an information processing system, an information processing method, and a computer program capable of providing, in a virtual space, content with special added value that is difficult to realize in a real space. can be provided.

1 is a system configuration diagram showing an example of an information processing system according to the present disclosure; FIG. FIG. 4 is a system configuration diagram showing another example of an information processing system according to the present disclosure; 1 is a configuration diagram showing an example of a hardware configuration of an information processing device according to the present disclosure; FIG. 1 is a configuration diagram showing an example of a functional configuration of an information processing device according to the present disclosure; FIG. FIG. 3 is an image diagram showing an example of an image of a virtual space in the present disclosure; FIG. FIG. 4 is an image diagram showing an example of a passage in virtual space in the present disclosure; FIG. 4 is an image diagram showing an example of a specific area in virtual space in the present disclosure; FIG. 5 is an image diagram showing another example image of a specific area in the virtual space in the present disclosure; FIG. 4 is an image diagram showing a display example of an image of real space in the present disclosure; FIG. 4 is an image diagram showing a display example of an image of real space in the present disclosure; FIG. 4 is an image diagram showing an example of a menu screen in the present disclosure; 1 is an image diagram showing an example of an object in the present disclosure; FIG. FIG. 4 is an image diagram showing a display example of an image of real space in the present disclosure; 1 is an image diagram showing an example of an object in the present disclosure; FIG. FIG. 4 is an image diagram showing a display example of an image of real space in the present disclosure; FIG. 2 is an image diagram showing an example of a prize in the present disclosure; FIG. FIG. 4 is an image diagram of a display screen displaying an image of a real space according to the present disclosure; FIG. 4 is a configuration diagram showing another example of the functional configuration of the information processing device according to the present disclosure; FIG. 4 is an image diagram showing a display example of AR mode in the present disclosure; FIG. 2 is an image diagram showing an example of a second information processing device and a third information processing device in the present disclosure; FIG. 3 is an image diagram showing an example of an image of a virtual space in the present disclosure; FIG. FIG. 2 is a flow diagram showing an example of the flow of an information processing method according to the present disclosure; 1 is a circuit configuration diagram showing an example of a circuit configuration for realizing a computer program according to the present disclosure; FIG. 1 is a configuration diagram showing an example of a functional configuration of a first information processing device in the present disclosure; FIG. FIG. 2 is a flow diagram showing an example of the flow of an information processing method according to the present disclosure; 1 is a circuit configuration diagram showing an example of a circuit configuration for realizing a computer program according to the present disclosure; FIG.

An example of an embodiment of an information processing system according to the present disclosure will be described with reference to the drawings.

<System configuration>
FIG. 1 shows an example of a system configuration of an information processing system 1000 according to the present disclosure.

As shown in FIG. 1, an information processing system 1000 according to the present disclosure includes an information processing device 400 connected to a first information processing device 100 via a network.

In addition, as shown in FIG. 2 as an example, the information processing system 1000 according to the present disclosure may further include a second information processing device 200 and/or a third information processing device 300 .

In the present disclosure, the first information processing device 100 is a device capable of communicating with the information processing device 400 via a network. The first information processing device 100 includes a wearable computer such as a smartphone (multifunctional telephone terminal), a tablet terminal, a personal computer, a console game machine, a head-mounted display (HMD), a glasses-type wearable terminal (AR glasses, etc.), and Information processing devices other than these devices that can reproduce moving images can be used. These devices may be stand-alone devices that operate independently, or may be composed of a plurality of devices connected to each other so as to be able to transmit and receive various data.

<Hardware configuration>
Here, the hardware configuration of the information processing device 400 included in the information processing system 1000 will be described using FIG. The information processing device 400 includes a processor 401 , a memory 402 , a storage 403 , an input/output interface (input/output I/F) 404 and a communication interface (communication I/F) 405 . Each component is interconnected via a bus B. FIG.

The information processing apparatus 400 can implement the functions and methods described in this embodiment through cooperation of the processor 401, memory 402, storage 403, input/output I/F 404, and communication I/F 405. can.

Processor 401 executes functions and/or methods implemented by codes or instructions contained in programs stored in storage 403 . The processor 401 is, for example, a central processing unit (CPU), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), microprocessor, processor core, multiprocessor, ASIC (Application- Specific Integrated Circuit), FPGA (Field Programmable Gate Array), etc., implemented by logic circuits (hardware) and dedicated circuits formed in integrated circuits (IC (Integrated Circuit) chip, LSI (Large Scale Integration)), etc. Each process disclosed in the form may be implemented. Also, these circuits may be implemented by one or more integrated circuits, and multiple processes shown in each embodiment may be implemented by one integrated circuit. LSIs are also called VLSIs, super LSIs, ultra LSIs, etc., depending on the degree of integration.

The memory 402 temporarily stores programs loaded from the storage 403 and provides a work area for the processor 401 . The memory 402 also temporarily stores various data generated while the processor 401 is executing the program. The memory 402 includes, for example, RAM (Random Access Memory), ROM (Read Only Memory), and the like.

The storage 403 stores programs. The storage 403 includes, for example, a HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, and the like.

The communication I/F 405 is implemented as hardware such as a network adapter, communication software, or a combination thereof, and transmits and receives various data via a network. The communication may be performed by wire or wirelessly, and any communication protocol may be used as long as mutual communication can be performed. A communication I/F 405 communicates with other information processing apparatuses via a network. Communication I/F 405 transmits various data to other information processing apparatuses according to instructions from processor 401 . Communication I/F 405 also receives various data transmitted from other information processing apparatuses and transmits the data to processor 401 .

The input/output I/F 404 includes an input device for inputting various operations to the information processing device 400 and an output device for outputting processing results processed by the information processing device 400 . The input/output I/F 404 may be integrated with the input device and the output device, or may be separated into the input device and the output device.

The input device is realized by any one or a combination of all kinds of devices capable of receiving input from the user and transmitting information related to the input to the processor 401 . The input device includes, for example, hardware keys such as a touch panel, a touch display, and a keyboard, a pointing device such as a mouse, a camera (operation input via images), and a microphone (operation input by voice).

The input device can include a sensor unit. The sensor unit is one or more sensors that detect face motion, which indicates changes in the user's facial expression, and body motion, which indicates changes in the relative position of the user's body with respect to the sensor unit. Face motion includes movements such as blinking and opening and closing the mouth. A well-known thing can be used for the sensor unit. An example of the sensor unit is a ToF sensor that measures and detects the time of flight until the light emitted toward the user is reflected by the user's face and returns, a camera that captures the user's face, and It includes an image processing unit that processes the data captured by the camera. Also, the sensor unit may include an RGB camera that captures visible light and a near-infrared camera that captures near-infrared light. RGB cameras and near-infrared cameras use, for example, the True Depth of “iPhoneX (registered trademark)”, or “LIDER” of “iPad Pro (registered trademark)”, and other ToF sensors installed in smartphones. It is possible. Specifically, this camera projects tens of thousands of invisible dots onto the user's face or the like using a dot projector. The reflected light of the dot pattern is then detected and analyzed to form a depth map of the face, and an infrared image of the face or the like is captured to capture accurate face data. The arithmetic processing unit of the sensor unit generates various information based on the depth map and the infrared image, compares the information with registered reference data, and determines the depth of each point on the face (each point and the near-infrared camera). (distance between ) and position deviation other than depth.

Also, the sensor unit may have a function of tracking not only the face of the user but also the hand (hand tracking). The sensor unit may further include sensors other than the above sensors, such as acceleration sensors and gyro sensors. The sensor unit may have a spatial mapping function of recognizing an object in the real space where the user exists based on the detection results of the ToF sensor and other known sensors and mapping the recognized object on a spatial map. . In the following description, face motion detection data and body motion detection data will simply be referred to as "tracking data" when they are not distinguished from each other. The image processing section of the sensor unit may be included in the control section that can be included in the information processing system of the present invention.

An operation unit as an input device can be used according to the type of user terminal. Examples of the operation unit include a touch panel integrated with a display, operation buttons, a keyboard, a mouse provided on a housing of the user terminal, a controller manually operated by the user, and the like. The controller may incorporate various known sensors such as an acceleration sensor and an inertial measurement sensor (IMU: Inertial Measurement Unit) such as a gyro. Further, another example of the operation unit may be a tracking device that identifies the user's hand movement, eye movement, head movement, line of sight direction, and the like. In this aspect, for example, based on the movement of the user's hand, the user's instruction is determined, the distribution of the video is started or ended, the message or video is evaluated, and a predetermined object (for example, a gift described later) is displayed. Various operations such as requests can be executed. Note that if the sensor unit also has an input interface function such as a hand track function, the operation section can be omitted.

The output device outputs the processing result processed by the processor 401 . The output device includes, for example, a display device, a touch panel, a speaker, and the like.

In addition, the first information processing apparatus 100 in the present disclosure can also be configured with the same hardware configuration as in FIG. 2, except for special cases. On the other hand, the second information processing device 200 and the third information processing device 300 may have at least a communication unit and an imaging unit for communicating with the information processing device 400, but may have functions beyond these. may

All of the one or more computer processors included in the information processing system 1000 according to the present disclosure may be included in the information processing device 400 or the first information processing device 100, or may be included in the information processing device 400 and the first information processing device. The processing device 100 may share the responsibility. Hereinafter, unless otherwise specified, the information processing apparatus 400 will be described as including all of one or a plurality of computer processors for realizing functions described later.

<Functional configuration>
The information processing system 1000 according to the present disclosure is for displaying a virtual space image V100 on the display screen of the first information processing apparatus 100 .

A virtual space realized by the information processing system 1000 is, for example, a space like an arcade where arcade game machines are arranged. In this space, at least a plurality of game machine objects imitating arcade game machines shall be arranged.

It is also assumed that one or more avatar objects that can be operated by each user are displayed in the virtual space. Note that the avatar object may include an object operated by a computer.

In the following explanation, the virtual space will be described as a space that imitates a game arcade, but it can also be a virtual space that imitates any real space, such as a movie theater, shopping mall, karaoke, or school. is.

One or more computer processors included in the information processing system 1000 include a first display processing unit 410, a movement processing unit 420, a determination unit 430, and a second display processing unit 440, as shown in FIG.

The first display processing unit 410 causes the display screen to display an image in which the user's avatar object and the first object are displayed in the virtual space.

FIG. 5 is a diagram showing an image of a virtual space image V100 in which a user's avatar object V101 and a first object V102 are displayed on the display screen of the first information processing apparatus 100. As shown in FIG.

The avatar object V101 is at least an object that can be operated by the user. Also, the avatar object may reflect the user's facial expression and/or actions. Such facial expressions and/or actions of the user are assumed to be acquired by the input device included in the first information processing apparatus 100, and are reflected in the avatar object using a known technique.

The first object V102 is, for example, a game machine object imitating an arcade game machine. Examples of arcade game machines include game machines such as crane game machines and pusher game machines.

Note that the shape of the first object V102 is not particularly limited as long as it can serve as a criterion for determining the first condition, which will be described later.

The movement processing unit 420 moves the avatar object V101 in the virtual space according to the user's operation.

It is assumed that the user's operation is performed via the input device. As an example, if the first information processing apparatus 100 is a device having a touch panel such as a smart phone or a tablet terminal, the user can move the avatar object V101 by performing a touch operation on the touch panel.

Also, if the first information processing apparatus 100 is an apparatus having a controller such as a personal computer or a console game machine, the user can move the avatar object V101 by operating the controller.

Also, if the first information processing apparatus 100 is a wearable computer such as an HMD or AR glasses, the user can move the avatar object V101 by moving the line of sight or the head.

It should be noted that it is possible to set restrictions on the area in which the avatar object V101 can move. As an example, as shown in FIG. 6, the avatar object V101 may be movable only in the area of a passage V103 set in the virtual space. Note that FIG. 6 is an image diagram of a bird's-eye view of the virtual space, and the avatar object V101 is set so as to be able to move along the passage V103.

The determination unit 430 determines whether the relationship between the avatar object V101 and the first object V102 satisfies a first condition.

The first condition can be, for example, a condition based on the positional relationship between the avatar object V101 and the first object V102.

Specifically, the determining unit 430 calculates the distance between the avatar object V101 and the coordinate position of the first object V102 in the virtual space based on the coordinate position of the avatar object V101, and determines if the distance is equal to or less than a predetermined value. If so, it can be determined that the first condition is satisfied. The distance used for determination may be the distance between the center coordinates of these objects, or the shortest distance between the surface coordinates of these objects.

Alternatively, the determination unit 430 can determine that the first condition is satisfied when the coordinate position of the avatar object V101 in the virtual space enters a specific area set around the first object V102. .

The specific area can be a square area V104 in front of the first object V102 as shown in FIG. 7, or a semi-circular area V104 in front of the first object as shown in FIG. The shape of these specific regions is not particularly limited.

As an alternative to determining the first condition, the determining unit 430 can determine that the first condition is satisfied in response to the user's selection of the first object V102.

Then, when the determination unit 330 determines that the first condition is satisfied, the second display processing unit 440 displays the image of the physical space on the display screen.

The image of the physical space can be, for example, a still image or a moving image of the physical space that is the target modeled by the virtual space, details of which will be described later.

The image of the real space can be displayed in place of or superimposed on at least part of the virtual space displayed on the display screen.

The image of the physical space may be displayed as part of the image V100 of the virtual space displayed on the display screen, or may be displayed as a screen separate from the screen displaying the image V100 of the virtual space. It may be

FIG. 9 shows an example in which an image R100 of the real space is embedded in a part of the image V100 of the virtual space displayed on the display screen. This is an example in which an image R100 is displayed on the display screen. Note that in FIG. 10, the image R100 in the physical space is displayed superimposed on the image V100 in the virtual space.

According to the above configuration, it is possible to provide a technical improvement that solves or mitigates at least part of the above-described problems of the prior art. Specifically, according to the above configuration, it is possible to provide, in the virtual space, content with a special added value that is difficult to achieve in the real space.

In particular, by displaying the image of the real space in the image of the virtual space under predetermined conditions, content with special value that cannot be realized with only one of the real space and the virtual space can be created. It is possible to generate.

Next, the details of the display of the image R100 of the physical space will be described.

The second display processing unit 440 can display the menu screen on the display screen when the determination unit 430 determines that the first condition is satisfied.

This can be done before displaying the above-described physical space image R100.

The menu screen is for selecting one physical space image R100 from a plurality of physical space images.

FIG. 11 shows an example of the menu screen M100. A plurality of item objects M101 are displayed on the menu screen M100, and each of the item objects is associated with one physical space image. Also, different item objects may be associated with the same physical space image.

Then, the second display processing unit 440 can determine one image R100 of the physical space to be displayed on the display screen according to the selection by the user.

With this configuration, it is not necessary to arrange the first object V102 for each image R100 in the physical space, so it is possible to reduce the drawing load and save space in the virtual space.

According to the above configuration, one first object V102 is sufficient. It is also possible to display it in a virtual space.

Further, the second display processing unit 440 uses a real-time image of the object 500 arranged in the physical space captured by the second information processing device 200 different from the first information processing device 100 as the image R100 of the physical space. A certain first image R110 can be displayed.

In other words, the user can check the real-time image of the object at the remote location on his/her own terminal.

The second information processing device 200 can be a device that includes at least an imaging unit or to which an imaging device is connected. In the following description, the second information processing apparatus 200 is a smartphone having a camera as an imaging unit. It is good also as a structure connected.

As in the above example, when the first object V102 is a game machine object imitating an arcade game machine, the object 500 is a real arcade game machine, but the first object V102 and the object are not necessarily the same. They do not have to be of the same shape.

Note that the real-time image of the object 500 displayed by the second display processing unit 440 can be an image of at least part of the object.

According to the above configuration, the user can view a real-time image of a remote physical space in the virtual space.

The target object 500 includes an operation unit that can be remotely operated by the user, and the user can obtain a prize set within the target object 500 .

As an example, the target object 500 may be a crane game machine, the operation unit may be a crane unit, and the first image R110 may include at least a space including a prize of the crane game machine.

FIG. 12 shows an example of a crane game machine as an object 500. The crane game machine is composed of a housing 501, a crane section 502, a prize installation section 503, an instruction section 504, and the like. One or more prizes 505 are placed on 503 .

A general crane game machine is a game machine in which a user operates an instruction unit 504 to operate a crane unit 502 to grab a prize 505 and drop the prize into an opening 506 to obtain a prize.

At this time, as shown in FIG. 13, an operation screen C110 including one or more operation objects is displayed together with the first image R110 on the display screen.

On the operation screen C110, as an example, a button C111 for instructing movement of the crane unit 502 in the left-right direction and a button C112 for instructing movement in the front-rear direction of the crane unit 502 are arranged. In addition, a start button C113 for starting play, a display C114 of the number of reserved people, a display C115 of the number of viewers, a camera switching button C116, a favorite registration button C117, an end button C118, and the like can also be displayed. Note that the arrangement of the favorite registration button C117 and the end button C118 is not limited to the operation screen C110.

As another example, the target object 500 can be a medal game machine, the operating unit can be a medal drop part, and the first image R110 can be a space containing at least medals of the medal game machine.

The medal game machine referred to here is a pusher game machine called a coin dropping game, and is a game machine that allows medals (coins) to be dropped as prizes or items other than medals to be obtained as prizes. .

FIG. 14 shows an example of a medal game machine, which is an object 500. The medal game machine is composed of a housing 511, a movable part 512, a medal placing part 513, a medal dropping part 514, and the like. One or more medals 515 and prizes 516 other than medals are placed on the placing portion 513 .

In a typical medal game machine, a user operates a medal dropping portion 514 to drop medals onto a medal placing portion 513 , and the movable portion 512 pushes the medals out, causing medals 515 and/or prizes to be placed in an opening 517 . It is a game machine in which a prize can be obtained by dropping 516.

At this time, it is assumed that an operation screen C120 including one or more operation objects is displayed together with the first image R110 on the display screen shown in FIG.

On the operation screen C120, as an example, a number button C121 for designating the number of medals to be dropped and a medal drop button C122 are arranged. Also, although not shown, similarly to FIG. 13, display of the number of reserved people, display of the number of viewers, a favorite registration button, an end button, and the like can also be displayed. Note that the arrangement of the favorite registration button and the end button is not limited to the operation screen C120.

According to the above configuration, the user can not only view a real-time image of a remote physical space in the virtual space, but also remotely operate an object in the physical space.

Then, in the embodiment of the present disclosure, the prizes 505 and 516 can be provided with predetermined markers.

The prize referred to here may be a box in which an article such as a prize is actually placed, or may be an empty box. It is assumed that the predetermined marker is displayed on at least one surface of the box, preferably on the entire surface of the box.

FIG. 16 shows an example in which an AR marker 507 is attached to a prize 505 as a predetermined marker. Note that any image may be used as the marker as long as it functions as the AR marker 507 .

Then, based on the markers included in the physical space image R100, the second display processing unit 440 can superimpose and display the related objects associated with the markers on the physical space image R100.

A related object may be related to the prize 505 or unrelated. As an example, the related object can be character object A101 as shown in FIG. This character object A101 can perform actions such as speaking, humming a song, and dancing.

These actions may be performed in a predetermined order based on data stored in advance in the information processing device 400 and/or the first information processing device 100, or may be performed in accordance with the user's operation status of the operation unit. It may be configured such that the action is performed at an appropriate timing according to the acquisition status of the prize.

Also, as an alternative, the related object A101 may generate a production effect. That is, it may be a production effect object that visually or aurally enlivens the game.

Here, the second display processing unit 440 is described as being provided on the server side (information processing device 400), but may be provided on the client side (first information processing device 100). That is, as long as an image in which the image R100 in the physical space and the related object A101 are synthesized is displayed on the display screen, the synthesis processing may be performed by any device and step. .

According to the above configuration, in addition to being able to show a real-time image of a remote real space to the user in the virtual space, it is possible to express that a real character exists in the real space. realization becomes possible.

Next, the details of acquisition of the premium will be described.

As shown in FIG. 18, the one or more computer processors may further comprise a prize storage 450. FIG.

The prize storage unit 450 stores the data of the related object in association with the user when the user successfully acquires the prize.

That is, the related object associated with the marker displayed on the prize won by the user is given to the user as digital data. In addition to giving the digital data, if an actual item is set as a prize, the item may be actually mailed to the user.

Usually, when playing a crane game in the real space, the user knows the contents of the prize when he/she wins the prize because there is an object in front of the user. However, since the digital prize is virtual, the user cannot see it with his own eyes before winning it. In the past, it was common to check the content by reading the QR code (registered trademark) attached to the prize with a compatible PC or smartphone after winning it.

On the other hand, according to the present disclosure, AR technology is used in the camera system of the online crane game, digital prizes are displayed before the prize is won in the crane game, and the user can check the details of the prize with his/her own eyes in advance. become able to. In addition, since the displayed objects move and make sounds, it is possible to obtain an unprecedented promotional effect.

Furthermore, if the prizes are only digital prizes, there is no need to send the prizes or take the number of days until the prizes are delivered, and both the service provider side and the service user side can efficiently reduce costs.

Further, the first display processing unit 410 can display the related object stored in the prize storage unit 450 as an attached object of the avatar object V101 displayed in the virtual space according to the user's operation.

Wearable objects can be objects such as clothes, bags, shoes, and accessories that can be worn by the avatar object V101. The wearable objects also include various parts such as the hair, eyes, nose, and mouth of the avatar object V101.

Alternatively, the first display processing unit 410 can display the related object stored in the prize storage unit 450 as a character object in the virtual space or the real space according to the user's operation.

A procedure for displaying a related object as a character object in the virtual space will be described.

First, the user displays his/her own room by performing a predetermined operation in the virtual space. Then, a list of character objects corresponding to the related objects stored in the prize storage unit 450 is displayed by activating a dedicated application for displaying the character object as the prize in my room.

By selecting one or a plurality of character objects from the character objects displayed as a list, a desired character object can be displayed in one's own room.

The list display here can be a display in which 3D character objects are arranged in a row instead of a list display. Also, the name and profile of the character object may be displayed together with the character object.

It is also possible to invite avatar objects of other users to one's own room and enjoy communication (for example, text/voice chat) around the character object.

In the above example, the explanation has been given assuming that the digital prize is an object attached to a character object or an avatar object, but the prize is not limited to these. For example, my room decorations (object groups such as walls, floors, windows, roofs, furniture, plants, pets, etc.), pose motions of emotions (emotion production), cartoon character objects (gogogogo / heartbeat) Character effects such as /Gang), deathblow effects, sound effects, music and voices such as BGM, etc. can also be applied as prizes.

Next, a procedure for displaying the related object as a character object in the physical space will be described.

By performing a predetermined operation in the virtual space, the user switches from the state of displaying the virtual space (VR mode) to the state of displaying the real space (AR mode).

In the AR mode, a character object corresponding to the related object stored in the prize storage unit 450 can be displayed in the image of the real space captured by the imaging unit provided in the first information processing apparatus 100 .

FIG. 19 shows an image in which a physical space image R200 and a character object A102 are displayed on the display screen of the first information processing apparatus 100. As shown in FIG. As shown in FIG. 19, in the AR mode, the display screen further includes a shooting button C201 for recording a still image or moving image, a menu button C202, a rotation button C203 for rotating the character object A102, and a character object A102. A move button C204 for moving the character object, an enlargement/reduction button C205 for enlarging/reducing the character object, and the like can be displayed. Note that the rotation, movement, and scaling of the character object may be performed by other operations such as a swipe operation and a pinch-in/out operation.

As in the case of the virtual space, a list of character objects corresponding to the related objects stored in the prize storage unit 450 is displayed, and one or more character objects are selected from the list of character objects to obtain a desired character object. character object may be displayed in the image of the real space.

In addition, by displaying the character object A102 in the image of the real space, it is possible to enjoy various poses of the character object from 360 degrees and to take pictures together in the real space.

Also, the second display processing unit 440 can display an avatar object as a related object.

Specifically, it is possible to make one's own avatar object appear as a prize in the crane game machine to be played.

The avatar object may be displayed wearing clothes set by the user, or may be displayed in a special state.

A special state is, for example, a state in which a limited accessory is worn, a state in which a limited motion is performed, or the like.

And at this time, the one or more computer processors can comprise an output section 460, as shown in FIG.

The output unit 460 outputs the data of the related object to the 3D printing device when the user successfully acquires the prize.

According to the above configuration, the user can obtain his or her original avatar object as a realistic figure.

Next, another example of the image of the physical space displayed by the second display processing unit 440 will be described.

The second display processing unit 440 further displays a second image R120, which is a real-time image of the object 500 captured by the third information processing device 300 different from the first information processing device 100, as the image R100 of the physical space. can be displayed.

The second information processing apparatus 200 images the object 500 from the front of the object 500 , and the third information processing apparatus 300 images the object 500 from the side of the object 500 .

The third information processing device 300 can be a device that includes at least an imaging unit or to which an imaging device is connected.

The second information processing device 200 and the third information processing device 300 can be arranged on the front and sides of the object 500 as shown in FIG.

It should be noted that the apparatus including the imaging unit or to which the imaging device is connected is not limited to the second information processing apparatus 200 and the third information processing apparatus 300, and may include more information processing apparatuses.

As an alternative, the second information processing device 200 may be configured to be connected to a plurality of imaging devices. In this case, the one imaging device can be read as the second information processing device 200 and the other imaging device can be read as the third information processing device 300 .

Also, the second information processing device 200 may capture an image of the surroundings from inside the object 500 .

For example, the second information processing device 200 can be equipped with a 360-degree video camera or connected to a 360-degree video camera.

Also, the second information processing apparatus 200 can be provided with a movable video camera or connected to a movable video camera. Such movement can be realized, for example, by attaching wheels to the video camera and moving it on a rail, or by mounting the video camera on a flight device such as a drone and moving it.

The one or more computer processors can further include a switching operation reception unit 470 as shown in FIG. 18 .

The switching operation reception unit 470 is for switching between the first image R110 and the second image R120.

At this time, the second display processing unit 440 can switch the displayed image according to the operation accepted by the switching operation accepting unit 470 .

An example of the operation by the user is selection of the camera switching button C116 as shown in FIG. In addition, the user's operation includes moving the first information processing apparatus 100, moving the user's line of sight, moving the user's head, and the like.

The determination unit 430 may determine whether the relationship between the avatar object and the first object satisfies the second condition.

Then, the second display processing unit 440 can switch the image to be displayed when the determination unit 430 determines that the second condition is satisfied.

A second condition is, for example, based on the position of the avatar object. For example, when the avatar object moves from the front to the side of the first object, the displayed image is switched from the first image to the second image. Similarly, when the avatar object moves from the side of the first object to the front, the displayed image is switched from the second image to the first image.

According to the above configuration, the image to be displayed can be automatically switched according to the relationship between the avatar object and the first object.

Further, the second display processing section 440 can change the displayed image according to the relationship between the avatar object and the first object.

The relationship between the avatar object and the first object is, for example, the relationship such as the distance between the avatar object and the first object and the orientation of the avatar object with respect to the first object.

Changing the image to be displayed includes, for example, changing the image by enlarging or reducing the captured image, changing the angle of view of the captured image, or the like.

For example, as the avatar object approaches the first object, the prize can be zoomed in, and as the avatar object moves away from the first object, the prize can be zoomed out.

In addition, as described above, the second display processing unit 440 can display the image of the physical space by superimposing it on a part of the first object or replacing it with a part of the first object.

Also, the image of the physical space may be displayed in a manner that it fits in the virtual space displayed on the display screen, or may be displayed as a screen separate from the screen that displays the virtual space. These images are shown in FIGS. 9 and 10.

In addition, the first display processing unit 410 can display in the virtual space a display object V105 that displays information related to the image in the physical space associated with the first object.

As shown in FIG. 21 as an example, the display object V105 can be displayed around the first object V102 in the form of a signboard or flag. Information related to the image of the physical space associated with the first object V102, for example, information of a prize that can be acquired by a connectable crane game machine, is displayed in the display object V105. As an example, the display object V105 can display information about a character object that will be a featured prize among character objects that can be obtained from a crane game machine that can be connected from the first object V102.

Also, the display object V105 may be displayed in a form like a signage object that displays an advertisement. At this time, the display object V105 displays information related to the image of the physical space associated with the first object V102, for example, information related to the moving image content of the prize obtainable by the connectable crane game machine. shall be As an example, when the prize is a character object, the display object V105 can display information about video content in which the character appears.

According to the above configuration, even when the user's avatar is at a position away from the first object, it becomes easier to recognize the existence of the first object, and the menu screen can be opened from the first object. It becomes possible to check the prize without going through the steps.

Also, as shown in FIG. 13, the number of reservations and/or the number of viewers can be displayed on the screen. The number of reserved users is the number of users waiting to play the game machine, and the number of viewers is the number of users watching the play.

While one user is playing the game machine, other users can choose to reserve or watch. When one user has finished playing the game machine and it becomes possible for the other user to play, a notice to that effect will be sent to the other users who have made the reservation. In that case, by selecting such notification, even if another user is in a place away from the first object, it can be automatically moved to the vicinity of the first object.

According to the above configuration, even if a plurality of user avatars wish to play the first object, they can be played fairly and the chance of playing is not lost. effect can be obtained.

Subsequently, an information processing method according to an embodiment of the present disclosure will be described.

An information processing method according to an embodiment of the present disclosure is an information processing method in an information processing system for displaying an image of virtual space on the display screen of the first information processing device. A specific example of the information processing system is as shown in FIGS. 1 and 2. FIG.

As shown in FIG. 22, the information processing method according to the embodiment of the present disclosure causes one or more computer processors included in the information processing system to perform a first display processing step S410, a movement processing step S420, a determination step S430, A second display processing step S440 is executed.

A first display processing step S410 causes the display screen to display an image in which the user's avatar object and the first object are displayed in the virtual space. The first display processing step S410 can be executed by the first display processing section 410 described above.

The first display processing step S410 may be executed on the server side (information processing device 400) or may be executed on the client side (first information processing device 100).

A movement processing step S420 moves the avatar object in the virtual space according to the user's operation. The movement processing step S420 can be performed by the movement processing unit 420 described above.

The movement processing step S420 may be executed on the server side (information processing device 400) or may be executed on the client side (first information processing device 100).

A determination step S430 determines whether the relationship between the avatar object and the first object satisfies a first condition. Such determination step S430 can be performed by the determination unit 430 described above.

The determination step S430 may be executed on the server side (information processing device 400) or may be executed on the client side (first information processing device 100).

A second display processing step S440 displays an image of the physical space on the display screen when it is determined in the determination step that the first condition is satisfied. The second display processing step S440 can be executed by the second display processing section 440 described above.

The second display processing step S440 may be executed on the server side (information processing device 400) or may be executed on the client side (first information processing device 100).

According to the above configuration, it is possible to provide a technical improvement that solves or mitigates at least part of the problems of the prior art described above.

Next, a computer program according to an embodiment of the present disclosure will be described.

A computer program according to an embodiment of the present disclosure is a computer program executed in an information processing system for displaying an image of virtual space on a display screen of a first information processing device. A specific example of the information processing system is as shown in FIGS. 1 and 2. FIG.

A computer program according to the present disclosure causes one or more computer processors included in an information processing system to perform a first display processing function, a movement processing function,
A determination function and a second display processing function are realized.

The first display processing function causes the display screen to display an image in which the user's avatar object and the first object are displayed in virtual space.

The first display processing function may be implemented on the server side (information processing device 400) or may be implemented on the client side (first information processing device 100).

The movement processing function moves the avatar object in the virtual space according to the user's operation.

The movement processing function may be implemented on the server side (information processing device 400) or may be implemented on the client side (first information processing device 100).

A determination function determines whether the relationship between the avatar object and the first object satisfies a first condition.

The determination function may be implemented on the server side (information processing device 400) or may be implemented on the client side (first information processing device 100).

The second display processing function displays an image of the physical space on the display screen when the determination unit determines that the first condition is satisfied.

The second display processing function may be implemented on the server side (information processing device 400) or may be implemented on the client side (first information processing device 100).

The above functions can be realized by the first display processing circuit 1410, the movement processing circuit 1420, the determination circuit 1430 and the second display processing circuit 1440 shown in FIG. First display processing circuit 1410, movement processing circuit 1420, determination circuit 1430, and second display processing circuit 1440 are performed by first display processing section 410, movement processing section 420, determination section 430, and second display processing section 440, respectively. shall be realized. The details of each part are as described above.

According to the above configuration, it is possible to provide a technical improvement that solves or mitigates at least part of the problems of the prior art described above.

Next, the first information processing device 100 according to the embodiment of the present disclosure will be described. It should be noted that either one of the information processing apparatus 400 and the first information processing apparatus 100 can be provided with each of the configurations described later.

The first information processing device 100 in the embodiment of the present disclosure is, as described above, a device that can communicate with the information processing device 400 via a network. The first information processing device 100 includes a wearable computer such as a smartphone (multifunctional telephone terminal), a tablet terminal, a personal computer, a console game machine, a head-mounted display (HMD), a glasses-type wearable terminal (AR glasses, etc.), and An information processing device other than these devices that can reproduce moving images can be used. These devices may be stand-alone devices that operate independently, or may be composed of a plurality of devices connected to each other so as to be able to transmit and receive various data.

Then, as shown in FIG. 24, the first information processing apparatus 100 according to the embodiment of the present disclosure includes a first display processing unit 110, a movement processing unit 120, a determination unit 130, and a second display processing unit 140. Prepare.

The first display processing unit 110 causes the display screen to display an image in which the user's avatar object and the first object are displayed in the virtual space. The first display processing unit 111 may have the same configuration as the first display processing unit 410 described above.

The movement processing unit 120 moves the avatar object in the virtual space according to the user's operation. The movement processing unit 120 may have the same configuration as the movement processing unit 420 described above.

The determination unit 130 determines whether the relationship between the avatar object and the first object satisfies the first condition. Such determination unit 130 can have the same configuration as the determination unit 430 described above.

The second display processing unit 140 displays an image of the physical space on the display screen when the determination unit determines that the first condition is satisfied. The second display processing section 140 may have the same configuration as the second display processing section 440 described above.

According to the above configuration, it is possible to provide a technical improvement that solves or mitigates at least part of the problems of the prior art described above.

Subsequently, an information processing method in the first information processing terminal according to the embodiment of the present disclosure will be described.

An information processing method according to an embodiment of the present disclosure is an information processing method in an information processing system for displaying an image of virtual space on the display screen of the first information processing device. A specific example of the information processing system is as shown in FIGS. 1 and 2. FIG.

As shown in FIG. 25, the information processing method according to the embodiment of the present disclosure causes one or more computer processors included in the first information processing device to perform a first display processing step S110, a movement processing step S120, and a determination step. S130 and second display processing step S140 are executed.

A first display processing step S110 causes the display screen to display an image in which the user's avatar object and the first object are displayed in the virtual space. The first display processing step S110 can be executed by the first display processing section 110 described above.

A movement processing step S120 moves the avatar object in the virtual space according to the user's operation. The movement processing step S120 can be executed by the movement processing unit 120 described above.

A determination step S130 determines whether the relationship between the avatar object and the first object satisfies a first condition. Such determination step S130 can be performed by the determination unit 130 described above.

A second display processing step S140 displays an image of the physical space on the display screen when it is determined in the determination step that the first condition is satisfied. The second display processing step S140 can be executed by the second display processing section 140 described above.

According to the above configuration, it is possible to provide a technical improvement that solves or mitigates at least part of the problems of the prior art described above.

Finally, a computer program according to the embodiment of the present disclosure will be described.

A computer program according to an embodiment of the present disclosure is a computer program executed in an information processing system for displaying an image of virtual space on a display screen of a first information processing device. A specific example of the information processing system is as shown in FIGS. 1 and 2. FIG.

A computer program according to the present disclosure causes one or more computer processors included in a first information processing apparatus to implement a first display processing function, a movement processing function, a determination function, and a second display processing function.

The first display processing function causes the display screen to display an image in which the user's avatar object and the first object are displayed in virtual space.

The movement processing function moves the avatar object in the virtual space according to the user's operation.

A determination function determines whether the relationship between the avatar object and the first object satisfies a first condition.

The second display processing function displays an image of the physical space on the display screen when the determination unit determines that the first condition is satisfied.

The above functions can be realized by the first display processing circuit 1110, the movement processing circuit 1120, the determination circuit 1130 and the second display processing circuit 1140 shown in FIG. First display processing circuit 1110, movement processing circuit 1120, determination circuit 1130, and second display processing circuit 1140 are performed by first display processing unit 110, movement processing unit 120, determination unit 130, and second display processing unit 140, respectively. shall be realized. The details of each part are as described above.

According to the above configuration, it is possible to provide a technical improvement that solves or mitigates at least part of the problems of the prior art described above.

Further, an information processing device such as a computer or a mobile phone can be preferably used to function as the server device or the terminal device according to the above-described embodiments. Such an information processing device stores a program describing processing details for realizing each function of the server device or the terminal device according to the embodiment in the storage unit of the information processing device, and executes the program by the CPU of the information processing device. It can be realized by reading and executing.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and equivalents thereof.

Further, the method described in the embodiment can be executed by a computer (computer), for example, magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD, MO, etc.), It can be stored in a recording medium such as a semiconductor memory (ROM, RAM, flash memory, etc.), or can be transmitted and distributed via a communication medium. The programs stored on the medium also include a setting program for configuring software means (including not only execution programs but also tables and data structures) to be executed by the computer. A computer that realizes this apparatus reads a program recorded on a recording medium, and in some cases, builds software means by a setting program, and executes the above-described processes by controlling the operation by this software means. The term "recording medium" as used herein is not limited to those for distribution, and includes storage media such as magnetic disks and semiconductor memories provided in computers or devices connected via a network. The storage unit may function, for example, as a main storage device, an auxiliary storage device, or a cache memory.

1000 information processing system 100 first information processing device 200 second information processing device 300 third information processing device 400 information processing device 410 first display processing unit 420 movement processing unit 430 determination unit 440 second display processing unit

Claims (22)

An information processing system for displaying an image of a virtual space on a display screen of a first information processing device, comprising one or more computer processors,
The one or more computer processors are configured to display, on the display screen, an image in which a user's avatar object and a first object are displayed in the virtual space;
a movement processing unit that moves the avatar object in the virtual space according to a user's operation;
a determination unit that determines whether the relationship between the avatar object and the first object satisfies a first condition;
an information processing system comprising: a second display processing unit that displays an image of a physical space on the display screen when the determination unit determines that the first condition is satisfied. The second display processing unit
displaying on the display screen a menu image for selecting one physical space image from among a plurality of physical space images when the determination unit determines that the first condition is satisfied;
2. The information processing system according to claim 1, wherein the real space image of the position to be displayed on the display screen is determined according to the selection by the user. The second display processing unit, as the image of the physical space, is a real-time image of an object arranged in the physical space captured by a second information processing device different from the first information processing device. 3. The information processing system according to claim 1, wherein an image is displayed. the object includes an operation unit that can be remotely operated by the user;
4. The information processing system according to claim 3, wherein said user can acquire a prize placed within said object. the object is a crane game machine,
The operation unit is a crane unit,
5. The information processing system according to claim 4, wherein said first image includes a space of said crane game machine including at least said prize. the object is a medal game machine,
The operation unit is a medal throwing unit,
5. The information processing system according to claim 4, wherein said first image is a space containing at least medals of said medal game machine. A predetermined marker is attached to the prize,
5. The second display processing unit superimposes and displays a related object associated with the marker on the image of the physical space based on the marker included in the image of the physical space. , 5 or 6. The one or more computer processors further comprise:
8. The information processing system according to claim 7, further comprising a prize storage unit that stores the data of the related object in association with the user when the user successfully acquires the prize. The first display processing unit
9. The information processing system according to claim 8, wherein the related object stored in the prize storage unit is displayed as an attached object of the avatar object displayed in the virtual space in accordance with the user's operation. . The first display processing unit
9. The information processing system according to claim 8, wherein the related object stored in the prize storage unit is displayed as a character object in the virtual space or the real space according to the user's operation. 11. The information processing system according to claim 8, wherein the second display processing unit displays the avatar object as the related object. The one or more computer processors further comprise:
9. The information processing system according to claim 8, further comprising an output unit that outputs data of the related object to a 3D printing device when the user successfully acquires the prize. The second display processing unit further displays a second image, which is a real-time image of an object captured by a third information processing device different from the first information processing device, as the image of the physical space,
The second information processing device captures an image of the object from the front of the object,
13. The information processing system according to any one of claims 3 to 12, wherein said third information processing device takes an image of said object from a side of said object. The one or more computer processors further comprise:
A switching operation receiving unit for switching between the first image and the second image;
14. The information processing system according to claim 13, wherein the second display processing unit switches the image to be displayed according to the operation accepted by the switching operation accepting unit. The determining unit determines whether a relationship between the avatar object and the first object satisfies a second condition,
14. The information processing system according to claim 13, wherein the second display processing unit switches the displayed image when the determination unit determines that the second condition is satisfied. 13. The information processing system according to any one of claims 3 to 12, wherein said second information processing device captures an image of the surroundings from inside said object. 17. The information processing system according to claim 16, wherein said second display processing unit changes the displayed image according to the relationship between said avatar object and said first object. 18. The second display processing unit displays the image of the physical space by superimposing it on a part of the first object or replacing it with a part of the first object. The information processing system according to . 19. The method according to any one of claims 1 to 18, wherein the first display processing unit displays a display object displaying information related to the image of the physical space associated with the first object. Information processing system as described. An information processing method in an information processing system for displaying an image of a virtual space on a display screen of a first information processing device, comprising:
One or more computer processors included in the information processing system,
a first display processing step of displaying, on the display screen, an image in which the user's avatar object and the first object are displayed in the virtual space;
a movement processing step of moving the avatar object in the virtual space according to a user's operation;
a determination step of determining whether the relationship between the avatar object and the first object satisfies a first condition;
and a second display processing step of displaying an image of the physical space on the display screen when it is determined in the determination step that the first condition is satisfied. An information processing method for displaying an image of a virtual space on a display screen of a first information processing device, comprising:
One or more computer processors included in the first information processing device,
a first display processing step of displaying, on the display screen, an image in which the user's avatar object and the first object are displayed in the virtual space;
a movement processing step of moving the avatar object in the virtual space according to a user's operation;
a determination step of determining whether the relationship between the avatar object and the first object satisfies a first condition;
and a second display processing step of displaying an image of the physical space on the display screen when it is determined in the determination step that the first condition is satisfied. A computer program for displaying an image of a virtual space on a display screen of a first information processing device,
One or more computer processors included in the first information processing device,
a first display processing function for displaying, on the display screen, an image in which the user's avatar object and the first object are displayed in the virtual space;
a movement processing function for moving the avatar object in the virtual space according to a user's operation;
a determination function for determining whether the relationship between the avatar object and the first object satisfies a first condition;
and a second display processing function for displaying an image of the physical space on the display screen when the determination function determines that the first condition is satisfied.

Images