JP2021052224A - Program, information processing apparatus, and display method - Google Patents

Abstract

To provide a program, an information processing apparatus, and a display method that can improve a method for displaying an image with motion.SOLUTION: A program causes an information processing apparatus 30 displaying a screen based on screen information received from one or more information processing apparatuses via a network to function as an image acquisition unit that acquires an image to be displayed on the screen, an image playback unit that plays back the image while changing a display mode based on a display pattern, and a display state storage unit that stores a display state of the image played back by the image playback unit, and in displaying the image stored by the display state storage unit in the same display pattern again, the image playback unit resumes the playback of the image from the display state stored by the display state storage unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a program, an information processing device, and a display method.

Images may be displayed on screen information such as Web pages. In addition to still images, animations and moving images can be displayed as images. In addition, some images, such as 3DCG (three-dimensional computer graphic) images and wide-angle images, have interactive images whose display mode changes automatically or by the operation of the viewer.

Further, there is known a distribution type display advertisement in which an advertisement is displayed on an image displayed on a Web page. Advertisers can display advertisements for products and services in the display frames provided by various websites. Viewers who are interested in the products and services displayed in the display frame press (click or tap) the image in the display frame, so the advertiser can attract the viewer to their website (called a landing page). Yes (see, for example, Patent Document 1). Patent Document 1 discloses a system in which a terminal device displays a wide-angle image as an advertisement on a display frame.

However, the conventional technique has a problem that there is room for improvement in the method of displaying a moving image. For example, some delivery-type display advertisements automatically switch images of products to be displayed in the advertisement during display. On the other hand, in a video advertisement or an advertisement in which a wide-angle image is displayed by changing the display position, the entire advertisement is reproduced by displaying the advertisement for a certain period of time. However, since the cycle of switching images determined by the display frame is not related to the time it takes for the moving video to play the entire advertisement, the moving advertisement is displayed even though it is played only halfway. It is possible that the frame will switch to another image. For example, if an advertisement that displays like a movie while moving the display position of a wide-angle image is an advertisement that reproduces the entire image in 10 seconds, but the cycle of switching images determined by the display frame is 5 seconds, the terminal device. Can only play for the first 5 seconds. In this case, the side that provides the advertisement may not be able to show the viewer the part that the advertisement wants to be shown, and the effect of the advertisement may not be exhibited.

Further, in the case of an interactive image in which the viewer can operate the display mode such as the movement of the image, when the terminal device switches the image at a cycle determined by the display frame, the image is forcibly switched during the operation by the viewer. The next time the same image is viewed, the viewer must operate the same image in the same display mode, which may reduce operability.

In view of the above problems, an object of the present invention is to provide a program, an information processing apparatus, and a display method capable of improving a display method of a moving image.

In view of the above problems, the present invention provides an information processing device that displays a screen based on screen information received from one or more information processing devices via a network, and an image acquisition unit that acquires an image displayed on the screen. The display state is functioned as an image reproduction unit that reproduces the image while changing the display mode based on the display pattern and a display state storage unit that stores the display state of the image reproduced by the image reproduction unit. When the image saved by the storage unit is displayed again with the same display pattern, the image reproduction unit restarts the reproduction of the image from the display state saved by the display state storage unit. To do.

It is possible to provide a program, an information processing device, and a display method that can improve the display method of a moving image.

It is a figure which shows an example of the advertising space. It is a figure which shows an example of the delivery type display advertisement displayed in the display frame. It is a figure explaining the outline of the image display method. This is an example of a schematic configuration diagram of a browsing system. It is a hardware block diagram of an example of a computer system. It is a hardware block diagram of an example of a terminal device. It is a use image figure of the omnidirectional camera. It is a figure explaining the outline of the process from the image taken by the omnidirectional camera to the creation of the omnidirectional image. It is a figure explaining the outline of the process from the image taken by the omnidirectional camera to the creation of the omnidirectional image. This is an example of a diagram for explaining the line of sight of the viewer. The state of projective transformation of an object to a virtual camera is schematically shown. This is an example of a diagram showing objects displayed from different viewpoints. This is an example of a functional block diagram showing the functions of the terminal device, the advertiser Web server, the partner site Web server, the image distribution device, the SSP, and the DSP in a block shape. This is an example of a functional block diagram for explaining the function of the image display unit in detail. It is a figure which shows an example of the display pattern. It is a figure which shows an example of the detection point and the calculation example of the coordinates. It is a figure explaining the hierarchical structure of the image displayed in a display frame. This is an example of a sequence diagram illustrating a procedure for the viewing system to deliver a spherical image displayed as an advertisement. This is an example of a flowchart showing a procedure in which the advertisement distribution unit of the image distribution device distributes the same spherical image or 3DCG image in the same display pattern. This is an example of a flowchart for explaining a process when the terminal device displays a spherical image or a 3DCG image. It is a figure which shows the display example which a terminal device reproduces from the last display position. This is an example of a functional block diagram for explaining the function of the image display unit in detail (Example 2). It is a figure explaining the reproduction time of a display pattern. This is an example of a flowchart for explaining a process when the terminal device displays a spherical image or a 3DCG image. It is a figure which shows some form in which advertisement data is delivered.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

<About the display frame provided on the Web page>
First, the display frame 7 on which the image data is displayed will be described with reference to FIG. FIG. 1 shows an example of the display frame 7. The display frame 7 is a space for inserting a preset advertisement on a Web page distributed on the Internet. Often referred to as an advertising space. There are various methods for displaying the advertisement on the display frame 7. For example, the advertisement may be fixed in advance in the display frame 7. Further, the ad tag is described in the display frame 7, and the terminal device 30 may acquire the advertisement from the URL (ad server) indicated by the ad tag. Alternatively, a third-party distribution that delivers an advertisement to a Web page from a server owned by an advertising agency or the like instead of a server owned by a media company that provides a Web page having a display frame 7 may be used. In the following, an example of using third-party distribution will be described as an example.

<Display example of display frame advertisement>
FIG. 2 shows an example of a delivery-type display advertisement displayed in the display frame 7. The advertisement shown in FIG. 2A is first displayed in the display frame 7. In FIG. 2A, images 313a to 313c of the three products are displayed vertically arranged in one display frame 7. Each of these is different image data. The state of FIG. 2A is referred to as a "plurality display form".

As shown in FIGS. 2 (b) to 2 (d), the three images 313a to FIG. 2 (a) are formed by the viewer hovering over the display frame 7 or automatically switching the display due to the passage of time or the like. The 313c switches to images 313a to 313c of individual products (each image is referred to as a product image). Further, the images 313a to 313c of FIGS. 2 (b) to 2 (d) return to the display of FIG. 2 (a) by the operation of the viewer or automatically. After returning, the three images 313a to 313c of the display frame 7 may be automatically switched to another image. When a plurality of images are displayed, the number is not limited to three, and may be two or four or more. The states of FIGS. 2 (b) to 2 (d) are referred to as "single display form".

The images 313a to 313c are wide-angle images that do not fit in the display frame 7, and the display position (angle of view) displayed on the display frame 7 by the terminal device 30 can be changed according to a script. In this embodiment, as an example of a wide-angle image, a spherical image in which a circumference of 360 degrees is captured is used.

Further, when the three images 313a to 313c are automatically switched to another image, in the plurality of display modes of FIG. 2A, for example, the three images 313a to 313c are set as one set and the other three images are displayed. Is displayed. The other three images are also displayed in a single display form. Of the plurality of products displayed in the plurality of display formats in this way, one product is enlarged and becomes an advertisement for the original plurality of products, and then an advertisement for a product different from the previous one is displayed. The process is repeated, and the advertisement becomes eye-catching to consumers. Further, the display frame 7 as shown in FIG. 2 has an advantage that a large number of products can be introduced in a limited space.

On the other hand, when the images 313a to 313c are moving images such as moving images, the same image is continuously displayed for a certain period of time, so that the entire advertisement is reproduced. However, since the cycle of switching the image determined by the display frame 7 and the time until the moving image reproduces the entire advertisement are not related, the moving advertisement is reproduced only halfway. The display frame 7 may switch to another image. In this case, the side that provides the advertisement cannot show the viewer the part that the advertisement wants to show.

<Outline of processing of this embodiment>
Therefore, in the present embodiment, the display state when the image displayed in the display frame 7 is switched to another image (or before or after that) is recorded, and when the same image is displayed next time, the same image is displayed. Resume playback of the image from the recorded display state.

FIG. 3 is a diagram illustrating an outline of an image display method according to the present embodiment. First, as shown in FIG. 3A, the terminal device displays three images 313a to 313c. Although the image is displayed on the entire screen in FIG. 3 for convenience of explanation, it is often displayed in the display frame 7 of the Web page.

Next, as shown in FIG. 3 (b), the three images 313a to 313c of FIG. 3 (a) are any one product unit by mouse over or automatic display switching due to the passage of time or the like. It switches to the image 313a of. Since the image 313a is a moving image, if it is already displayed in the state of FIG. 3A or is displayed alone, the movement is reproduced like a moving image.

Next, when a certain period of time determined by the display frame 7 elapses, as shown in FIG. 3C, the terminal device 30 displays images 313d to 313f of three products different from the above. When the terminal device 30 switches to the next product during playback, the terminal device 30 records the display state immediately before the switch. As a recording mechanism, a cookie of a Web browser can be used. The display state is information indicating which time point in the whole reproduction was being reproduced.

Next, the terminal device 30 may display the same image (advertisement) again. One of the cases is a case where a mechanism called retargeting advertisement is applied when a viewer is browsing some Web page on the terminal device 30 (transition from FIG. 3A to FIG. 3D). ). Another case is a case where the terminal device 30 makes one round of switching images while displaying the same Web page (transition from FIG. 3C to FIG. 3D).

When the same image is delivered, as shown in FIG. 3D, the terminal device 30 reads the recorded display state and resumes playback from the last display state. In FIG. 3 (d), the reproduction of the image is restarted in the case of the single display mode, but the reproduction may be restarted from the plurality of display modes as shown in FIG. 3 (a). In this case, in the state of FIG. 3A, the terminal device 30 reads the recorded display state and restarts the reproduction from the last display state.

In this way, the browsing system of the present embodiment can show the entire image to the viewer by displaying the movement of the image from the continuation of the previous time even when the image is switched during playback, and the advertisement. Can fully demonstrate the effect of.

In addition, the viewer changed the display state such as the display position of the image by operation, but even if the image is switched, the viewer can display the same display state for the same image by displaying the movement of the image from the previous continuation. It is not necessary to operate the image, and it is possible to prevent the operability from being lowered.

<Terminology>
The image whose display mode changes may not be a still image. It is included in the image whose display mode changes even when the still image is switched. The image does not have to be for advertising, it can be an image. An image in which the display mode changes is also included in the moving image in which the frames are changed one after another.

When the display mode of a still image changes, it means that a part that has not been displayed up to that point is displayed or the viewpoint changes. Such a change in the display mode is called "moving". Still images can be enlarged / reduced. In addition, the color may change or the brightness may change. The viewer can change the display mode by operation. In the case of moving images, the viewer can, for example, play, stop, fast forward, rewind, and so on.

When the spherical image and the 3DCG image are not distinguished, it may be simply an image.
<System configuration example>
FIG. 4 is an example of a schematic configuration diagram of the browsing system 100. The browsing system 100 includes a terminal device 30, an image distribution device 10, a DSP 20, an SSP (Supply Side Platform) 50, a partner site Web server 60, and an advertiser Web server 70 that can communicate via the network N. ..

The network N is constructed by a LAN constructed in a facility or the like where the terminal device 30 is installed, a provider network of a provider that connects the LAN to the Internet, a line provided by a line operator, or the like. When the network N has a plurality of LANs, the network N is called a WAN or the Internet. The network N may be constructed either wired or wireless, and may be a combination of wired and wireless. Further, when the terminal device 30 is directly connected to the public network, it can be connected to the provider network without going through a LAN.

The terminal device 30 is an information processing device that operates as a client terminal in the present embodiment. Browser software or application software having a function equivalent thereto is operating in the terminal device 30, and the Web page requested by the terminal device 30 is received from the partner site Web server 60 and displayed on the display.

The terminal device 30 is, for example, a PC (Personal Computer), a tablet device, a smartphone, a PDA (Personal Digital Assistant), a game machine, a navigation terminal, a wearable PC, or the like, but it is sufficient if a Web page can be displayed. For example, if the printer has a function or a display for displaying a Web page, the printer may be used as the terminal device 30. In addition, digital signage may display a Web page. Digital signage is a system, display device, or information to be displayed that uses an electronic display device such as a display to transmit information in places where people can pass or exist, such as outdoors, storefronts, public spaces, and transportation facilities. Say. In this embodiment, it is assumed that the Web page includes the Web application. A Web application refers to software or a mechanism thereof that operates by coordinating a program in a programming language (for example, JavaScript (registered trademark)) that operates on a Web browser and a program on the Web server side, and is used on the Web browser. ..

The terminal device 30 may be connected to the network N via a wired LAN or wireless LAN access point, or may be connected to the network N by an exchange line type communication such as 3G, 4G or LTE (Long Term Evolution). May be good.

The partner site Web server 60 is a server that provides information and functions to a client computer (terminal device 30 of the present embodiment) used by a viewer through a network (“one or more information processing devices” within the scope of claims. An example). A display frame 7 is provided on the Web page provided by the partner site Web server 60. The partner site Web server 60 requests the SSP 50 to display an advertisement in the display frame 7. As a result, the display frame 7 can be associated with the advertisement tag issued by the SSP 50.

The advertiser Web server 70 is also a server (general information processing device) that provides information and functions to the client computer (terminal device 30 of the present embodiment) used by the viewer via a network. The advertiser asks the DSP 20 to purchase the display frame 7. Further, the image distribution device 10 that holds the advertisement data is registered in the DSP 20. Further, the advertiser uploads the advertisement data to the image distribution device 10. In the advertisement data, in addition to the image whose display mode changes depending on the operation of the viewer, the URL of the website (landing page) to be viewed by the viewer such as the advertiser Web server 70 is registered (URL is linked). There is).

The advertiser Web server 70, the partner site Web server 60, and the terminal device 30 communicate with each other using, for example, the communication protocol of HTTP or HTTPS. In response to the request from the terminal device 30, the advertiser Web server 70 and the partner site Web server 60 transmit screen information to the terminal device 30. The screen information is a program described in HTML, a script language, CSS (cascading style sheet), etc., and the structure of the web page is mainly specified by HTML, the operation of the web page is defined by the script language, and the operation of the web page is defined by CSS. The style of the web page is specified. In the present embodiment, it is a script language that reflects the operation of the viewer on the Web page in the spherical image. Specifically, as a script language, a programming language called JavaScript (registered trademark) or ECMAScript is known.

Both the advertiser Web server 70 and the partner site Web server 60 record a cookie in the browser software of the terminal device 30. The advertiser Web server 70 records both its own cookie and the DSP20 cookie, and the partner site Web server 60 records both its own cookie and the SSP50 cookie.

The SSP 50 is a mechanism for the partner site Web server 60 that publishes the display frame 7 to maximize the profit by selling the display frame 7. Although it is described as one device in the figure, it is a network of one or more information processing devices. The partner site Web server 60 requests the SSP 50 to sell the display frame 7. Specifically, the advertisement tag issued by the SSP 50 is described in the display frame 7, and when the terminal device 30 displays the Web page of the partner site Web server 60, the advertisement tag requests the SSP 50 to deliver the advertisement (advertisement request). .. The SSP 50 accepts a bid for the display frame 7 from the DSP 20 and transmits the access information 1 for accessing the successful bid DSP 20 to the terminal device 30.

The DSP 20 is a mechanism for efficiently and effectively delivering advertisements for advertisers, such as purchasing the display frame 7 and delivering advertisements. Although it is described as one device in the figure, it is a network of one or more information processing devices. The DSP 20 acquires a cookie (referred to as an SSP cookie to distinguish it) from the SSP 50, and determines the attributes of the viewer based on the correspondence information between the DSP cookie and the SSP cookie managed by the DSP 20. Then, the bid is placed on the SSP50 at a price determined based on the distribution setting of the advertisement from the advertiser who requests based on the attributes of the viewer, the budget, and the like.

The successful bidder DSP 20 is requested to advertise by the terminal device 30 by the access information 1. The DSP 20 notifies the image distribution device 10 of the advertisement request, and acquires the access information 2 for the terminal device 30 to request the advertisement data from the image distribution device 10 from the image distribution device 10. The advertisement data includes a display program for displaying spherical images (an example of a program in the scope of claims) and an operation history program for acquiring operation history.

The DSP 20 transmits the access information 2 to the terminal device 30. The DSP 20 charges the advertiser according to the contract. Billing is performed according to the contract between the image distribution device 10 and the advertiser. However, it depends on the contract whether or not you will be charged just for displaying the advertisement.

The image distribution device 10 is one or more information processing devices that provide advertisement data including advertisement materials (omnidirectional images) that are optimal for the attributes of the viewer to the display frame 7 purchased by the DSP 20 (claims). An example of "one or more information processing devices" in the range of). The image distribution device 10 holds submitted materials such as banners and texts and advertisement images. Advertising data may be just a banner (an image made up of text, photos or pictures), or it may include a scripting language in addition to the image. In the present embodiment, a display program that rotates the spherical images in the display order and reflects the viewer's operations on the spherical images in the display mode of the spherical images, and an operation history on the spherical images. Includes an operation history program for recording images. Both are written in a scripting language.

When the terminal device 30 requests advertisement data from the image distribution device 10 based on the access information 2, the image distribution device 10 transmits the advertisement data to the terminal device 30 (the terminal device 30 is an example of an information processing device within the scope of claims). ). The advertisement data includes the spherical image, and the display program automatically rotates the spherical image so as to follow the point of interest of the spherical image, and enlarges or reduces the spherical image. Further, the operation history program transmits the operation history for the spherical image to the image distribution device 10 together with the cookie of the image distribution device 10 (referred to as an image cookie for distinction) and the image ID. The image cookie is written in the browser software from the image distribution device 10. The operation history is, for example, information as to which viewing angle of view is displayed. Since this operation history is used to determine the point of interest, the operation history may not be transmitted if the operation history is not used to determine the point of interest.

Further, the operation history program transmits to the image distribution device 10 that the image has been clicked together with the pattern ID of the display pattern described later. The image distribution device 10 defines a plurality of display patterns for one spherical image, and by monitoring the click rate of each display pattern, only the display pattern having a high click rate can be distributed.

<Hardware configuration example>
Subsequently, the hardware configurations of the advertiser Web server 70, the partner site Web server 60, the image distribution device 10, the SSP 50, and the DSP 20 in the browsing system of the present embodiment will be described.

The advertiser Web server 70, the partner site Web server 60, the image distribution device 10, the SSP 50 and the DSP 20 are realized by, for example, a computer system having a hardware configuration shown in FIG. FIG. 5 is a hardware configuration diagram of an example of the computer system 200 according to the present embodiment.

The computer system 200 shown in FIG. 5 includes an input device 201, a display device 202, an external I / F 203, a RAM (Random Access Memory) 204, a ROM (Read Only Memory) 205, a CPU (Central Processing Unit) 206, and a communication I /. It includes an F207, an HDD (Hard Disk Drive) 208, and the like, and each is connected to each other by a bus B.

The input device 201 includes a keyboard, a mouse, a touch panel, and the like, and is used by the operator to input each operation signal. The display device 202 includes a display and the like, and displays the processing result by the computer system 200.

The communication I / F 207 is an interface for connecting the computer system 200 to the in-house network, the Internet, and the like. As a result, the computer system 200 can perform data communication via the communication I / F 207.

HDD 208 is a non-volatile storage device that stores programs and data. The stored programs and data include, for example, an OS (Operating System), which is basic software that controls the entire computer system 200, and application software that provides various functions on the OS. The HDD 208 manages the stored programs and data by a predetermined file system and / or DB (database).

The external I / F 203 is an interface with an external device. The external device includes a recording medium 203a and the like. As a result, the computer system 200 can read and / or write to the recording medium 203a via the external I / F 203. The recording medium 203a includes a flexible disk, a CD (Compact Disk), a DVD (Digital Versatile Disk), an SD memory card (SD Memory card), a USB memory (Universal Serial Bus memory), and the like.

The ROM 205 is a non-volatile semiconductor memory (storage device) capable of holding programs and data even when the power is turned off. The ROM 205 stores programs and data such as a BIOS (Basic Input / Output System), an OS setting, and a network setting that are executed when the computer system 200 is started. The RAM 204 is a volatile semiconductor memory (storage device) that temporarily holds programs and data.

The CPU 206 is an arithmetic unit that realizes control and functions of the entire computer system 200 by reading programs and data from a storage device such as the ROM 205 and the HDD 208 onto the RAM 204 and executing processing.

Although each server may support cloud computing, it may be a so-called single information processing device. Cloud computing refers to a usage pattern in which resources on a network are used without being aware of specific hardware resources.

<< Terminal equipment >>
FIG. 6 is a hardware configuration diagram of an example of the terminal device 30. The terminal device 30 in FIG. 6 is assumed to be a tablet device. The terminal device 30 includes a CPU 601, a ROM 602, a RAM 603, an EEPROM 604, a CMOS sensor 605, an acceleration / orientation sensor 606, and a media drive 608.

The CPU 601 controls the overall operation of the terminal device 30. The ROM 602 stores a basic input / output program. The RAM 603 is used as a work area for the CPU 601. The EEPROM 604 reads or writes data under the control of the CPU 601. The CMOS sensor 605 takes an image of the subject under the control of the CPU 601 and obtains image data. The acceleration / orientation sensor 606 is an electronic magnetic compass, a gyro compass, an acceleration sensor, etc. that detect the geomagnetism.

The media drive 608 controls reading or writing (storage) of data to the media 607 such as a flash memory. The media drive 608 has a structure in which the media 607 from which already recorded data is read out or new data is newly written and stored is detachable.

In addition, the program 604p executed by the CPU 601 is stored in the EEPROM 604. The program 604p is application software, an OS, or the like for executing various processes in the embodiment. The program 604p may be distributed in a state of being stored in the media 607 or the like, or may be distributed from a server for program distribution.

Further, the CMOS sensor 605 is a charge coupling element that converts light into electric charges and digitizes the image of the subject. The CMOS sensor 605 may be, for example, a CCD (Charge Coupled Device) sensor as long as it can image a subject. The CMOS sensor 605 can read barcodes and two-dimensional barcodes.

Further, the terminal device 30 includes an RF tag reader / writer 622, an antenna I / F 623, and a vibration actuator 624. The RF tag reader / writer 622 communicates according to a standard such as NFC (Near Field Communication).

The vibration actuator 624 is a motor that vibrates the terminal device 30. For example, when an incoming call or an e-mail is received, it vibrates to notify the viewer to that effect.

Further, the terminal device 30 includes a voice input unit 609, a voice output unit 610, an antenna 611, a communication unit 612, a wireless LAN communication unit 613, a short-range wireless communication antenna 614, a short-range wireless communication unit 615, a display 616, a touch panel 617, and the like. It is equipped with a bus line 619.

The voice input unit 609 converts the voice into a voice signal. The voice output unit 610 converts a voice signal into voice. The communication unit 612 uses the antenna 611 to communicate with the nearest base station device by a wireless communication signal. The wireless LAN communication unit 613 performs wireless LAN communication conforming to the IEEE802.11 standard.

The short-range wireless communication unit 615 is a communication device that uses the short-range wireless communication antenna 614 and conforms to, for example, Bluetooth (registered trademark) or Bluetooth Low Energy (registered trademark) communication standard.

The display 616 is a liquid crystal, an organic EL, or the like that displays an image of a subject, various icons, and the like. The touch panel 617 is mounted on the display 616 and is composed of a pressure-sensitive or electrostatic panel, and detects a touch position on the display 616 by touching with a finger, a touch pen, or the like. The bus line 619 is an address bus, a data bus, or the like for electrically connecting each of the above parts.

In addition, the terminal device 30 includes a dedicated battery 618 and can be driven by either the battery 618 or a commercial power source. The voice input unit 609 includes a microphone for inputting voice. The audio output unit 610 includes a speaker that outputs audio.

<About spherical images>
Spherical images and 3DCG (Three-dimensional computer graphics) are known as examples of images that change according to the operation of the viewer. First, the spherical image will be described.

The spherical image will be described with reference to FIGS. 7 to 10. FIG. 7 is a usage image diagram of the spherical camera 9. As shown in FIG. 7, the spherical camera 9 is an imaging device that the user holds in his / her hand to take an image of a subject around the user. The spherical camera 9 has a structure in which the back surfaces of the two image pickup elements are opposed to each other, and two hemispherical images are obtained by photographing a subject around the user.

Next, with reference to FIGS. 8 and 9, the outline of the process from the image captured by the spherical camera 9 to the creation of the spherical image 6 will be described. 8 (a) is a hemispherical image (front side) captured by the spherical camera 9, FIG. 8 (b) is a hemispherical image (rear side) captured by the spherical camera 9, FIG. 8 (c). Is a diagram showing an image represented by the equirectangular projection method (hereinafter, referred to as “equal-distance cylindrical projection image”). FIG. 9A is a conceptual diagram showing a state of covering a sphere in an equirectangular projection image, and FIG. 9B is a diagram showing a spherical image 6 of the whole celestial sphere.

As shown in FIG. 8A, the image obtained by the spherical camera 9 is a hemispherical image (front side) curved by a fisheye lens. Further, as shown in FIG. 8B, the image obtained by the spherical camera 9 is a hemispherical image (rear side) curved by the fisheye lens. Then, the hemispherical image (front side) and the hemispherical image (rear side) inverted 180 degrees are combined by the spherical camera 9, and as shown in FIG. 8 (c), the equirectangular projection image. Is created.

Then, by using OpenGL ES (Open Graphics Library for Embedded Systems), as shown in FIG. 9A, the equirectangular projection image is pasted so as to cover the spherical surface, and FIG. 9 (A) The spherical image 6 as shown in b) is created. As described above, the spherical image 6 is represented as an image in which the equirectangular projection image faces the center of the sphere. OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. The spherical image 6 may be a still image or a moving image.

The advertisement data distributed by the image distribution device 10 includes the spherical image 6. Since the spherical image 6 is an image pasted so as to cover the spherical surface, it is curved and gives a sense of discomfort to humans. Therefore, the terminal device 30 displays a part of the predetermined region T of the spherical image 6 as a flat image with little curvature so as not to give a sense of discomfort to humans. The predetermined area T is indicated by the coordinates (X, Y, Z) in the three-dimensional virtual space. On the other hand, since the display 616 is a two-dimensional plane, the terminal device 30 cannot be displayed if the predetermined area T remains as it is. Therefore, the terminal device 30 obtains a predetermined region T by perspective projection conversion that projects a three-dimensional object onto a two-dimensional plane using a technique of 3D computer graphics.

FIG. 10 is an example of a diagram for explaining the line of sight of the viewer. Since the spherical image 6 has three-dimensional coordinates, the line-of-sight direction is specified by the three-dimensional coordinates or information that specifies the coordinates of the sphere such as latitude and longitude. In the present embodiment, the center cp of the predetermined region T is the line-of-sight direction. This line-of-sight direction corresponds to the display position of the display frame 7.

The viewer can change the line-of-sight direction by operation, but assuming that the virtual camera IC does not translate, the virtual camera IC will be a rigid body with roll (rotation around the Z axis) and yaw (centered on the Y axis). Three rotations are possible: rotation) and pitch (rotation about the X-axis). The line-of-sight direction changes regardless of which of these three rotations occurs. For example, when the viewer rotates the spherical image 6 in the horizontal direction, the yaw angle changes, and when the viewer rotates the spherical image 6 in the vertical direction, the pitch angle changes, and the spherical image 6 is rotated around the center of the display 616. And the roll angle changes. In the present embodiment, the operation of the viewer on the Web page is reflected in the line-of-sight direction (roll angle, yaw angle, pitch angle) and the like. How it is reflected shall be described in advance in the display program. The predetermined area T can be enlarged or reduced.

The viewer can change the line-of-sight direction (rotate the image data) by flicking the display frame 7 up / down / left / right.

<About 3DCG>
3DCG is a method of creating an image with a sense of depth (three-dimensional effect) by converting a virtual three-dimensional object in a three-dimensional space into information on a two-dimensional plane by computer calculation. 3DCG is created by processes such as modeling, world transformation, view transformation, projective transformation, and lighting. Modeling and world conversion are performed on dedicated software, and view conversion, projection conversion, and lighting are executed by the display program.

Modeling means that the creator defines the shape of an object in a local coordinate system, for example, creating a shape with polygons or three-dimensional points. World transformation refers to transforming the coordinates of an object into the world coordinate system that represents the three-dimensional space to be drawn (translation and rotation) (arranging the object in the world coordinate system). View conversion is equivalent to determining the viewpoint of the viewer in the world coordinate system, and is a process of setting the coordinates of the camera, the gazing point of the camera, and the upward direction of the camera, and setting the object to be viewed from a specific direction. .. The viewpoint may be determined by the operation of the viewer, or may be changed automatically. Projection transformation refers to projecting an object in three-dimensional space onto a two-dimensional plane. Lighting is a process that gives light and darkness according to the position of the light source.

FIG. 11 schematically shows how the object 302 is projected and transformed on the virtual camera 301. A wine glass object 302 made of polygons or three-dimensional points is projected onto a plane. By changing the position of the virtual camera 301 (view conversion), the wine glass can be projected according to the position of the camera. Since the position (viewpoint) of the virtual camera 301 can be changed to an arbitrary position of 360 degrees around the wine glass, the viewer can view the wine glass from various angles within the limited display frame 7.

FIG. 12 is an example of an object displayed from a different viewpoint. In FIG. 12, the ring is used as an object. In FIG. 12A, the viewpoint is obliquely upward, in FIG. 12B, the viewpoint is upward, in FIG. 12C, the viewpoint is diagonally downward, and in FIG. 12d, the side surface is the viewpoint. In this way, the viewer can see one object from various directions in one display frame 7. The viewer can see the object from various directions by flicking the display frame 7 up / down / left / right.

<About functions>
FIG. 13 is an example of a functional block diagram showing the functions of the terminal device 30, the advertiser Web server 70, the partner site Web server 60, the image distribution device 10, the SSP 50, and the DSP 20 in a block shape.

<< Advertiser Web Server, Partner Site Web Server >>
The advertiser Web server 70 and the partner site Web server 60 have a Web page providing unit 71. Each of these functions possessed by the advertiser Web server 70 and the partner site Web server 60 is a function or means realized by the CPU 206 executing the program 208p expanded from the HDD 208 to the RAM 204.

The Web page providing unit 71 performs, for example, general HTTP communication, constructs a Web page in response to a request for a Web page from the terminal device 30, and transmits the Web page to the terminal device 30. If necessary for processing, the application server is requested to process, and the processing result is arranged on the Web page and transmitted.

The Web page of the advertiser Web server 70 includes an advertiser cookie and a DSP cookie. This is because the advertiser Web server 70 requests the DSP 20 to deliver the advertisement. This enables retargeting advertisements and the like. The Web page of the partner site Web server 60 includes the cookie of the partner site Web server 60 and the SSP cookie. This is because the partner site Web server 60 requests the SSP 50 to sell the display frame 7.

<< SSP >>
The SSP 50 has an advertisement request unit 51, an advertisement request acquisition unit 52, an access information transmission unit 53, and a successful bid unit 54. Each of these functions possessed by the SSP 50 is a function or means realized by the CPU 206 executing the program 208p expanded from the HDD 208 to the RAM 204.

The advertisement request acquisition unit 52 acquires the advertisement request requested together with the SSP cookie by executing the advertisement tag associated with the display frame 7 by the terminal device 30. The advertisement request acquisition unit 52 sends an advertisement request including an SSP cookie to the advertisement request unit 51.

The advertisement request unit 51 transmits an advertisement request to the DSP 20. The advertisement request includes an SSP cookie, a domain of the partner site Web server 60, a display frame ID, a display frame size, an advertisement format, a browser type, an OS type, and the like.

The successful bidder 54 accepts bids from the DSP 20 and basically conducts an auction to sell the display frame 7 to the highest bid price DSP 20 (depending on the advertiser's product or service, the bid is not accepted). The successful bidder 54 generates a successful bid ID and notifies the DSP 20.

The access information transmission unit 53 generates access information 1 for requesting an advertisement from the successful bidder DSP 20 and transmits it to the terminal device 30 together with the successful bid ID. The access information 1 includes the URL (IP address) of the DSP 20.

<< DSP >>
The DSP 20 has a request reception unit 21, a bid determination unit 22, a bid unit 23, an advertisement request reception unit 24, and an image request unit 25. Each of these functions possessed by the DSP 20 is a function or means realized by the CPU 206 executing the program 208p expanded from the HDD 208 to the RAM 204.

Further, in the storage unit 29, a cookie information DB 291 and a distribution setting DB 292 are constructed. The storage unit 29 is realized by the HDD 208 or the RAM 204 shown in FIG.

表１はクッキー情報ＤＢ２９１に記憶される情報を模式的に示す。クッキー情報ＤＢ２９１はＤＳＰクッキーとＳＳＰクッキーを対応付けると共に、閲覧者が訪問した訪問ドメインが登録されている。ＤＳＰクッキーとＳＳＰクッキーの対応付けはクッキーシンクと呼ばれる技術により可能である。また、訪問ドメインはＤＳＰ２０がタグ（行動監視タグ）を貼り付けたＷｅｂサイトを閲覧者が訪問することで得られる。したがって、ＤＳＰ２０は、ＳＳＰクッキーからＤＳＰクッキーを特定でき、更に、この閲覧者がどのようなＷｅｂサイトに興味を持っているかを判断できる。

Table 1 schematically shows the information stored in the cookie information DB 291. The cookie information DB 291 associates the DSP cookie with the SSP cookie, and the visited domain visited by the viewer is registered. The association between the DSP cookie and the SSP cookie is possible by a technique called cookie sink. In addition, the visit domain is obtained by a visitor visiting a website to which the DSP 20 has a tag (behavior monitoring tag) attached. Therefore, the DSP 20 can identify the DSP cookie from the SSP cookie, and can further determine what kind of website the viewer is interested in.

表２は配信設定ＤＢ２９２に記憶される情報を模式的に示す。配信設定ＤＢ２９２には、広告主ＩＤ、広告対象属性（好ましい対象者の属性）、及び、非広告対象属性（好ましくない対象者の属性）が登録されている。広告主ＩＤは広角画像を提供する提供元になる広告主を特定又は識別するための識別情報である。したがって、ＤＳＰ２０又は画像配信装置１０は、広角画像の提供元に対し設定されている好ましい対象者の属性又は好ましくない対象者の属性を参照して、広角画像を送信するか否かを決定することができる。

Table 2 schematically shows the information stored in the distribution setting DB 292. Advertiser ID, advertisement target attribute (favorable target person attribute), and non-advertisement target attribute (undesirable target person attribute) are registered in the distribution setting DB 292. The advertiser ID is identification information for identifying or identifying the advertiser who is the provider of the wide-angle image. Therefore, the DSP 20 or the image distribution device 10 determines whether or not to transmit the wide-angle image by referring to the attributes of the preferred target person or the unfavorable target person set for the provider of the wide-angle image. Can be done.

ID is an abbreviation for Identification and means an identifier or identification information. An ID refers to a name, a code, a character string, a numerical value, or a combination of one or more of these, which is used to uniquely distinguish a specific object from a plurality of objects. The same applies to the following IDs. The advertising target attribute is the attribute of the viewer who the advertiser wants to deliver the advertisement, and the non-advertising target attribute is the attribute of the viewer who the advertiser does not want to deliver the advertisement. The DSP 20 compares the attribute of the viewer determined from the cookie information DB 291 and the information of the distribution setting DB 292, quantifies the attribute of the viewer, and determines the bid amount. In addition, various information such as the time zone, the day of the week, and the area of the viewer may be taken into consideration when determining the bid amount.

(DSP function)
The request reception unit 21 receives an advertisement request from the SSP 50. From the request information, basic information such as the display frame ID of the terminal device 30 can be obtained.

The bid determination unit 22 determines the bid amount by evaluating the advertisement request received by the request reception unit 21 with reference to the cookie information DB 291 and the distribution setting DB 292 (the bid amount may not be bid). As described above, the DSP cookie is specified from the SSP cookie, and the bid amount is determined by how much the attribute information stored in association with the DSP cookie matches the advertiser's request stored in the distribution setting DB 292. To do.

The bidding unit 23 bids on the SSP 50 at the determined bid amount. If a successful bid is successful, the successful bid ID is acquired and associated with the advertisement request.

The advertisement request receiving unit 24 acquires an advertisement request (successful bid ID, DSP cookie) from the terminal device 30 based on the access information 1. The advertisement request reception unit 24 identifies the advertisement request by the winning bid ID. The DSP cookie does not have to be present, and is used when the SSP cookie and the DSP cookie are not associated with each other.

The image requesting unit 25 requests the image distribution device 10 for access information 2 together with the DSP cookie, the advertiser ID, and the request contents. The advertiser is identified by the advertiser ID. The display frame 7 is specified by the display frame ID of the request content. Further, the attributes of the viewer may be transmitted to the image distribution device 10. As a result, the image distribution device 10 can distribute the spherical image 6 suitable for the viewer.

<< Terminal equipment >>
The terminal device 30 has a Web page acquisition unit 31, a Web page analysis unit 32, a Web page display unit 33, an operation reception unit 34, and a script execution unit 35. Each of these functions possessed by the terminal device 30 is a function or means realized by the CPU 601 executing the program 604p developed from the EEPROM 604 to the RAM 603.

The operation reception unit 34 receives various operations on the terminal device 30. Specifically, it accepts operations on the browser executed by the terminal device 30. The operation reception unit 34 receives an operation for a Web page. The image operation receiving unit 43 accepts the operation on the image data based on the script.

The Web page acquisition unit 31 communicates with the advertiser Web server 70 and the partner site Web server 60 by the operation of the viewer or the operation of the script, and acquires the Web page from the advertiser Web server 70 and the partner site Web server 60. ..

The Web page analysis unit 32 reads HTML included in the screen information in order from the beginning and analyzes the structure of sentences and image data included in the HTML. In addition, the association with the HTML text or image data described in CSS is detected, and the style of the text or image data included in the HTML is associated. Further, the Web page analysis unit 32 detects the script tag from HTML and extracts the script written in the script language. The Web page analysis unit 32 sends HTML and CSS to the Web page display unit 33, and sends a script to the script execution unit 35.

The Web page display unit 33 displays the Web pages on the display 616 in the order in which the analysis is completed from the beginning of the HTML. In addition, the Web page is updated according to the operation on the Web page.

The script execution unit 35 executes the script extracted by the Web page analysis unit 32. The specific content of the script varies depending on the Web page. In the present embodiment, as a script, an advertisement tag associated with the display frame 7, access information 1 acquired from the SSP 50, access information 2 acquired from the DSP 20, and advertisement data distributed from the image distribution device 10 are detected. The script execution unit 35 is realized by the CPU 601 shown in FIG. 6 executing the program 604p or the like. Further, each function or means possessed by the script execution unit 35 is realized by the CPU 601 executing the script.

The display frame detection unit 36 is a function realized by the terminal device 30 executing an advertisement tag associated with the display frame 7 included in the screen information acquired by the Web page acquisition unit 31. The display frame detection unit 36 transmits an advertisement request to the SSP 50 together with the SSP cookie based on the URL associated with the display frame 7.

The access information acquisition unit 37 acquires the access information 1 from the SSP 50. Access information 1 is described by a script. The advertisement requesting unit 38 accesses the DSP 20 based on the URL included in the access information 1 and requests an advertisement together with the winning bid ID and the DSP cookie. The advertisement request unit 38 acquires the access information 2 and the advertisement opportunity ID in response to this request. Access information 2 is also described in a script.

The image acquisition unit 39 specifies an advertisement opportunity ID based on the URL of the image distribution device 10 included in the access information 2, and requests the image distribution device 10 for advertisement data. Since the image distribution device 10 generates the advertisement data, the image acquisition unit 39 acquires the advertisement data. The advertising data of this embodiment includes a spherical image 6 (or a 3DCG image), a display pattern, a display program, and an operation history program. The operation history program causes the image distribution device 10 to transmit the operation history. The display program rotates the spherical image 6 or the 3DCG image in the display pattern, and reflects the operation on the spherical image 6 or the 3DCG image in the display mode of the spherical image 6 or the 3DCG image. The operation history program and the display program are also written in the script language and are executed by the script execution unit 35.

The image display unit 40 displays the advertisement data acquired from the image distribution device 10 in the display frame 7. The image display unit 40 is mainly realized by a display program, displays a spherical image 6 or a 3DCG image (that is, an advertisement) on a display frame 7, and automatically rotates the image display unit 40. In addition, the display mode of the spherical image 6 or 3DCG image is changed according to the operation on the spherical image 6 or 3DCG image.

The image operation reception unit 43 has the coordinates of a pointing device (a finger when the display 616 is a touch panel, a rod-shaped member or a dedicated pen and a touch panel, a mouse pointer when a mouse is used, and a mouse pointer when a trackball is used). When it is included in the display frame 7, the operation on the image data is accepted with priority over the operation on the Web page. As a result, when the viewer flicks or swipes on the display frame 7, the display mode of the image data instead of the Web page can be changed.

By acquiring the advertisement data from the image distribution device 10, the terminal device 30 can acquire the image cookie of the image distribution device 10 and store it in the storage unit 49. When the image acquisition unit 39 transmits the image cookie to the image distribution device 10, the image distribution device 10 can associate the DSP cookie with the image cookie.

The operation history recording unit 41 is mainly realized by an operation history program, and records operation information for the spherical image 6 or 3DCG as an operation history. The details of the operation history will be described in the image distribution device 10. The operation history transmission unit 42 is realized mainly by the operation history program, and displays the operation history for the spherical image 6 or 3DCG displayed on the display frame 7 together with the image cookie and the image ID (identifying the spherical image 6). It is transmitted to the image distribution device 10. Further, when the image is clicked, the image distribution device 10 is further transmitted to that effect together with the display pattern.

<< Image distribution device >>
The image distribution device 10 includes an image information response unit 11, an advertisement distribution unit 12, a point of interest determination unit 13, and an operation history acquisition unit 14. Each of these functions included in the image distribution device 10 is a function or means realized by the CPU 206 executing the program 208p expanded from the HDD 208 to the RAM 204.

Further, the image distribution device 10 has a storage unit 19 realized by the HDD 208 or the RAM 204 shown in FIG. A distribution history DB 191, an operation history DB 192, a cookie-compatible DB 193, an advertisement image DB 194, a pattern DB 195, and a display program 196 are constructed in the storage unit 19.

表３は、操作履歴ＤＢ１９２に記憶される情報を模式的に示す。操作履歴ＤＢ１９２には各閲覧者が全天球画像６に対しどのような操作を行ったのかという履歴が記録されている。操作履歴ＤＢ１９２には画像ＩＤ、閲覧時刻、画像クッキー、閲覧画角１〜３の各項目が記録されている。画像ＩＤは全天球画像６を特定するための情報である。画像クッキーは端末装置３０を操作する閲覧者又は端末装置３０の同一性を判別する情報になる。閲覧画角１〜３は閲覧者が全天球画像６を閲覧した際の画角である。各閲覧画角１〜３には閲覧時間、拡大の有無、拡大した場合は拡大後の画角、縮小の有無、及び、クリックの有無が格納されている。このように、各画像ごとに閲覧者が興味を持ったと考えられる画角が記録される。閲覧画角１〜３は、閲覧者が全天球画像６を回転させずに静止した時間が所定時間（例えば１秒）以上の画角である。端末装置３０の操作履歴記録部４１は静止した時間が上位３つの画角を記録する。１つだけでもよいし４つ以上でもよい。同様に、操作履歴記録部４１はそれぞれの閲覧画角１〜３で閲覧者が拡大又は縮小したか否かを記録する。画像配信装置１０はこのような操作履歴を注目点Ｐとする場合がある。更に、画像ＩＤに画像クッキーが対応付けられていればリターゲッティング広告が可能になる。また、閲覧画角には操作履歴としてクリックの有無が記録される。したがって、クリックがある閲覧画角はクリック時画角となる。

Table 3 schematically shows the information stored in the operation history DB 192. The operation history DB 192 records a history of what kind of operation each viewer has performed on the spherical image 6. Each item of the image ID, the viewing time, the image cookie, and the viewing angle of view 1 to 3 is recorded in the operation history DB 192. The image ID is information for identifying the spherical image 6. The image cookie is information for determining the identity of the viewer who operates the terminal device 30 or the terminal device 30. The viewing angles of view 1 to 3 are the angles of view when the viewer views the spherical image 6. The viewing angle of view 1 to 3 stores the viewing time, the presence / absence of enlargement, the angle of view after enlargement when enlarged, the presence / absence of reduction, and the presence / absence of clicking. In this way, the angle of view considered to be of interest to the viewer is recorded for each image. The viewing angles of view 1 to 3 are angles of view such that the time when the viewer stands still without rotating the spherical image 6 is a predetermined time (for example, 1 second) or more. The operation history recording unit 41 of the terminal device 30 records the top three angles of view when it is stationary. It may be only one or four or more. Similarly, the operation history recording unit 41 records whether or not the viewer has expanded or contracted at each viewing angle of view 1 to 3. The image distribution device 10 may set such an operation history as a point of interest P. Further, if the image ID is associated with the image cookie, the retargeting advertisement becomes possible. In addition, the presence or absence of a click is recorded in the viewing angle of view as an operation history. Therefore, the viewing angle of view with a click becomes the angle of view at the time of clicking.

表４は、広告画像ＤＢ１９４に記憶されている情報を模式的に示す。広告画像ＤＢ１９４には、広告の対象となる全天球画像６又は３ＤＣＧに関する情報が登録されている。広告画像ＤＢ１９４には、広告主ＩＤ、画像ＩＤ、優先度、及び、対象属性が対応付けて登録されている。ＤＳＰ２０により広告主が決定されるので、この広告主が配信したい画像うち、閲覧者の属性と表４の対象属性の一致度又は優先度の少なくとも一方に基づいて配信する画像が決定される。なお、リターゲッティング広告の場合は、操作履歴ＤＢ１９２で画像クッキーに対応付けられた画像が配信される。

Table 4 schematically shows the information stored in the advertisement image DB 194. Information about the spherical image 6 or 3DCG to be advertised is registered in the advertisement image DB 194. In the advertisement image DB 194, the advertiser ID, the image ID, the priority, and the target attribute are registered in association with each other. Since the advertiser is determined by the DSP 20, among the images that the advertiser wants to distribute, the image to be distributed is determined based on at least one of the degree of matching or the priority of the attribute of the viewer and the target attribute in Table 4. In the case of the retargeting advertisement, the image associated with the image cookie is delivered in the operation history DB 192.

表５は、クッキー対応ＤＢ１９３に記憶されている情報を模式的に示す。クッキー対応ＤＢ１９３には、ＤＳＰクッキーと画像クッキーが対応付けられている。ＤＳＰクッキーはＤＳＰ２０から通知され、画像クッキーは端末装置３０から通知される。両者が対応付けられることでＤＳＰクッキーだけで画像配信装置１０が閲覧者を判別できるようになる。

Table 5 schematically shows the information stored in the cookie-compatible DB193. The DSP cookie and the image cookie are associated with the cookie-compatible DB193. The DSP cookie is notified from the DSP 20, and the image cookie is notified from the terminal device 30. By associating the two, the image distribution device 10 can identify the viewer only by the DSP cookie.

表６は、パターンＤＢ１９５に登録されている情報を模式的に示す。パターンＤＢ１９５には全天球画像６又は３ＤＣＧ画像の注目点Ｐを表示させる順番及び画角の組み合わせが登録されている。順番及び画角の組み合わせを表示パターンという。注目点Ｐは上記のように閲覧者が閲覧した閲覧画角であるので、表示パターンには閲覧者がよく閲覧する表示位置が含まれている。あるいは、注目点Ｐは画像を解析した結果により決定される。表６では４つの座標１〜４が注目点Ｐである。注目点Ｐの数は１つの全天球画像６又は３ＤＣＧ画像に４つとは限らず３つ以下又は５つ以上でもよい。また、全天球画像６又は３ＤＣＧ画像によって様々でよい。表示パターンは座標１〜４の表示順を有している。したがって、注目点Ｐが４つの場合、表示順は４の階乗個（＝２４）ある。また、各注目点Ｐの画角が例えば３段階に変更される場合、３の４乗の画角の組み合わせがある。したがって、画角に関して１つの表示順について８１通りの組み合わせがある。最終的に１つの全天球画像又は３ＤＣＧ画像について２４×８１の表示パターンがある。クリック回数はこの表示パターンごとに閲覧者がクリックした回数である。つまり、画像配信装置１０は同じ全天球画像６又は３ＤＣＧ画像を同じ表示パターンで全天球画像６又は３ＤＣＧ画像を複数の端末装置３０に表示させ、端末装置３０がクリックした旨を記録する。こうすることで、クリック率の低い表示パターンによる配信を徐々に回避できる。なお、クリック回数は閲覧者の属性（性別、年代、家族構成等）ごとに記録することが好適である。これにより、閲覧者の属性に対しクリック回数が多い表示パターンで表示できる。

Table 6 schematically shows the information registered in the pattern DB 195. In the pattern DB 195, a combination of the order and the angle of view for displaying the attention point P of the spherical image 6 or the 3DCG image is registered. The combination of order and angle of view is called a display pattern. Since the point of interest P is the viewing angle of view viewed by the viewer as described above, the display pattern includes the display position often viewed by the viewer. Alternatively, the point of interest P is determined by the result of analyzing the image. In Table 6, the four coordinates 1 to 4 are the points of interest P. The number of points of interest P is not limited to four in one spherical image 6 or 3DCG image, and may be three or less or five or more. Further, it may vary depending on the spherical image 6 or the 3DCG image. The display pattern has the display order of coordinates 1 to 4. Therefore, when there are four points of interest P, the display order is a factorial of 4 (= 24). Further, when the angle of view of each point of interest P is changed to, for example, three stages, there is a combination of angles of view of 3 to the 4th power. Therefore, there are 81 combinations for one display order regarding the angle of view. Finally, there is a 24x81 display pattern for one spherical image or 3DCG image. The number of clicks is the number of clicks by the viewer for each display pattern. That is, the image distribution device 10 displays the same spherical image 6 or 3DCG image on a plurality of terminal devices 30 with the same display pattern, and records that the terminal device 30 has clicked. By doing so, it is possible to gradually avoid distribution by a display pattern with a low click rate. It is preferable to record the number of clicks for each viewer's attributes (gender, age, family structure, etc.). As a result, it is possible to display in a display pattern with a large number of clicks with respect to the attributes of the viewer.

表７は、配信履歴ＤＢ１９１に登録されている情報を模式的に示す。配信履歴ＤＢ１９１には画像クッキーに対応付けて配信された全天球画像６又は３ＤＣＧ画像の画像ＩＤと表示パターン、クリック有無、及び、クリック時注目点が登録されている。画像配信装置１０が広告データを送信する際に端末装置３０に画像クッキーが登録されている場合、画像配信装置１０はこの画像クッキーを端末装置３０から取得できる。画像配信装置１０は同一の閲覧者に同じ全天球画像６又は３ＤＣＧ画像の配信を避けたり、同じ全天球画像６又は３ＤＣＧ画像を同じ表示パターンで配信することを避けたりすることができる。逆に、リターゲッティング配信することもできる。

Table 7 schematically shows the information registered in the distribution history DB 191. The image ID and display pattern of the spherical image 6 or 3DCG image distributed in association with the image cookie, the presence or absence of a click, and the point of interest at the time of clicking are registered in the distribution history DB 191. If the image cookie is registered in the terminal device 30 when the image distribution device 10 transmits the advertisement data, the image distribution device 10 can acquire the image cookie from the terminal device 30. The image distribution device 10 can avoid distributing the same spherical image 6 or 3DCG image to the same viewer, or can avoid distributing the same spherical image 6 or 3DCG image in the same display pattern. Conversely, retargeting distribution can also be performed.

The attention point at the time of clicking is the attention point P displayed at the time of clicking. Since the same image is distributed to various viewers, the click rate of the attention point P can be calculated by counting the clicked attention points P by the image distribution device 10. When the image distribution device 10 distributes the advertisement data, the click rate of each attention point P is distributed together with the display pattern, so that the terminal device 30 can display the attention point P with a high click rate as a still image.

(About the function of the image distribution device)
The image information response unit 11 assigns an advertisement opportunity ID to the DSP cookie, the advertiser ID, and the request content (mainly the display frame ID) acquired from the DSP 20, and returns the advertisement opportunity ID together with the access information 2 to the DSP 20. The advertisement distribution opportunity can be specified for each display frame 7 of the terminal device 30 by the advertisement opportunity ID.

The advertisement distribution unit 12 determines the spherical image 6 and the display pattern to be distributed when the terminal device 30 requests the advertisement data together with the advertisement opportunity ID. First, the spherical image 6 or the 3DCG image associated with the advertiser ID acquired from the DSP 20 is determined from the advertisement image DB 194. Preferably, the attributes of the viewer are considered. The attribute of the viewer may be notified from the DSP 20, or may be stored by the image distribution device 10 in association with the image cookie. Next, the advertisement distribution unit 12 determines the display pattern of the spherical image 6 with reference to the pattern DB 195. For example, the display pattern having the largest number of clicks is determined, or the display pattern having the number of clicks equal to or greater than the threshold value is arbitrarily determined. It is preferable that the attributes of the viewer are taken into consideration when determining the display pattern.

The operation history acquisition unit 14 acquires the operation history together with the image cookie and the image ID from the terminal device 30, and sets the operation history in the operation history DB 192. There are two types of operation history: an operation history for determining a point of interest and an operation history (clicking) for updating the number of clicks.

The image distribution device 10 may have an operation history recording unit 41 of the terminal device 30. Since the operation history recording unit 41 requires time-series operation contents, it may be difficult for the image distribution device 10 to acquire the operation history depending on the communication band or the like. On the other hand, the operation history recording unit 41 of the terminal device 30 only needs to transmit the final operation history to the image distribution device 10, so that the communication load can be reduced.

The attention point determination unit 13 determines the attention point P of the spherical image 6. Various methods for determining the point of interest P can be considered. For example, a browsing image having a high score is calculated by weighting the browsing time, enlargement, reduction, and click for each viewing angle of view with reference to the operation history DB192. Priority is given to the point of interest from the corner. The attention point determination unit 13 registers a display pattern using the determined attention point P in the pattern DB 195. Alternatively, the image may be divided into several regions and, for example, four of each region may be randomly noted. The image of the region may be evaluated by a predetermined method to determine the point of interest.

<About the function of the image operation reception unit of the terminal device>
FIG. 14 is an example of a functional block diagram for explaining the function of the image display unit 40 in detail. The image display unit 40 includes a display frame display position detection unit 61, a display form control unit 62, a display state storage unit 63, the same image detection unit 64, and an image reproduction unit 65. These are realized by the CPU 601 executing the script (display program 196) included in the advertisement data.

The display frame display position detection unit 61 detects the position of the display frame 7 on the Web page. Thereby, it can be determined whether or not the display frame 7 (that is, image data) is displayed on the screen (display area on the display). The position of the display frame 7 for determining whether or not the display frame 7 is displayed on the Web page is called a detection point.

The display form control unit 62 switches between a single display form and a plurality of display forms of an image. For example, switching is performed at a regular timing such as a user operation such as mouse over, or a lapse of a certain period of time after starting display in a plurality of display modes. When a certain period of time has elapsed after the display frame display position detection unit 61 detects that the display frame 7 is displayed on the screen, the display form control unit 62 switches from the plurality of display forms to the single display form. When the mouse is over, the display mode is switched to the single display mode before a certain period of time has elapsed. When a certain period of time elapses after the single display mode is set, the display form control unit 62 switches to the plurality of display modes. You may return to the multiple display form before a certain period of time elapses by some operation. When the display form control unit 62 returns from the single display form to the plurality of display forms, the display form control unit 62 switches to a plurality of images different from the plurality of images displayed when the detection point is entered on the screen while maintaining the plurality of display forms.

The same image detection unit 64 refers to the display state storage unit 69 for each of the plurality of images included in the advertisement data, and detects whether or not the terminal device 30 has displayed the same image in the past. The display state storage unit 69 will be described in Table 8. The same image detection unit 64 notifies the image reproduction unit 65 of the image ID of the same detected image. When the terminal device 30 displays the same image, the same Web browser (that is, the same Web browser is used on the same terminal device) displays the same image, or when the viewer logs in to the Web browser, the viewer The same thing is said (if the viewers are the same even in different terminal devices 30, it is determined that the same image was displayed in the past).

When the display frame display position detection unit 61 determines that the detection point of the display frame 7 has entered the screen 315, the image reproduction unit 65 starts reproducing the image (changes the display mode such as movement). The reproduction of the image may be started from a plurality of display forms or a single display form. Therefore, when starting from the single display mode, the image is not reproduced even if the display frame display position detection unit 61 determines that the detection point of the display frame 7 has entered the screen 315. In the present embodiment, for convenience of explanation, when it is determined that the detection point of the display frame 7 has entered the screen 315 (a plurality of display states are set), the image reproduction unit 65 starts reproducing the image.

Further, when the image reproduction unit 65 is notified of the image ID of the same image, the image reproduction unit 65 acquires the display state of the image associated with the image ID from the display state storage unit 69, and reproduces from this display state. resume. On the other hand, when the image ID of the same image is not notified, the image is reproduced from the beginning of the display pattern.

When the display form control unit 62 receives a notification that the image displayed in the display frame 7 is switched to another image, the display state storage unit 63 acquires the current display state from the image reproduction unit 65 and stores the display state. Store in part 69. By doing so, when the same image is included in the advertisement data next time, the image reproduction unit 65 can resume the reproduction of the image from the saved display state. The timing of switching the image displayed in the display frame 7 to another image can also be said to be the timing of ending the display of the plurality of images before the display form control unit 62 switches. Further, the display state saving unit 63 may save the timing display state at the timing when the display form control unit 62 switches from the single display form to the plurality of display forms. That is, it is sufficient to save the display state at the timing when the viewer cannot easily see the image any more.

表８は表示状態記憶部６９に記憶されている表示状態に関する情報を示す。表示状態記憶部６９は、再生の全体のうちどの時点を再生していたかを示す情報である。例えば、画像ごとに表示パターンのどこまでを端末装置３０が表示したかの情報である。このため画像ＩＤ、表示パターン、最後の表示位置、及び、操作による表示位置の項目を有している。例えば表示パターンが座標１→座標２→座標３→座標４を移動するとした場合、どの座標間のどの表示位置を表示したかを記憶している。１つの表示パターンが同じ座標を通る場合もあるので、座標のみでは最後の表示位置を特定できない。このため、２点間のどこまで表示したかを記憶している。

Table 8 shows information on the display state stored in the display state storage unit 69. The display state storage unit 69 is information indicating which time point in the whole reproduction was being reproduced. For example, it is information on how far the terminal device 30 has displayed the display pattern for each image. Therefore, it has items of an image ID, a display pattern, a final display position, and a display position by operation. For example, when the display pattern moves coordinate 1 → coordinate 2 → coordinate 3 → coordinate 4, it stores which display position between which coordinates is displayed. Since one display pattern may pass through the same coordinates, the last display position cannot be specified only by the coordinates. Therefore, the distance between the two points is memorized.

As a method of realizing the display state storage unit 69, a cookie managed by a Web browser can be used. Since the cookie is valid only in the domain set by this cookie, the image distribution device 10 writes the cookie in which the domain is appropriately set in the Web browser so that only the image distribution device 10 can access the display state storage unit 69. become.

When the display of the image ends (when the image is switched to the next plurality of images) when the viewer has displayed an arbitrary display position by operation, and the terminal device 30 displays the same image again. It is preferable that the terminal device 30 displays the display position that was displayed when the display of the image is completed. Therefore, in such a case, the display state saving unit 63 saves the display position by the operation instead of the display position in the display pattern. Either the last display position or the display position by the operation is saved.

<About display pattern>
The display pattern will be supplemented with reference to FIG. FIG. 15 is a diagram showing an example of a display pattern. FIG. 15 is a display pattern in which the four points of interest P1 to P4 continuously move in the order of the arrows shown. It is assumed that the angle of view of each point of interest is also determined in advance.

The image reproduction unit 65 displays a part of the image on the display frame 7 while continuously moving the points of interest P1 to P4 based on the display pattern. By doing so, even if a wide-angle image such as the spherical image 6 is a still image, it can be displayed like a moving image.

For example, when the spherical image 6 displayed in the display frame 7 is switched to another image during the movement from the attention points P3 to P4, the display state storage unit 63 switches between the coordinates of the attention points P3 and P4. The coordinates (display position) at that time are stored in the display state storage unit 69. The coordinates can be specified by the latitude θ (0 to 180 degrees) and the longitude φ (0 to 360 degrees).

Even in the case of a 3DCG image, the viewpoint can be changed and the image can be displayed like a moving image. In the case of 3DCG, the terminal device 30 fixes the point of interest at the three-dimensional origin and stores the center of the image sensor of the virtual camera 301 as a display state.

<About detection points>
Next, the detection points will be described with reference to FIG. FIG. 16 is a diagram showing an example of a detection point and an example of calculating the coordinates thereof. In FIG. 16A, a part of the display frame 7 is displayed on the screen 315. The screen 315 is an area of the entire Web page that is actually displayed on the display 616. For the sake of explanation, the upper left corner of the screen 315 is set as the origin (0,0), and the coordinates of the lower left corner of the display frame 7 are represented by (A, B). The origin is the upper left corner of the screen 315 of the Web page, but in the present embodiment, it has the same meaning as the origin of the display 616 (actually, there is a status column such as the remaining battery level). Further, in the terminal device 30 capable of displaying two applications on two screens at the same time, the upper left end of the screen 315 on which the Web page is displayed may not coincide with the origin of the display.

FIG. 16B is a diagram illustrating the size of the display frame 7. Let W be the width and H be the height of the display frame 7. The display program 196 has a function of designating the display frame 7 by a script function and acquiring the coordinates (X, Y) with respect to the origin from the browser software. That is, the relative position of the display frame 7 with respect to the screen 315 on which the Web page 310 is displayed can be acquired. For example, the value of Y becomes negative until the upper left corner of the display frame 7 enters the screen 315. Further, the width W and the height H of the display frame 7 can also be obtained from the browser software.

The display frame display position detection unit 61 can grasp the coordinates (X, Y) of the display frame 7 in real time, and knows the width W and the height H of the display frame 7, so that the coordinates (A) of the lower left corner of the display frame 7 are known. , B) can be calculated.
A = X
B = Y + H
Further, for convenience of explanation, the detection point 320 is set to the center of the display frame 7, and its coordinates are set to (C, D). C and D can be calculated in the same way.
C = X + W / 2
D = Y + H / 2
In this way, the display frame display position detection unit 61 calculates the coordinates of the detection point 320, and when the value of D indicating how much the display frame 7 is displayed in the height direction becomes a positive value, the display frame 7 is displayed. Can be detected on the screen 315. It should be noted that where in the display frame 7 the detection point 320 is set is arbitrary.

Further, the display frame 7 may be included in the screen 315 immediately after the Web page is displayed on the terminal device 30.

<About the structure of the image>
Subsequently, the structure of the image displayed in the display frame 7 will be described with reference to FIG. FIG. 17 is a diagram illustrating a hierarchical structure of images displayed in the display frame 7. In the present embodiment, the image display unit 40 displays the image of the product in the foreground with the spherical image as the background.

As described with reference to FIGS. 7 to 9, since the spherical image is provided as an equirectangular projection image, the image display unit 40 attaches the spherical image to the sphere and converts it into a spherical image. Since the spherical image is a sphere, it cannot be displayed on a flat display as it is. Therefore, the image display unit 40 perspectively transforms the spherical image into the layer L2 for the spherical image. By doing so, the viewer can rotate the spherical image and change the display mode. Alternatively, the display mode changes automatically and looks like a moving image.

On the other hand, the creator of the advertisement prepares the transparent layer L1 in the foreground on the layer L2 of the spherical image, and the image display unit 40 arranges the product image or logo to be always displayed on the layer L1. That is, since the product image and the logo are transparent except for the product image, the layer L2 of the spherical image is visible to the viewer. Layer L1 in the foreground is fixed.

Further, since it is possible to provide a transparent portion in the spherical image, a layer L3 of a transparent background image is provided behind the spherical image. The layer L3 of the transparent background image is fixed.

The image display unit 40 is from the bottom.
Layer L3 of transparent background image
b. Layer L2 of the spherical image obtained by perspective-projection transformation of the predetermined region T
c. Foreground layer L1
To generate an image of the display frame 7 by superimposing.

The image generation method of FIG. 17 is only an example, and only the spherical image may be displayed.

<Operation procedure>
FIG. 18 is an example of a sequence diagram illustrating a procedure for the viewing system 100 to deliver a spherical image 6 or a 3DCG image displayed as an advertisement.

S1: First, the viewer operates the terminal device 30 so as to connect to the partner site Web server 60. The operation reception unit 34 of the terminal device 30 accepts the operation, and the Web page acquisition unit 31 acquires the Web page.

S2: The Web page analysis unit 32 of the terminal device 30 analyzes the Web page, extracts HTML, CSS, and a script, and the Web page display unit 33 displays the Web page based on HTML, CSS, and the like.

S3: The display frame detection unit 36 of the terminal device 30 transmits an advertisement request to the SSP 50 by executing an advertisement tag (script) associated with the display frame 7. The advertisement request includes, for example, an SSP cookie, a domain of the partner site Web server 60, a display frame ID, a display frame size, an advertisement format, a browser type, an OS type, and the like.

S4: The advertisement request acquisition unit 52 of the SSP 50 acquires the advertisement request, and the advertisement request unit 51 accepts the advertisement request. The advertisement request unit 51 detects the SSP cookie, notifies the DSP 20 of the advertisement request, and sells the display frame 7.

S5: The request reception unit 21 of the DSP 20 sends an advertisement request to the bid determination unit 22. The bid determination unit 22 of the DSP 20 refers to the cookie information DB 291 and identifies the DSP cookie from the SSP cookie.

S6: The bid determination unit 22 determines the attribute from the DSP cookie. The visiting domain of the cookie information DB 291 may be referred to, or the attribute information of the viewer specified by the DSP cookie may be referred to as appropriate.

S7: The bid determination unit 22 refers to the distribution setting DB 292, and based on the attributes of the preferred target person or the unfavorable target person set for the provider of the wide-angle image or the 3DCG image, the wide-angle image is given to this viewer. Decide whether to send. Here, it is assumed that it is decided to deliver to at least one advertiser.

S8: The bid judgment unit 22 determines the bid amount according to the advertiser's budget, the degree of matching of attributes, and the like.

S9: The bidding section 23 of the DSP 20 bids on the SSP 50. The successful bidder 54 of the SSP50 determines as the winning bidder the DSP20 that has offered the highest bid amount in principle for the bids from the plurality of DSP20s.

S10: Here, it is assumed that the illustrated DSP20 has made a successful bid. The SSP 50 notifies the DSP 20 of the winning bid ID.

S11: The access information transmission unit 53 of the SSP 50 transmits the access information 1 to the terminal device 30 together with the successful bid ID.

S12: The advertisement request unit 38 of the terminal device 30 transmits the advertisement request to the DSP 20 together with the winning bid ID by executing the access information 1.

S13: The advertisement request reception unit 24 of the DSP 20 identifies the advertisement request by the winning bid ID. The image requesting unit 25 requests an image from the image distribution device 10 together with the advertiser ID, the DSP cookie, and the advertisement request.

S14: When the image information response unit 11 of the image distribution device 10 receives the image request, it generates an advertisement opportunity ID for specifying the advertisement request.

S15: The image information response unit 11 transmits the access information 2 associated with the advertisement opportunity ID to the DSP 20.

S16: The image requesting unit 25 of the DSP 20 receives the access information 2, and the advertisement request receiving unit 24 transmits it to the terminal device 30.

S17: The image acquisition unit 39 of the terminal device 30 transmits an advertisement request (advertisement opportunity ID) based on the URL of the image distribution device 10 included in the access information 2. When the terminal device 30 holds the image cookie, the image cookie can be transmitted.

S18: The advertisement distribution unit 12 of the image distribution device 10 identifies the image request by the advertisement opportunity ID. In the present embodiment, it is assumed that the display frame 7 displays a plurality of images.

S19: The advertisement distribution unit 12 determines the attribute from the DSP cookie or determines the attribute from the image cookie.

S20: The advertisement distribution unit 12 specifies an image to be distributed based on the advertiser ID and attributes. In order to display a plurality of images in the display frame 7, the advertisement distribution unit 12 specifies N images. This N is determined by the display frame 7 (advertising request). The advertisement distribution unit 12 refers to the advertisement image DB 194 and determines an image to be transmitted based on the attribute or priority of the distribution target person set for the image. There are the following combinations of images to be delivered.
(i) N spherical images (or 3DCG images)
(ii) One spherical image (or 3DCG image) + N product images
(iii) N spherical images (or 3DCG images) + N product images
(i) is a case where the advertisement is composed only of the spherical image or the 3DCG image, and is the case where the spherical image or the 3DCG image is the advertisement. (ii) is a case where the spherical image or the 3DCG image as the background is common to the N product images. (iii) is a case where the spherical image or the 3DCG image as the background is different for each product image.

In any of the cases (i) to (iii), the number of images is determined by the number of images displayed in the display frame 7. For example, if the display frame 7 can display three images, the three images are determined. Alternatively, the advertisement distribution unit 12 may determine more images than the number of images that the display frame 7 can display at one time. In this case, the terminal device 30 switches the image to be displayed in time.

Although the number of clicks is higher than the threshold value, the display pattern is randomly selected while the number of distributed spherical images 6 or 3DCG images is small.

S21: The advertisement distribution unit 12 includes N total celestial sphere images 6 or 3DCG images, the size of each image, the display pattern (including the click rate of each attention point), the image ID, the image cookie, and the display frame ID. The advertisement data is transmitted to the terminal device 30. The URL of the advertiser Web server 70 is associated with the spherical image 6 or the 3DCG image.

S22: The image acquisition unit 39 of the terminal device 30 acquires the advertisement data, and the image display unit 40 displays the spherical image 6 or the 3DCG image on the display frame 7. Immediately after the image is displayed, the display form control unit 62 displays the image in a plurality of display forms.

S23: The display form control unit 62 switches from the plurality of display forms to the single display form at regular intervals or in response to the mouse over of the viewer. Therefore, the image display unit 40 requests the image distribution device 10 for the spherical image 6 or the 3DCG image displayed on the display frame 7 one by one or collectively. The request destination (URL) is transmitted to the terminal device 30 together with the advertisement data. This is because the appropriate images are different, such as different aspect ratios, depending on whether N images are displayed in the display frame 7 or only one is displayed. Alternatively, the image transmitted from the image distribution device 10 in step S21 may be enlarged and displayed on the display frame 7. In this case, the communication load and the time until display can be reduced.

Further, the image may be transmitted from the image distribution device 10 to the terminal device 30. In this case, the image distribution device 10 detects the timing of updating the image and transmits the newly determined image to the terminal device.

When the viewer presses (clicks or taps) the display frame 7, the operation reception unit 34 accepts the display frame 7, and the Web page acquisition unit 31 starts communication with the advertiser Web server 70 based on the URL of the advertiser Web server 70.

<When displaying the same spherical image 6 or 3DCG image>
A viewer whose image cookie is registered in the distribution history DB 191 has viewed the spherical image 6 or the 3DCG image in some display pattern. This viewer may browse another Web page of the partner site Web server 60. In this case, the image distribution device 10 can improve the click rate by providing the same spherical image 6 or 3DCG image to the viewer. Such an advertisement is called a retargeting advertisement.

FIG. 19 is an example of a flowchart showing a procedure in which the advertisement distribution unit 12 of the image distribution device 10 distributes the same spherical image 6 or 3DCG image in the same display pattern. This process is executed in step S20 of FIG.

First, the advertisement distribution unit 12 acquires a request for advertisement data from the terminal device 30 together with the image cookie (S101). In the case of the terminal device 30 that has received the distribution of the spherical image 6 or the 3DCG image in the past, since the image cookie is stored, the image distribution device 10 can identify the same Web browser or viewer.

The advertisement distribution unit 12 determines whether or not there is a distribution history of the same Web browser or viewer by referring to the distribution history DB 191 (S102). That is, it is determined whether or not the image cookie is registered in the distribution history DB 191 and the image ID and the delivered display pattern are registered.

If the determination in step S102 is No, the advertisement distribution unit 12 determines the spherical image 6 or the 3DCG image to be distributed from the attributes of the viewer and the like as described in FIG. 18 (S103).

If the determination in step S102 is Yes, the advertisement distribution unit 12 determines that the same spherical image 6 or 3DCG image stored in the distribution history DB 191 is distributed in the same distributed display pattern (S104). If the same viewer has a distribution history of a plurality of spherical images 6 or 3DCG images, the spherical images 6 or 3DCG images that have not been clicked may be distributed, and the spherical images with the latest distribution date and time may be distributed. Sphere image 6 or 3DCG image may be delivered.

In this way, the same spherical image 6 or 3DCG image may be delivered to the terminal device 30 in the same display pattern. In FIG. 19, the case where the same spherical image 6 or 3DCG image is delivered in the same display pattern in the retargeting advertisement has been described, but the same spherical image 6 or 3DCG image is the same based on the attributes of the viewer. It may be delivered in a display pattern.

Alternatively, while the same Web page is continuously displayed, the display form control unit 62 repeatedly switches a plurality of images as a set, but the same image may be displayed after the switching is completed.

<How to display the image of the terminal device>
FIG. 20 is an example of a flowchart for explaining the process when the terminal device 30 displays the spherical image 6 or the 3DCG image. The process of FIG. 20 is executed in step S22 of FIG.

First, the same image detection unit 64 determines whether or not the same spherical image 6 or 3DCG image as in the past is delivered in the same display pattern (S201). That is, it is determined whether or not the image ID and the display pattern included in the advertisement data are already registered by referring to the display state storage unit 69.

If the same image is displayed as the switching goes through while the same Web page is continuously displayed, the same image may be distributed again from the image distribution device 10, and the terminal device 30 may distribute the same image again. Images displayed in the past may be stored and read out.

When the same spherical image 6 or 3DCG image as in the past is not delivered (No in S201), the image reproduction unit 65 reproduces the spherical image 6 or 3DCG image from the beginning at the timing when the detection point enters the screen. (S202). That is, when the display pattern is the point of interest P1 → P2 → P3 → P4, playback is performed from the point of interest P1.

When the same spherical image 6 or 3DCG image as in the past is delivered (Yes in S201), the image reproduction unit 65 resumes reproduction from the last display position at the timing when the detection point enters the screen, or The display position by the operation is displayed (S203). That is, when the display ends while the moving spherical image 6 or the 3DCG image is being displayed, the two points of interest P3 and P4 stored in the display state storage unit 69 and the display position (coordinates) between them are displayed. Resume playback. When the display ends while the viewer operates and displays an arbitrary display position, the display position (coordinates) by the operation stored in the display state storage unit 69 is displayed. In this case, it is displayed as a still image.

Therefore, the viewer can browse the entire advertisement, and the provider of the advertisement can show the viewer the part that he / she wants to show as an advertisement, and the effect of the advertisement can be exhibited. Further, when the viewer is browsing at an arbitrary display position, the display position is displayed again, so that the operability of the viewer can be less likely to be deteriorated.

When the image reproduction unit 65 restarts the reproduction from the last display position when the plurality display form is switched to the single display form, or when the display position by the operation is displayed, Yes and No in step 201 are displayed. Next, it may be determined whether or not the display mode has been switched from the plurality of display modes to the single display mode.

Since the viewer can change the display mode of the whole celestial sphere image 6 or 3DCG image during the reproduction of the whole celestial sphere image 6 or 3DCG image, the image operation reception unit 43 displays the display mode of the whole celestial sphere image 6 or 3DCG image. It is determined whether or not the operation of changing is accepted (S204). For example, the spherical image can be rotated and the viewpoint of the 3DCG image can be changed.

When the operation of changing the display mode of the spherical image 6 or the 3DCG image is accepted (Yes in S204), the image reproduction unit 65 switches to the still image and changes the display position according to the operation of the viewer (S205). .. The reason for switching to a still image is that the viewer wants to see a certain display position. When a certain time has passed since the operation of the viewer is completed, the reproduction of the spherical image 6 or the 3DCG image may be restarted.

Next, the display state storage unit 63 determines whether or not the display form control unit 62 switches the spherical image 6 or the 3DCG image to another image (S206). That is, it is determined whether or not to end the display of the spherical image 6 or the 3DCG image.

If the determination in step S206 is Yes, the display state storage unit 63 stores the image ID, the display pattern, and the last display position (or the display position by the operation) in the display state storage unit 69 (S207). If the same image ID and display pattern are already saved, the display state storage unit 69 overwrites the last display position. This is because it is preferable that the last display position is one for the same image ID and display pattern.

Further, the display state saving unit 63 does not save the display position displayed when the spherical image 6 or the 3DCG image is switched, but the display position displayed when the spherical image 6 or the 3DCG image is switched. You may save a little earlier. As a result, when the playback is resumed, the viewer can see the playback portion that has been viewed once, so that it is easy to remember that the continuation is being played and increase the effect of the advertisement. Further, the display state saving unit 63 may save a little after the display position displayed at the time of switching the spherical image 6 or the 3DCG image.

<Display example>
FIG. 21 is a diagram showing a display example in which the terminal device 30 reproduces from the last display position. The image of FIG. 21 is a spherical image 6 in which the interior of the rental property is captured. The display position of the image 323a changed from FIG. 21 (a) to FIG. 21 (b), and the display was completed. The display position of the image 323b changed from FIG. 21 (c) to FIG. 21 (d), and the display was completed. The display position of the image 323c changed from FIG. 21 (e) to FIG. 21 (f), and the display was completed. Therefore, the display positions of the images 323a to 323c of FIGS. 21 (b), (d), and (f) are stored in the display state storage unit 69.

When the same images 323a to 323c are displayed on the terminal device 30, the terminal device resumes playback from the display positions shown in FIGS. 21 (b), (d), and (f). FIG. 21 (g) shows three images 323a to 323c in a plurality of display states in which reproduction is resumed from the last display position. In FIG. 21 (g), three images 323a to 323c at the display positions of FIGS. 21 (b), (d), and (f) are displayed.

By doing this, the viewer can see the spherical image from the continuation of the previous time. The side that delivers the advertisement can easily display the entire advertisement.

<Summary>
Therefore, the browsing system of the present embodiment can show the entire image to the viewer by displaying the movement of the image from the continuation of the previous time even when the image of the advertisement is switched during playback, and the advertisement. Can fully demonstrate the effect of.

Further, when the viewer changes the display mode of the image by operation, even if the image of the advertisement is switched during the operation, when the terminal device 30 displays the same image next time, the movement of the image is changed from the previous time. By displaying from the continuation, the viewer does not have to operate the same image in the same display mode, and it is possible to suppress deterioration of operability.

In the present embodiment, the advertisement data for switching between the plurality of display modes and the single display form has been described, but it can also be applied to the advertisement having only the single display form. In this case, the display of the image ends when the viewer scrolls the web page during playback of the image, displays another web page, or displays the web page while displaying an arbitrary display position. This is the case when scrolling or displaying another Web page. The display form control unit 62 can detect that the display frame 7 does not enter the screen based on the detection point, and can detect that the terminal device 30 has transitioned to another Web page by the operation of the viewer. Therefore, when the terminal device 30 displays the same image next time, the terminal device 30 can restart the reproduction from the last display position and display the display position by the operation.

Further, in the case of an advertisement having only a single display form, the display may be switched after a certain period of time instead of ending the display by scrolling or the like.

Further, in the present embodiment, since the display mode is changed by changing the display position, the display position is recorded. However, if the reproduction time is the same, the display position is also the same (when the display pattern is also the same), so that the terminal device 30 may record the reproduction time instead of the display position. Further, in the case of a moving image, the terminal device 30 may record the reproduction time. In the case of moving images, the display pattern is unnecessary.

In this embodiment, a display method will be described in which the terminal device 30 reproduces from the display position determined based on the current time and the reproduction time of the spherical image 6 or the 3DCG image instead of the display position. In this embodiment, the terminal device 30 cannot reproduce from the continuation displayed last time, but it is possible to avoid reproducing only the beginning part of the spherical image 6 or the 3DCG image.

In this embodiment, changing the display position by operation may be prohibited. This is because, in this embodiment, it is not always possible to display from the continuation displayed last time, and if the viewer operates the display position, the entire advertisement may not be reproduced. However, the display position may be changed by operation. In this case, as in the first embodiment, when the same spherical image 6 or 3DCG image as in the past is delivered in the same display pattern, the terminal device 30 displays the display position by operation.

<About the function of the image operation reception unit of the terminal device>
FIG. 22 is an example of a functional block diagram for explaining the function of the image display unit 40 in detail. In the description of FIG. 22, since the components having the same reference numerals in FIG. 14 perform the same function, only the main components of the present embodiment may be mainly described.

The image display unit 40 of this embodiment does not have the display state storage unit 63 and the display state storage unit 69. This is because in this embodiment, it is determined from which display position to display based on the current time and the reproduction time. Therefore, it has a reproduction start time determination unit 66. For example, when the playback time is 15 seconds (the movement of the spherical image 6 or the 3DCG image makes a round), the playback start time determination unit 66 divides the current time by 15 seconds and sets the remaining time as the display position for starting playback. And.

If the current time is 13:04:35, 13:04 can be ignored, so
35 ÷ 15 ＝ 2… 5
Is. Since the remainder is 5 seconds, the image reproduction unit 65 resumes reproduction from the display position when 5 seconds have elapsed from the beginning.

<About playback time>
FIG. 23 is a diagram for explaining the reproduction time of the display pattern. FIG. 23 shows the correspondence between time and points of interest P1 to P4. The display pattern contains such information. In this example, it is shown that the display position moves to each attention point P1 to P4 at a constant velocity and stays at each attention point P1 to P4 for a certain period of time. If the same spherical image 6 or 3DCG image has been delivered in the same display pattern in the past, the image reproduction unit 65 sets the above remaining time as the restart position. If the same spherical image 6 or 3DCG image has not been delivered in the same display pattern in the past, the image is reproduced from the beginning of the display pattern.

<How to display the image of the terminal device>
FIG. 24 is an example of a flowchart for explaining the process when the terminal device 30 displays the spherical image 6 or the 3DCG image. The process of FIG. 24 is executed in step S22 of FIG. Further, in the description of FIG. 24, the difference from FIG. 20 will be mainly described.

Step S201 may be the same as in FIG. When the same spherical image 6 or 3DCG image as in the past is delivered (Yes in S201), the playback start time determination unit 66 calculates the remainder from the current time and the playback time (S201-2). The current time is obtained from RTC (Real Time Clock) or the like.

Then, the image reproduction unit 65 resumes reproduction from the remaining time at the timing when the detection point enters the screen (S203-2).

Therefore, the viewer may not always be able to view the entire advertisement, but the provider of the advertisement can prevent the viewer from seeing only the beginning of the advertisement.

The subsequent processing is the same as in FIG. 20, but when the display of the spherical image 6 or the 3DCG image is completed (Yes in S206), the display state storage unit 63 does not record anything because there is no need to record.

Although the current time is determined in this embodiment, the terminal device 30 may simply determine the time at which reproduction is started randomly (randomly). For example, when the reproduction time is 15 seconds, the reproduction start time determination unit 66 divides the reproduction time into, for example, 5 equal parts and determines the time (0,3,6,9,12 seconds) to start the reproduction of 5 points. Then, the time is randomly determined from 0, 3, 6, 9, and 12 seconds.

<Summary>
According to this embodiment, the viewer may not always be able to view the entire advertisement, but as the provider of the advertisement, it is possible to prevent the viewer from seeing only the beginning of the advertisement, and it is easy for the viewer to exert the effect of the advertisement. Become.

<Advertisement data distribution example>
In the first and second embodiments, the advertisement is distributed by a mechanism of third-party distribution, but the advertisement may be finally displayed on the Web page displayed by the terminal device 30. FIG. 25 shows some forms in which the advertisement data is delivered.

In FIG. 25A, the Web page is transmitted with the advertisement data included in the Web page distributed by the partner site Web server 60. In this case, the advertiser has previously submitted the image data to the partner site Web server 60.

In FIG. 25B, the advertisement distribution server 80 distributes the advertisement data. The partner site Web server 60 registers the Web page on which the advertisement is to be placed on the advertisement distribution server 80. As a result, the advertisement distribution server 80 outputs the ad tag, so that the partner site Web server 60 pastes the ad tag on the Web page on which the advertisement is to be placed. Further, the advertiser uploads the advertisement data to the advertisement distribution server 80.

When the viewer accesses the partner site Web server 60, the advertisement distribution server 80 is called by the ad tag. The advertisement distribution server 80 determines which advertisement submitted by each advertiser is to be distributed, and transmits the advertisement data to the terminal device 30.

FIG. 25C is an example of distribution of advertisement data in the embodiment described in this embodiment. As described in the present embodiment, the image distribution device 10 may distribute the advertisement data, or the DSP 20 may distribute the advertisement data. The advertiser may upload the advertisement data to the DSP 20. Further, in the configuration of FIG. 25C, the DSP 20 or the image distribution device 10 serves as the advertisement distribution server 80.

Further, although the image displayed in the display frame 7 has been described as a preferable example of the image, the present embodiment is not limited to the advertisement as long as the image data is displayed on the Web page and changed by the operation of the viewer.

<Other application examples>
Although the best mode for carrying out the present invention has been described above with reference to examples, the present invention is not limited to these examples, and various modifications are made without departing from the gist of the present invention. And substitutions can be made.

For example, in the present embodiment, the example in which the spherical image or the 3DCG image is displayed on the display frame 7 has been described, but the spherical image may be a wide-angle image. That is, the image data may be any image data having an angle of view that does not fit in the display frame 7 and whose viewing range changes depending on the operation of the viewer.

Further, the application of the terminal device 30 for displaying the spherical image is not limited to the browser software, and the present embodiment can be applied when any application software displays the spherical image or the 3DCG image on the screen.

In the present embodiment, the terminal device 30 accesses the DSP 20 and then accesses the image distribution device 10 to acquire advertisement data. However, the DSP 20 may acquire the advertisement data directly from the image distribution device 10 and transmit it to the terminal device 30. Since the terminal device 30 can acquire the advertisement data only by accessing the DSP 20, the time until the advertisement is displayed can be shortened. Further, the DSP 20 and the image distribution device 10 are integrally configured, and the DSP 20 may distribute the advertisement data (omnidirectional image) to the terminal device 30.

Further, the advertisement distribution method in which the SSP 50, DSP 20 and the image distribution device 10 linked with each other described in the present embodiment is only an example, and there is no intention of limiting the process until the terminal device 30 displays the advertisement. For example, as a simple process, the partner site Web server 60 may store the advertisement data from the advertiser and distribute it to the terminal device 30. In this case, the SSP 50, DSP 20 and the image distribution device 10 are not required.

Further, the spherical image is not limited to the case where the two images are pasted together, and for example, the overlapping portion of the captured image data may be superposed while moving the imaging direction little by little. .. That is, the method of creating the spherical image is not limited.

Further, the configuration examples such as FIGS. 13 and 14 shown in the above examples are divided according to the main functions in order to facilitate understanding of the processing of the browsing system 100. However, the present invention is not limited by the method and name of division of each processing unit. The browsing system 100 can be divided into more processing units according to the processing content. It is also possible to divide one processing unit so as to include more processing.

7 Advertising space 10 Image distribution device 20 DSP
30 Terminal device 40 Advertising display unit 50 SSP
60 Partner site web server 70 Advertiser web server 100 Browsing system

Japanese Unexamined Patent Publication No. 2019-036807

Claims (9)

An information processing device that displays a screen based on screen information received from one or more information processing devices via a network.
An image acquisition unit that acquires an image displayed on the screen,
An image reproduction unit that reproduces the image while changing the display mode based on the display pattern,
The image reproduction unit functions as a display state storage unit that stores the display state of the reproduced image.
When the image saved by the display state storage unit is displayed again with the same display pattern, the image reproduction unit restarts the reproduction of the image from the display state saved by the display state storage unit. A program featuring. The program according to claim 1, wherein the display state saving unit saves the display state of the image when the image is switched to another image or the display of the image is finished. The information processing device is made to function as an image operation reception unit that receives a change in the display mode of the image.
The display state storage unit saves the display state of the image received by the image operation reception unit when the image is switched to another image or the display of the image is finished.
When the image saved by the display state storage unit is displayed again with the same display pattern, the image reproduction unit displays the image in the display state saved by the display state storage unit. The program according to claim 2. The image is displayed in a display frame arranged on the screen, has a wider angle of view than the display frame, and has a wider angle of view.
The display state storage unit stores the display position of the image reproduced by the image reproduction unit.
When the image reproduction unit displays the image saved by the display state storage unit again while switching the display position in the same display pattern, the image is displayed from the display position saved by the display state storage unit. The program according to any one of claims 1 to 3, wherein the reproduction of the above is resumed. The image acquisition unit acquires a plurality of images and obtains a plurality of images.
When the plurality of display modes for displaying the plurality of the images in one display frame arranged on the screen and the single display mode for displaying only one of the images are switched and the single display mode is completed, the image acquisition unit is used. Functions as a display form control unit that displays multiple different images acquired by
The claim is characterized in that, when the display form control unit displays a plurality of images having different display forms in a plurality of display forms, the display state saving unit saves the display state of the images displayed in the single display form. The program described in 4. An information processing device that displays a screen based on screen information received from one or more information processing devices via a network.
An image acquisition unit that acquires an image displayed on the screen together with a display pattern,
A playback start time determination unit that determines the time to start playback based on the playback time of the display pattern,
An image reproduction unit that reproduces the image while changing the display mode based on the display pattern from the reproduction start time determined by the reproduction start time determination unit.
A program to function as. The program according to claim 6, wherein the reproduction start time determination unit determines a remainder obtained by dividing the current time by the reproduction time of the display pattern at the time at which the reproduction starts. An information processing device that displays a screen based on screen information received from one or more information processing devices via a network.
An image acquisition unit that acquires an image displayed on the screen,
An image reproduction unit that reproduces the image while changing the display mode based on the display pattern,
The image reproduction unit has a display state storage unit that stores the display state of the reproduced image.
When the image saved by the display state storage unit is displayed again with the same display pattern, the image reproduction unit restarts the reproduction of the image from the display state saved by the display state storage unit. An information processing device characterized by. A display method performed by an information processing device that displays a screen based on screen information received from one or more information processing devices via a network.
A step in which the image acquisition unit acquires an image displayed on the screen, and
A step in which the image reproduction unit reproduces the image while changing the display mode based on the display pattern.
A step in which the display state storage unit saves the display state of the image reproduced by the image reproduction unit, and
When the image saved by the display state storage unit is displayed again with the same display pattern, the step of restarting the reproduction of the image from the display state saved by the display state storage unit. And, a display method characterized by having.

Images