JP2018160225A - Browsing system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a browsing system that can display an image in which a browser is expected to have an interest.SOLUTION: In a browsing system 100 having an information processing apparatus 10 that acquires screen information from a first screen information providing device 30 and a second screen information providing device 50, and an image distribution device 20 that manages the history of operations of the information processing apparatus, the image distribution device comprises: operation history storage means 291 that stores the history of operations in association with identification information on an image and identification information on the information processing apparatus; wide angle image storage means 292 that stores a wide-angle image in association with the identification information on the image; display area determination means 22 that determines the wide-angle image on the basis of the identification information on the information processing apparatus acquired from the information processing apparatus, and determines a display area of the wide-angle image on the basis of the history of operations associated with the identification information of the information processing apparatus; and image transmission means 21 that transmits the wide-angle image and the display area to the information processing apparatus.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a browsing system and a program.

  The Web server provides screen information including various images, and an image can be browsed by a viewer operating an information processing apparatus in which browser software is operating. In addition, imaging devices that can capture a wider range of wide-angle images than conventional digital cameras are becoming popular.

  In addition, images displayed by browser software are often used for advertisements. An advertisement frame is provided on a web page distributed by the web server, and an image to be advertised is displayed in the advertisement frame. When the viewer clicks on the advertisement, the information processing apparatus can communicate with the linked Web server described in the advertisement and display detailed information regarding the target of the advertisement.

  Advertisers want to improve the frequency with which viewers click on advertisements, but the number of advertisements posted on one Web page is increasing, and advertisers devise display methods that attract the attention of viewers (For example, refer to Patent Document 1). Patent Document 1 discloses a technique for improving the effect (click rate) of an advertisement by distributing a video advertisement.

  However, although a moving image may attract the viewer's attention, there is a problem that an image in which the viewer is interested is not always displayed. That is, even if an image that the viewer is interested in is included in a part of the playback time of the moving image advertisement, the viewer must view the moving image advertisement at the playback timing in order to view the image. If the viewer does not watch the video advertisement at this timing, the effect of the advertisement may not be improved.

  In view of the above-described problems, an object of the present invention is to provide a browsing system that can display an image expected to be of interest to a viewer.

The present invention is a browsing system having an information processing device and an image distribution device,
The information processing apparatus includes:
Operation accepting means for accepting an operation on the image of the screen information acquired from the first screen information providing apparatus, operation history for the image accepted by the operation accepting means, identification information of the image, and identification information of the information processing apparatus When the screen information is acquired from the second screen information providing device and the operation history transmitting means for transmitting the information to the image distribution device, the identification information of the information processing device is transmitted to the image distribution device, and the information processing device The wide-angle image acquisition means for acquiring the wide-angle image determined based on the identification information and the display area of the wide-angle image, and the wide-angle image acquisition means based on the screen information acquired from the second screen information providing device. Display means for displaying the display area of the acquired wide-angle image on a display device;
The image distribution device stores an operation history storage unit that stores the operation history in association with the identification information of the image and the identification information of the information processing device, and stores the wide-angle image in association with the identification information of the image. A wide-angle image storage means, determine the wide-angle image based on the identification information of the information processing apparatus acquired from the information processing apparatus, and based on the operation history associated with the identification information of the information processing apparatus There is provided a browsing system including display area determining means for determining the display area of a wide-angle image, and image transmitting means for transmitting the wide-angle image and the display area to the information processing apparatus.

  It is possible to provide a browsing system that can display an image expected to be of interest to the viewer.

It is an example of the figure explaining an omnidirectional image as an example of a wide-angle image. It is a figure explaining the outline of a browsing system. It is a figure explaining the outline of a browsing system. It is an example of the schematic block diagram of a browsing system. It is the block diagram which showed the schematic hardware constitutions of information processing apparatus. It is a use image figure of a spherical camera. It is a figure explaining the outline | summary of the process until an omnidirectional image is produced from the image imaged with the omnidirectional camera. It is a figure explaining the outline | summary of the process until an omnidirectional image is produced from the image imaged with the omnidirectional camera. It is an example of the figure explaining a viewer's eyes | visual_axis. It is an example of the functional block diagram which shows the function of an information processing apparatus, an advertiser website, a partner site, a history management server, and an advertisement distribution server in a block shape. It is an example of the figure which illustrates typically the information which each Web page has. It is an example of the figure explaining the angle of view of a spherical image. It is an example of the figure explaining recording of operation history. It is an example of a sequence diagram until a history management server records an operation history. It is an example of a sequence diagram until an information processing apparatus displays an advertisement. It is an example of the figure explaining the image displayed on an advertiser Web site. It is a figure which shows typically the information which advertisement image DB has. It is a figure which shows typically the angle of view in which two spherical images and the object are reflected. It is an example of a sequence diagram of a procedure for determining a display area based on an operation history. It is a figure which shows an example of the web page of a partner site. It is an example of the figure explaining a heat map. It is a figure which shows the example of a display of the information regarding the omnidirectional image displayed on the advertisement frame. It is an example of the figure explaining the display target object superimposed and displayed on the omnidirectional image. It is an example of the figure explaining the outline of the display method of a display target object. It is an example of the functional block diagram which shows the function of information processing apparatus in a block form. It is an example of the figure explaining a color difference. It is an example of the flowchart figure which shows the procedure in which an information processing apparatus determines and displays the color of a display target object. It is a figure which shows the example of a display of the information regarding the omnidirectional image displayed on the advertisement frame. It is an example of the flowchart figure which shows the procedure in which an information processing apparatus determines and displays the color of a display target object when the position of the display target object with respect to an omnidirectional image changes. It is an example of the figure explaining the background object displayed on a display target object. It is an example of the flowchart figure which shows the procedure in which an information processing apparatus displays a background object. It is an example of the figure explaining the change of a display position. It is an example of the flowchart figure which shows the procedure in which an information processing apparatus displays a background object. It is an example of the figure explaining the example of a display target object. It is an example of the figure explaining the example of a display target object.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

<Outline of Electronic Device System of Present Embodiment>
FIG. 1 is an example of a diagram illustrating an omnidirectional image as an example of a wide-angle image. 1A shows an omnidirectional image represented by a three-dimensional solid sphere CS, and FIG. 1B shows an omnidirectional image represented by an equirectangular projection. The omnidirectional image generated by the omnidirectional camera has a three-dimensional structure in which an image as shown in FIG. The virtual camera IC corresponds to the viewer's viewpoint, and in FIG. 1, the viewpoint is at the center of the omnidirectional image. The viewer can display an arbitrary predetermined region T of the omnidirectional image by rotating three axes around the X axis, the Y axis, and the Z axis passing through the virtual camera IC.

  2 and 3 are diagrams for explaining the outline of the browsing system 100 of the present embodiment. This embodiment demonstrates the browsing system 100 which determines the display area | region of the omnidirectional image displayed by a retargeting advertisement. The retargeting advertisement is one of behavioral targeting advertisements, and is an advertising method for tracking the subsequent behavior of a visitor who has visited an arbitrary website and displaying the advertisement again. In many cases, a viewer previews a product or service at the first visit of the Web site and takes an action pattern of purchasing the product or service at another opportunity. The retargeting tag advertisement is a mechanism for encouraging browsing of a product or service for the second time or later, and may improve the rate at which a viewer clicks on the advertisement (product or service).

  However, many advertisements are displayed on the Web page, and the click rate is not improved simply by the browsing system 100 displaying again the products and services of the Web site visited by the viewer in the past.

  When the omnidirectional image is used for the advertisement image, only a part of the predetermined area T is displayed. The viewer can view the entire image by rotating the omnidirectional image, but only the predetermined area T is displayed immediately after the Web page is displayed.

On the other hand, it is known that the click rate has a large correlation with the initially displayed image, and in order to improve the click rate, the browsing system 100 displays the predetermined area T of the omnidirectional image immediately after the Web page is displayed. How to decide is important. Therefore, the browsing system 100 according to the present embodiment records what area is displayed when viewing the omnidirectional image on the first visit of the website, and displays an advertisement of the omnidirectional image. In this case, the predetermined area T of the omnidirectional image immediately after the Web page is displayed is determined with reference to the area displayed in the past. This will be described with reference to FIGS.
(1) For example, a viewer browses a real estate site (advertiser Web site 30) with browser software (hereinafter referred to as browser 8) of the information processing apparatus 10. In the real estate site, spherical images of various real estate properties are prepared, and the viewer selects a desired property. In FIG. 2, the kitchen part of a certain property is displayed. The real estate site transmits the retargeting tag to the browser 8. The retargeting tag is a script (program) for tracking a viewer.
(2) The retargeting tag communicates with the advertisement distribution server 40, and the advertisement distribution server 40 transmits the generated cookie (denoted as Cookie, but referred to as “cookie” in the present embodiment) to the browser 8. The browser 8 stores cookies. The cookie is identification information for specifying a visitor (or information processing apparatus 10, browser 8) who has visited a real estate site.
(3) While the viewer operates the browser 8 to operate the omnidirectional image, the retargeting tag transmits an operation history indicating which viewing angle of view is displayed to the history management server 20. For example, the viewing angle of view corresponding to a kitchen, close-in, bay window, and toilet is recorded, and how long each is displayed is recorded.

Next, the viewer finished browsing the real estate site and browsed the partner site 50.
(4) The browser 8 displays the web page of the partner site 50.
(5) The advertisement frame of the advertisement distribution server 40 is arranged on this Web page, and a script associated with the advertisement frame transmits a cookie to the advertisement distribution server 40.
(6) The advertisement distribution server 40 compares the transmitted cookie with the recorded cookie, and if it matches, the advertisement distribution server 40 detects that the viewer who has viewed the real estate site has viewed the partner site 50.
(7) The advertisement distribution server 40 performs an auction for the advertising space. In this embodiment, it is assumed that the history management server 20 purchases the advertising space by bidding or the like. The history management server 20 undertakes to advertise the property of the advertiser website 30 from the advertiser website 30. A mechanism like an auction is not always necessary.
(8) The advertisement distribution server 40 transmits an advertisement request tag (script) for requesting an advertisement to the history management server 20 to the information processing apparatus 10.
(9) The information processing apparatus 10 executes the advertisement request tag and transmits a cookie to the history management server 20.
(10) The history management server 20 determines the display area of the omnidirectional image with reference to the browsing history of the omnidirectional image associated with the cookie. For example, when a lot of kitchens are seen, the display area is determined so that the kitchen is included, and the advertisement information (the omnidirectional image and the display area) is transmitted.

  Therefore, when a wide-angle image such as a celestial sphere image is displayed by the retargeting advertisement, the browser 8 immediately displays the partner site website (initial state) immediately after displaying the entire display area in which the viewer is interested. A celestial sphere image can be displayed.

<Terminology>
The operation history in the claims or the detailed description of the invention refers to some operation on the image. Even if it is referred to as “history”, there may be only a single operation. The operation on the image includes, for example, displaying or rotating, enlarging / reducing, changing luminance, changing sharpness, and the like. The operation history may include image selection, image browsing time, and the like.

  The identification information of the information processing apparatus is information for specifying the information processing apparatus, but the browser or the viewer may be identified together with the identification of the information processing apparatus. Moreover, a viewer means the person who browses an advertisement, and may be called a user, a user, an operator, etc.

  Determining the display area of the wide-angle image based on the operation history means estimating and indicating a part, area, range, or place of the omnidirectional image that the viewer is likely to be interested in.

<System configuration example>
FIG. 4 is an example of a schematic configuration diagram of the browsing system 100. The browsing system 100 includes an information processing apparatus 10, an advertiser website 30, a partner site 50, an advertisement distribution server 40, and a history management server 20 that can communicate via a network N.

  The network N is constructed by a LAN constructed in a facility where the information processing apparatus 10 is installed, a provider network of a provider that connects the LAN to the Internet, a line provided by a line operator, and the like. When the network N has a plurality of LANs, the network N is called a WAN or the Internet. The network N may be constructed by either wired or wireless, and wired and wireless may be combined. When the information processing apparatus 10 is directly connected to the public line network, it can be connected to the provider network without going through the LAN.

  The information processing apparatus 10 operates as a client terminal in this embodiment. In the information processing apparatus 10, the browser 8 or application software having an equivalent function is operating. The Web page requested by the information processing apparatus 10 is received from the advertiser website 30 and the partner site 50 and displayed ( It is displayed on an LCD (Liquid Crystal Display) described later.

  The information processing apparatus 10 is, for example, a PC (Personal Computer), a tablet device, a smartphone, a PDA (Personal Digital Assistant), a game machine, a navigation terminal, a wearable PC, or the like, as long as it can display a Web page. For example, when the printer has a function for displaying a Web page or a display, the printer may be used as the information processing apparatus 10. Digital signage may display a web page. Digital signage is a system, display device, or information that is displayed using electronic display devices such as a display in places where people can pass or exist, such as outdoors, stores, public spaces, and transportation facilities. Say.

  The information processing apparatus 10 may be connected to the network N via an access point of a wired LAN or a wireless LAN, or communicates with a switched line type communication standard such as 3G, 4G or LTE (Long Term Evolution) to N may be connected.

  Each of the advertiser website 30 and the partner site 50 is a server (general information processing apparatus) that provides information and functions over a network to a client computer (information processing apparatus 10 of the present embodiment) used by a viewer. is there. In this embodiment, the roles of the advertiser website 30 and the partner site 50 are different. The advertiser website 30 is the side that advertises, and the partner site 50 is the side that provides the advertising space. The roles are not fixed, and the two may be reversed.

  Since the advertiser website 30 requests the history management server 20 to distribute advertisements, if the history management server 20 identifies the information processing apparatus 10 with a cookie, the advertiser website 30 identifies the information processing apparatus 10. The same effect can be obtained. Since the history management server 20 is registered as an advertisement sale target in the advertisement distribution server 40, the advertisement distribution server 40 sends the cookie to the history management server 20 when the information processing apparatus 10 detects the advertisement frame and transmits the cookie. Send. The history management server 20 can determine whether it is worth buying the advertising space based on the cookie.

  The advertiser website 30 and the partner site 50 and the information processing apparatus 10 transmit screen information to the information processing apparatus 10 in response to a request from the information processing apparatus 10 in accordance with HTTP or HTTPs. . The screen information is a program described in HTML, script language, and CSS (cascading style sheet). The structure of the Web page is specified mainly by HTML, the operation of the Web page is specified by the script language, and the Web is specified by CSS. The page style is specified. In the present embodiment, it is a script language that reflects a browser's operation on the Web page in the omnidirectional image. Specifically, a program language called JavaScript (registered trademark) or ECMAScript is known as the script language.

  In the present embodiment, a web page will be described as including a web application. The definition of the web application is not clear, but for example, the advertiser website 30 and the partner site 50 indicate that the web page is dynamically constructed according to the request of the viewer, or the web page operates in a script language. Or point to. The former corresponds to the case where the advertiser website 30 arranges a photograph of real estate that matches the search conditions set by the viewer, for example, and constructs a web page. Is equivalent to rotating.

  The advertisement distribution server 40 can be said to be a server (general information processing apparatus) that sells advertisement opportunities to be distributed to the advertisement space. The advertisement distribution server 40 and the information processing apparatus 10 communicate according to, for example, HTTP or HTTPs. When the information processing apparatus 10 displays a Web page on which the advertising space of the partner site 50 is arranged, the information processing apparatus 10 transmits a cookie to the advertisement distribution server 40. At this time, the information processing apparatus 10 transmits all cookies related to the advertisement distribution server 40. Cookies are provided from various advertiser websites 30 as well as real estate sites. The advertisement distribution server 40 presents all cookies to the history management server 20 (actually, there are a plurality of history management servers 20) and accepts bids for this advertising space. The history management server 20 identifies the attribute of the viewer from the cookie, determines the bid amount in consideration of the attribute and the product to be advertised, and bids. In this embodiment, since the operation history is recorded from the cookie, the history management server 20 can purchase the advertising space as a bid with a higher amount (and faster response).

  The advertisement distribution server 40 charges the history management server 20 according to the contract, and charging according to the contract is performed between the history management server 20 and the real estate site (advertiser Web site). However, whether or not the user is charged only when the advertisement is displayed differs depending on the contract.

  The history management server 20 holds input materials such as banners and texts and advertisement images (hereinafter referred to as advertisement information), and transmits advertisement information in response to a request from the information processing apparatus 10. The advertisement information may be just a banner (an image made up of characters, photos or pictures) or may include a script language in addition to the image. In the present embodiment, a program written in a script language (the above display program) is included in order to reflect the operation of the browser on the Web page in the display mode of the omnidirectional image.

  When the advertisement is clicked, information such as the click of the advertisement is written in a cookie (other than the above) of the browser 8 of the browser and transmitted to the advertisement distribution server 40. Since the cookie contains information for identifying the advertising space, when the viewer accesses the advertiser website 30 by clicking on the advertisement, the advertising distribution server 40 grasps that the advertising space of the partner site 50 is passed. It becomes possible. The advertisement distribution server 40 pays a fee corresponding to the number of clicks to the operator of the partner site 50. Further, the advertiser website 30 detects that the click has been made by the advertisement space purchased by the history management server 20 based on the clicked cookie, and pays a predetermined fee to the history management server 20 based on the contract.

<Hardware configuration example>
FIG. 5 is a block diagram illustrating a schematic hardware configuration of the information processing apparatus 10. The information processing apparatus 10 can be generally implemented as a personal computer, a workstation, or an appliance server. The information processing apparatus 10 includes a CPU 201 and a memory 202 that enables high-speed access of data used by the CPU 201. The CPU 201 and the memory 202 are connected to other devices or drivers of the information processing apparatus 10, for example, a graphics driver 204 and a network device (NIC) 205 via the system bus 203.

  The graphics driver 204 is connected to an LCD (display device) 206 via a bus and monitors a processing result by the CPU 201. A touch panel may be integrally disposed on the LCD 206. In this case, the viewer can operate the information processing apparatus 10 using a finger as an operation means.

  The network driver 205 connects the information processing apparatus 10 to the network N at the transport layer level and the physical layer level to establish a session with the advertiser website 30 and the like.

  An I / O bus bridge 207 is further connected to the system bus 203. On the downstream side of the I / O bus bridge 207, a storage device such as an HDD 209 is connected by an IDE, ATA, ATAPI, serial ATA, SCSI, USB, etc. via an I / O bus 208 such as PCI. . Instead of the HDD 209 or together with the HDD 209, an SSD (Solid State Drive) may be included.

  The HDD 209 stores a program 209p that controls the entire information processing apparatus 10. The program 209p includes screen information transmitted by the advertiser Web site 30, the partner site 50, and the advertisement distribution server 40 in addition to the browser 8. The information processing apparatus 10 executes the display program included in the browser 8 and the screen information to reflect the operation on the Web page in the omnidirectional image. The program 209p may be distributed in a state of being stored in a portable storage medium such as a USB memory or an optical storage medium in addition to being distributed from the advertiser website 30 or the like.

  Further, an input device 210 such as a keyboard and a mouse (referred to as a pointing device) is connected to the I / O bus 208 via a bus such as a USB, and accepts inputs and commands by an operator such as a viewer. .

  The hardware configuration diagram of the advertiser website 30, the partner site 50, the advertisement distribution server 40, and the history management server 20 is the same as or different from that in FIG. There shall be no hindrance. It is preferable that the advertiser website 30, the partner site 50, the advertisement distribution server 40, and the history management server 20 support cloud computing. Cloud computing refers to a usage pattern in which resources on a network are used without being aware of specific hardware resources. In this case, the hardware configuration shown in FIG. 5 does not need to be housed in a single housing or provided as a single device, and is not limited to the advertiser website 30, partner site 50, and advertisement distribution server 40. , And hardware elements that the history management server 20 preferably includes. Further, in order to support cloud computing, the physical configurations of the advertiser website 30, the partner site 50, the advertisement distribution server 40, and the history management server 20 according to the present embodiment may not be fixed, such as a load. It may be configured by dynamically connecting / disconnecting hardware resources according to the situation.

<About spherical images>
The omnidirectional image will be described with reference to FIGS. FIG. 6 is a usage image diagram of the omnidirectional camera 9. As shown in FIG. 6, the omnidirectional camera 9 is used by the viewer to take an image of a subject around the viewer by holding it. The omnidirectional camera 9 has a structure in which the back surfaces of the two imaging elements are opposed to each other, and obtains two hemispherical images by imaging the subject around the viewer.

  Next, an outline of processing until an omnidirectional image is created from an image captured by the omnidirectional camera 9 will be described with reference to FIGS. 7 and 8. 7A shows a hemispherical image (front side) captured by the omnidirectional camera 9, FIG. 7B shows a hemispherical image captured by the omnidirectional camera 9 (rear side), and FIG. 7C. FIG. 4 is a diagram showing an image represented by equirectangular projection (hereinafter, referred to as “equal equirectangular image”). FIG. 8A is a conceptual diagram showing a state in which a sphere is covered with an equirectangular image, and FIG. 8B is a diagram showing an omnidirectional image.

  As shown in FIG. 7A, the image obtained by the omnidirectional camera 9 is a hemispherical image (front side) curved by the fisheye lens. Further, as shown in FIG. 7B, the image obtained by the omnidirectional camera 9 is a hemispherical image (rear side) curved by the fisheye lens. Then, the hemispherical image (front side) and the hemispherical image inverted by 180 degrees (rear side) are synthesized by the omnidirectional camera 9, and as shown in FIG. Created.

  Then, by using OpenGL ES (Open Graphics Library for Embedded Systems), as shown in FIG. 8A, the equirectangular image is pasted so as to cover the spherical surface, and FIG. A celestial sphere image as shown in FIG. In this way, the omnidirectional image is represented as an image in which the equirectangular image is directed toward the center of the sphere. OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data. The omnidirectional image may be a still image or a moving image.

  The omnidirectional image is included in the advertisement information distributed by the advertisement distribution server 40. Since the omnidirectional image is an image that is pasted so as to cover the spherical surface, it is curved and has a sense of incongruity when viewed by humans. Therefore, the information processing apparatus 10 displays a predetermined region T of the omnidirectional image as a flat image with little curvature so as not to give a sense of incongruity to humans. The predetermined area T is indicated by coordinates (X, Y, Z) in a three-dimensional virtual space. On the other hand, since the LCD 206 is a two-dimensional plane, the information processing apparatus 10 cannot display with the predetermined area T as it is. Therefore, the information processing apparatus 10 obtains the predetermined region T by perspective projection conversion in which a three-dimensional object is projected onto a two-dimensional plane using a 3D computer graphic technique. As described above, as shown in FIG. 1, the predetermined area T of the omnidirectional image is displayed on the LCD 206 as a display area.

  FIG. 9 is an example of a diagram illustrating a viewer's line of sight. Since the omnidirectional image has three-dimensional coordinates, the line-of-sight direction is specified by the information specifying the three-dimensional coordinates or the coordinates of the sphere such as latitude / longitude. In the present embodiment, the center cp of the predetermined region T is the line-of-sight direction.

  The viewer can change the line-of-sight direction by operation, but assuming that the virtual camera IC does not move in parallel, the virtual camera IC is a rigid body that rolls (rotates around the Z axis) and yaw (centers around the Y axis). Rotation) and pitch (rotation around the X axis) are possible. The line-of-sight direction changes regardless of which of these three rotations occurs. For example, the yaw angle changes when the viewer rotates the omnidirectional image in the horizontal direction, the pitch angle changes when the omnidirectional image rotates in the vertical direction, and the roll angle when the omnidirectional image rotates around the center of the LCD 206. Changes. In the present embodiment, a browser operation on the web page is reflected in the viewing direction (roll angle, yaw angle, pitch angle) and the like. It is assumed that how it is reflected is described in advance in the display program.

<About functions>
FIG. 10 is an example of a functional block diagram illustrating the functions of the information processing apparatus 10, the advertiser website 30, the partner site 50, the history management server 20, and the advertisement distribution server 40 in a block form.

<< Information processing device 10 >>
The information processing apparatus 10 includes a web page acquisition unit 11, a web page analysis unit 12, a web page display unit 13, an operation reception unit 14, and a script execution unit 15. Each of these functions of the information processing apparatus 10 is a function or means realized by the CPU 201 executing the program 209p expanded from the HDD 209 to the memory 202.

  The operation receiving unit 14 receives various operations on the information processing apparatus 10. Specifically, an operation on the browser 8 executed by the information processing apparatus 10 is accepted. When the Web page has a program (the above script) described in the script language, an operation on the Web page triggered by the script is accepted. In this embodiment, it is not strictly distinguished whether the operation target is the information processing apparatus 10, the browser 8, or the Web page, and the operation on the information processing apparatus 10, the operation on the browser 8, or the operation on the Web page is the same. With the meaning. The operation reception unit 14 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p to control the input device 210 and the touch panel.

  The web page acquisition unit 11 communicates with the advertiser website 30 and the partner site 50 by a browser operation or a script operation, and acquires a web page from the advertiser website 30 and the partner site 50. The Web page acquisition unit 11 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p and controlling the network driver 205.

  The Web page analysis unit 12 reads the HTML included in the screen information in order from the top and analyzes the structure of the text and image data included in the HTML. In addition, a link between HTML text and image data described in CSS is detected, and the style of text and image data included in HTML is associated. Further, the Web page analysis unit 12 detects a script tag from HTML and extracts a script described in a script language. The web page analysis unit 12 sends HTML and CSS to the web page display unit 13 and sends the script to the script execution unit 15. The Web page analysis unit 12 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p.

  The web page display unit 13 displays web pages on the LCD 206 in the order in which the analysis is completed from the top of the HTML. In addition, the web page is updated according to an operation on the web page. The Web page display unit 13 is realized by the CPU 201 shown in FIG. 5 executing the program 209p.

  The script execution unit 15 executes the script extracted by the web page analysis unit 12. The specific content of the script varies depending on the Web page. In the present embodiment, a retargeting tag, a script associated with an advertisement space, an advertisement request tag, and the like are detected as scripts. The script execution unit 15 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p. Each function or means of the script execution unit 15 is realized by the CPU 201 executing the script.

  The cookie acquisition unit 152 is a function realized by the information processing apparatus 10 executing the retargeting tag included in the screen information acquired by the Web page acquisition unit 11 in the script execution unit 15. The cookie acquisition unit 152 acquires a cookie from the advertisement distribution server 40.

  The cookie acquired by the cookie acquisition unit 152 is stored in the storage unit 19. The storage unit 19 is realized by the HDD 209 or the memory 202 illustrated in FIG. The cookies in the storage unit 19 are referred to by the advertisement acquisition unit 151 and the operation history transmission unit 155.

  The advertisement acquisition unit 151 is a function of acquiring an advertisement from the history management server 20 in the script execution unit 15. The advertisement acquisition unit 151 notifies the advertisement distribution server 40 of the cookie by accessing the advertisement distribution server 40 based on the URL associated with the advertisement space (or may be embedded or linked). To do. Thereby, an advertisement request tag is first acquired. An advertisement request tag is also a kind of script.

  The advertisement acquisition unit 151 executes the advertisement request tag and transmits a cookie to the history management server 20. Since the history management server 20 generates advertisement information based on the cookie, the advertisement acquisition unit 151 acquires the advertisement information. The advertisement information of the present embodiment includes a omnidirectional image and a display area, and has a display program for displaying the omnidirectional image. The display program reflects the direct operation on the omnidirectional image on the display mode of the omnidirectional image. The display program is also written in a script language and is executed by the script execution unit 15.

  The advertisement display unit 153 has a function of displaying the advertisement information acquired from the history management server 20 in the script execution unit 15. The advertisement display unit 153 displays an advertisement including an omnidirectional image in the advertisement frame. Further, the display mode of the omnidirectional image is changed according to a direct operation on the omnidirectional image.

  The operation history recording unit 154 records an operation history shown in Table 2 described later. As shown in Table 2, the operation history has an image ID, a browsing time, a cookie, and a viewing angle of view 1 to 3. The image ID is information for specifying the omnidirectional image included in the web page of the advertiser website 30. ID is an abbreviation for Identification and means an identifier or identification information. ID refers to a name, a code, a character string, a numerical value, or a combination of one or more of these used to uniquely distinguish a specific target from a plurality of targets. The same applies to the following IDs. The browsing time is the time when the viewer browses the omnidirectional image of the advertiser website 30. Cookies are described in Table 1. The viewing angle of view 1 to 3 is an angle of view in which the time during which the viewer is stationary without rotating the omnidirectional image is a predetermined time (for example, 1 second) or more. The operation history recording unit 154 records the upper three angles of view during the stationary time. There may be only one or four or more. Further, the operation history recording unit 154 records whether or not the viewer has enlarged or reduced at each viewing angle of view 1 to 3.

  The operation history transmission unit 155 is a function realized by executing a retargeting tag included in the screen information acquired by the Web page acquisition unit 11 in the script execution unit 15. The operation history transmission unit 155 transmits the operation history for the omnidirectional image displayed on the advertiser website 30 to the history management server 20 together with the cookie.

<< Advertiser Website 30, Partner Site 50 >>
The advertiser Web site 30 and the partner site 50 have a Web page providing unit 31. These functions of the advertiser website 30 and the partner site 50 are functions or means realized by the CPU 201 executing the program 209p loaded from the HDD 209 to the memory 202.

  The web page providing unit 31 interprets the HTTP request from the information processing apparatus 10, constructs a web page, and transmits it to the information processing apparatus 10 as an HTTP response. If necessary for the processing, the application server is requested to perform the processing, and the processing result is arranged on the Web page and transmitted. The Web page providing unit 31 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p.

<< Ad Distribution Server 40 >>
The advertisement distribution server 40 includes an advertisement providing unit 41 and a cookie generation unit 42. These functions of the advertisement distribution server 40 are functions or means realized by the CPU 201 executing the program 209p expanded from the HDD 209 to the memory 202.

  In the storage unit 49, a cookie information DB 491 is constructed. The storage unit 49 is realized by the HDD 209 or the memory 202 illustrated in FIG.

Table 1 schematically shows information stored in the cookie information DB 491. The cookie information DB 491 records information on each cookie. In the cookie information DB 491, for each cookie, items of cookie name, cookie value, generation time, expiration date, and domain name are recorded. The cookie name is an arbitrary name given to the cookie by the advertisement distribution server 40. The advertisement distribution server 40 assigns according to a predetermined rule. The cookie value is unique identification information. The generation time is the date and time when the viewer last visited the site. The expiry date is the time when the cookie is valid (available). The domain name is the domain of the advertiser website 30. The browser 8 (viewer) who browsed the advertiser website 30 is specified by the cookie.

(About the function of the advertisement distribution server)
The cookie generator 42 generates a cookie for identifying the browser 8 that has browsed the advertiser website 30. The cookie generation unit 42 stores the cookie in the cookie information DB 491 of the storage unit 49. The cookie generation unit 42 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p.

  The advertisement providing unit 41 interprets the HTTP request transmitted together with the cookie from the information processing apparatus 10, determines the history management server 20 for purchasing the advertisement space, and transmits the advertisement request tag to the information processing apparatus 10 as an HTTP response. . The advertisement providing unit 41 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p.

<< History management server 20 >>
The history management server 20 includes an advertisement image communication unit 21, a display area determination unit 22, and an operation history acquisition unit 23. These functions of the history management server 20 are functions or means realized by the CPU 201 executing the program 209p expanded from the HDD 209 to the memory 202.

  Further, the history management server 20 has a storage unit 29 realized by the HDD 209 or the memory 202 shown in FIG. In the storage unit 29, an operation history DB 291, an advertisement image DB 292, and a distribution setting DB 293 are constructed.

Table 2 schematically shows information stored in the operation history DB 291. The operation history DB 291 records a history of what operation each viewer has performed on the omnidirectional image. In the operation history DB 291, items of image ID, browsing time, cookie, and viewing angle of view 1 to 3 are recorded. The image ID is information for specifying the omnidirectional image. The cookies are as described above. The viewing angle of view 1 to 3 is an angle of view when the viewer browses the omnidirectional image. Each viewing angle of view 1 to 3 stores the viewing time, the presence / absence of enlargement, the angle of view after enlargement when enlarged, and the presence / absence of reduction. In this way, the angle of view that the viewer is considered interested in is recorded for each image. The history management server 20 refers to the viewing angle of view 1 to 3 and determines the display area of the omnidirectional image at the time of advertisement.

Table 3 schematically shows information stored in the advertisement image DB 292. In the advertisement image DB 292, information regarding the omnidirectional image to be advertised is registered. In addition, information regarding the omnidirectional image displayed on the advertiser Web site 30 is registered. The advertisement image DB 292 stores an image ID, an image, and an image content in association with the domain. The domain name is information for specifying the advertiser website 30. The image ID is information for specifying the omnidirectional image displayed in the advertising space. The image ID of the omnidirectional image displayed on the advertiser Web site 30 is included. The image is image data of a spherical image. The image content is information for the advertiser to determine what kind of image the image is. In the case of real estate, a plurality of omnidirectional images with different imaging locations are registered in one property.

Table 4 schematically shows information stored in the distribution setting DB 293. The distribution setting DB 293 is set with what rules the history management server 20 distributes advertisements. Each item of domain name, advertisement target, non-advertisement target, and advertisement content is recorded in the distribution setting DB 293. The domain name is information for associating cookies with advertisement distribution, and the domain of the advertiser Web site 30 is set. The advertisement target is an attribute of a viewer who wants to browse advertisements with priority. The non-advertisement target is an attribute of a viewer who does not want to browse the advertisement. The advertisement content is set as to what kind of advertisement is to be viewed. For example, in the case of a product with the same advertisement content, the same product as the product browsed by the viewer on the advertiser website 30 is the advertisement. When the advertisement content is a similar product, a product similar to the product browsed by the viewer on the advertiser website 30 is the advertisement. In Table 4, for convenience of explanation, a distribution setting is associated with a domain, but a distribution setting may be associated with each image ID. It is possible to determine the advertisement distribution method in detail for each image.

(About history management server functions)
The advertisement image communication unit 21 acquires a cookie from the information processing apparatus 10 and causes the display area determination unit 22 to determine the omnidirectional image and display area to be displayed as an advertisement. In addition, the advertisement information (the omnidirectional image and the display area) is transmitted to the advertisement distribution server 40.

  The display area determination unit 22 specifies the image ID that the viewer was browsing from the operation history DB 291 based on the cookie. Further, the advertisement image DB 292 is searched for the domain included in the cookie, and the advertiser website 30 is specified. The omnidirectional image of this advertiser Web site 30 is specified by the image ID. Then, the display area of this omnidirectional image is determined by the viewing angle of view 1 to 3 in the operation history DB 291.

  First, it is considered that the angle of view that the viewer is interested in is displayed for a long time. Therefore, first, the viewing angle of view with the longest viewing time is determined as the display area. Furthermore, enlargement and reduction may be considered. Enlarging is considered to be interested in the viewer, and reducing is considered to be interested in a wider range. The display area determination unit 22 converts the viewing time, enlargement, and reduction into points, and determines the angle of view with the highest point as the display area.

Point of view angle of view = 10 seconds x 1.5 (with enlargement) = 15
Point of view angle of view 2 = 5 seconds x 0.8 (with reduction) = 4.8
Point of view angle of view 3 = 3 seconds x 1.5 = 4.5
The operation history acquisition unit 23 acquires the operation history together with the cookie from the information processing apparatus 10 and sets it in the operation history DB 291. The operation history acquisition unit 23 is realized by the CPU 201 illustrated in FIG. 5 executing the program 209p to control the network driver 205, and the like.

  The history management server 20 may include the operation history recording unit 154 of the information processing apparatus 10. Since the operation history recording unit 154 requires time-series operation content, it may be difficult for the history management server 20 to acquire depending on the communication band. On the other hand, since the operation history recording unit 154 of the information processing apparatus 10 only needs to transmit only the final operation history to the history management server 20, the communication load can be reduced.

<Information held by each Web page>
FIG. 11 is an example of a diagram schematically illustrating information included in each Web page. FIG. 11A is a web page provided by the advertiser website 30, and FIG. 11B is a web page provided by the partner site 50.

  The web page provided by the advertiser website 30 has an omnidirectional image 401, and the omnidirectional image has an image ID. In addition, a retargeting tag 404 is attached to the information processing apparatus 10 by a web page provided by the advertiser website 30. The URL of the advertisement distribution server 40 is described in the retargeting tag 404, and the browser 8 executes the retargeting tag 404 and acquires the cookie 403 from the advertisement distribution server 40. Further, the URL of the history management server 20 is described in the retargeting tag 404, and the browser 8 executes the retargeting tag 404 and transmits the cookie 403, the image ID, and the operation history to the history management server 20.

  The web page provided by the partner site 50 has an advertising space 402. The advertisement frame 402 is associated with a script such as an ad tag and an advertisement frame ID, and the URL of the advertisement distribution server 40 is described in the script. The browser 8 executes the script and transmits the cookie 403 to the advertisement distribution server 40.

<About viewing angle of view>
FIG. 12 is an example of a diagram illustrating the viewing angle of view of the omnidirectional image. The omnidirectional image of FIG. 12 is represented by an equirectangular projection. When the viewer operates, the predetermined area T is displayed with the whole celestial sphere as shown in FIG. In this case, the viewing angle of view of the omnidirectional image is expressed by coordinates of four points cut out from the omnidirectional image. Since the predetermined area T is a rectangle, in this embodiment, the viewing angle of view is indicated only by the diagonal vertex.

<Operation history>
FIG. 13 is an example of a diagram for explaining operation history recording. The viewer operates the omnidirectional image of the web page provided by the advertiser website 30. In FIG. 13, the omnidirectional image of the web page is simplified by equirectangular projection. Since this omnidirectional image is of a real estate property, the kitchen 341, the entrance 342, the window 343, and the veranda 344 (the ceiling and the floor are shown but omitted) are shown.

  The viewer rotates the omnidirectional image and browses an object (location) of interest. Although it is not stationary during the rotation, the operation history recording unit 154 records the viewing angle of view 301 to 303 and the viewing time when the viewing time exceeds a predetermined value. Also, the presence / absence of enlargement / reduction, and the viewing angle of view after enlargement / reduction are recorded.

<Operation procedure>
The operation of the browsing system 100 will be described with reference to FIGS. FIG. 14 is a sequence diagram until the history management server 20 records an operation history, and FIG. 15 is a sequence diagram until the information processing apparatus 10 displays an advertisement.

  S1: First, the viewer performs an operation for starting the browser 8. The information processing apparatus 10 activates the browser 8.

  S2: The viewer performs an operation to display the advertiser website 30 and the operation reception unit 14 of the information processing apparatus 10 receives the URL and IP address of the advertiser website 30. The web page acquisition unit 11 of the information processing apparatus 10 requests a web page from the advertiser website 30.

  S3: The web page providing unit 31 of the advertiser website 30 transmits screen information including the retargeting tag to the information processing apparatus 10. The retargeting tag may not be transmitted simultaneously with the screen information.

  S4: The web page analysis unit 12 analyzes the web page, detects a script (retargeting tag), and sends HTML and CSS to the web page display unit 13. Thereby, the Web page of the advertiser Web site 30 is displayed.

  S5: When the retargeting tag is executed, the cookie acquisition unit 152 requests a cookie for the URL of the advertisement distribution server 40 described in the retargeting tag. At this time, the domain name of the web page is transmitted to identify the advertiser website 30.

  S6: The cookie generation unit 42 of the advertisement distribution server 40 generates a cookie including a cookie name, a cookie value, a generation time, an expiration date, and a domain name, and transmits the cookie to the information processing apparatus 10.

  S7: The cookie acquisition unit 152 of the information processing apparatus 10 receives the cookie and stores it in the storage unit 19.

  S8: The operation reception unit 14 of the information processing apparatus 10 receives an operation on the omnidirectional image of the viewer and notifies the operation history recording unit 154 of the operation. The operation history recording unit 154 records an operation history. When the viewer browses a plurality of omnidirectional images, the operation history is recorded for all the omnidirectional images.

  S9: The operation history transmission unit 155 of the information processing apparatus 10 transmits the operation history and the cookie together with the image ID to the history management server 20 with respect to the URL of the history management server 20 described in the retargeting tag. Note that the timing for transmitting the operation history may be when the viewer has finished browsing the advertiser website 30 or has finished browsing one omnidirectional image and transitioning to browsing the next omnidirectional image. It may be when you do. Or you may transmit by an explicit operation of a viewer.

  S10: The operation history acquisition unit 23 of the history management server 20 receives the image ID, the operation history, and the cookie, and stores them in the operation history DB 291.

  S11: Next, the viewer performs an operation of displaying the partner site 50, and the operation receiving unit 14 of the information processing apparatus 10 receives the URL and IP address of the partner site 50. The web page acquisition unit 11 of the information processing apparatus 10 requests a web page from the partner site 50.

  S12: The web page providing unit 31 of the partner site 50 transmits screen information including the advertising space to the information processing apparatus 10. The advertisement space may not be transmitted simultaneously with the screen information.

  S13: The web page analysis unit 12 analyzes the web page, detects a script associated with the advertising space, and sends HTML and CSS to the web page display unit 13. Thereby, the web page of the partner site 50 is displayed.

  S14: When the advertisement space script is executed, the advertisement acquisition unit 151 transmits the cookie and the advertisement space ID to the URL of the advertisement distribution server 40 described in the advertisement space.

  S15: The advertisement providing unit 41 of the advertisement distribution server 40 receives the cookie and the advertisement space ID. The advertisement providing unit 41 of the advertisement distribution server 40 searches the cookie information DB 491 with the received cookie. When a cookie is registered in the cookie information DB 491, the advertisement distribution server 40 detects that a viewer who is a target of retargeting has visited the partner site 50.

  S16: The browser 8 of the information processing apparatus 10 often stores a plurality of cookies. In this case, the advertisement acquisition unit 151 transmits all cookies related to the advertisement distribution server 40 to the advertisement distribution server 40. The advertisement providing unit 41 bids based on the domain that the cookie has. When a cookie is transmitted to the history management server 20, the history management server 20 bids based on the cookie value of the cookie, the generation time, and the attributes of the viewer.

  S17: The advertisement providing unit 41 sells the advertising space to the history management server 20 that satisfies the bidding criteria such as the bid amount. In the present embodiment, it is assumed that the history management server 20 has identified the viewer of the advertiser website 30 with the operation history based on the cookie value and purchased the advertising space.

  S18: The advertisement providing unit 41 of the advertisement distribution server 40 transmits the advertisement request tag (program) and the advertisement space ID to the information processing apparatus 10. The advertisement request tag includes a URL of the history management server 20 that purchased the advertisement space, and is a program for the information processing apparatus 10 to request an advertisement distributed to the advertisement space purchased by the history management server 20.

  S19: The advertisement acquisition unit 151 of the information processing apparatus 10 acquires an advertisement request tag and executes it, thereby transmitting an advertisement request and a cookie to the history management server 20.

  S20: The advertisement image communication unit 21 of the history management server 20 receives the advertisement request and sends a cookie to the display area determination unit 22. The display area determination unit 22 refers to the distribution setting DB 293 and confirms the distribution setting. Check the viewer ’s attributes and make sure it ’s advertised and not non-advertised. Viewer attributes include age, gender, occupation, family composition, annual income, and location. The history management server 20 stores viewer attributes in association with cookies. Note that the browser attributes may be acquired from the advertisement distribution server 40 or may be acquired from a server that specially stores the viewer attributes. The display area determination unit 22 of the history management server 20 determines the advertisement content with reference to the distribution setting DB 293. Here, it is assumed that the same product is designated.

  S21: The display area determination unit 22 searches the operation history DB 291 using cookies. Thereby, the image ID and the viewing angle of view 1 to 3 are determined. There may be a plurality of image IDs associated with cookies. This is a case where a viewer browses a plurality of spherical images. In this case, all the plurality of omnidirectional images may be transmitted. Alternatively, the omnidirectional image having the highest viewing angle of view may be determined, or the omnidirectional image having the earliest or latest viewing time may be determined.

  S22: The display area determination unit 22 converts the viewing angle of view 1 to 3 into points, and determines the viewing angle of view with the highest point as the display area. Since the advertisement content is “same image”, the display area determination unit 22 reads the omnidirectional image specified by the domain and the image ID from the advertisement image DB 292.

  S23: The advertisement image communication unit 21 transmits the advertisement information (the omnidirectional image and the display area) to the information processing apparatus 10.

  S24: The advertisement acquisition unit 151 of the information processing apparatus 10 receives the advertisement information. The advertisement display unit 153 displays the omnidirectional image at the angle of view determined in the display area in the advertisement frame for which the advertisement is requested by the advertisement request tag.

  As described above, the browsing system 100 according to the present embodiment can display the omnidirectional image at an angle of view where the viewer is interested by tracking the viewer with the cookie. Since the information processing apparatus 10 can display the omnidirectional image in the display area in which the viewer is interested immediately after displaying the website (initial state), the click rate can be improved.

  In this embodiment, the advertisement distribution server 40 transmits the advertisement request tag to the information processing apparatus 10, but the history management server 20 acquires the advertisement information from the advertisement distribution server and transmits it to the information processing apparatus 10 as it is. May be.

  In addition, the information processing apparatus 10 acquires screen information including a retargeting tag from the advertiser website 30 and acquires screen information including an advertising space from the partner site 50. For example, the information processing apparatus 10 includes a retargeting tag. Screen information including screen information and advertisement space may be acquired from the advertiser website 30. Furthermore, screen information including a retargeting tag and screen information including an advertising space may be acquired from the partner site 50.

  In step S9, the information processing apparatus 10 transmits an operation history, a cookie, and an image ID to the history management server 20, but the history management server 20 determines the image ID operated by the information processing apparatus 10 separately. (For example, an image to be displayed from the advertiser website 30 is notified to the history management server 20, and the history management server 20 transmits the image to the information processing apparatus 10, thereby the history management server 20. Can identify an image to be displayed on the information processing apparatus 10, or the information processing apparatus 10 analyzes the screen information and separately acquires and displays an image to be operated from the history management server 20, thereby displaying the history management server. When 20 can identify an image displayed on the information processing apparatus 10), the information processing apparatus 10 may not transmit the image ID in step S <b> 9.

  In this embodiment, a browsing system 100 that can record an operation history even when a viewer is browsing an image other than the omnidirectional image will be described. In addition, a browsing system 100 that can display different spherical images on the advertiser website 30 and the partner site 50 will be described.

  Since the system configuration, hardware configuration, and functional block diagram are the same as those in the first embodiment, the differences will be mainly described.

<When a flat image is viewed on the advertiser website>
FIG. 16 is an example of a diagram illustrating an image displayed on the advertiser website 30. In this advertiser Web site 30, four plane images 311 to 314 of a property with real estate are prepared. When the viewer clicks on any of the planar images 311 to 314, the clicked planar image is displayed in a large size in the display column 320. Since such plane images 311 to 314 have a fixed angle of view, the operation history recording unit 154 cannot record the viewing angle of view. However, since the advertiser wants to know the object that the viewer is interested in, if it is understood what is shown in the planar images 311 to 314, an appropriate advertisement can be provided using that.

  In the case of the planar images 311 to 314, for example, the imaging target is recorded in the metadata of the planar images 311 to 314 (data in which information regarding the planar image is recorded). The imaging target of the planar image 311 is the kitchen, the imaging target of the planar image 312 is the entrance, the imaging target of the planar image 313 is the window, and the imaging target of the planar image 314 is the veranda. The operation history recording unit 154 records an imaging target of the planar image enlarged and displayed in the display field 320 as an operation history.

  On the other hand, as illustrated in FIG. 17, the history management server 20 associates the angle of view of the omnidirectional image and the imaging target and holds them in the advertisement image DB 292. FIG. 17 is a diagram schematically illustrating information included in the advertisement image DB 292. In FIG. 17, the kitchen, the entrance, the window, and the veranda are shown in one spherical image. In the advertisement image DB 292, the angles of view of the kitchen 341, the entrance 342, the window 343, and the veranda 344 are registered.

  Therefore, if it is recorded in the operation history DB 291 that the display time of the flat image 311 (kitchen) is the longest, the display area determination unit 22 displays the angle of view in which the kitchen is reflected in the omnidirectional image as an advertisement. The initial display area can be determined.

  Thus, even when the viewer is browsing an image other than the omnidirectional image, the operation history is recorded and the display area of the advertisement can be determined. The four plane images 311 to 314 and the omnidirectional image may be images of the same property, or may be images of different properties.

<When displaying different spherical images on the advertiser website and partner site>
By adopting the same mechanism for recording the operation history, the browsing system 100 can display different spherical images on the advertiser website 30 and the partner site 50.

  FIG. 18 schematically shows two celestial sphere images and an angle of view in which the object is captured. Since the two spherical images are of different properties, what is reflected in which angle of view is not the same. However, as shown in FIG. 17, if the object shown in the angle of view of the omnidirectional image is set in the advertisement image DB 292, the operation history recording unit 154 records the display angle of view so that the display angle of view is displayed. The display area determination unit 22 can determine the object that is shown. For example, it is assumed that the omnidirectional image of FIG. 18A is displayed on the advertiser website 30 and (Xs1, Ys1) (Xe1, Ye1) is recorded as the viewing angle of view. Since the viewing angle of view is set in the advertisement image DB 292 to be a kitchen, when the omnidirectional image of FIG. 18B is displayed on the partner site 50, it is the angle of view of the kitchen (Xs2, Ys2 ) (Xe2, Ye2) is the display area.

  <Advertising image DB>

The advertisement image DB 292 of the present embodiment will be described using Table 5. Table 5 mainly explains differences from Table 3. Table 5 (a) shows the advertisement image DB 292 in which the planar image is registered. An image ID and a target are registered for each plane image. Table 5 (b) shows the advertisement image DB 292 in which the spherical image is registered. The advertisement image DB 292 in Table 5 (b) has items of target and target angle of view. The target and the target angle of view are information in which the target being imaged is associated with the angle of view.

  When a planar image is browsed on the advertiser website 30, the display area determination unit 22 identifies a target associated with the planar image browsed in Table 5 (a), and the whole sky in which this target is reflected. The angle of view of the spherical image can be specified from Table 5 (b). Further, when the omnidirectional image is browsed, the target associated with the display angle of view (the point is the highest) of the omnidirectional image browsed in Table 5 (b) is specified, and the same target as this target From Table 5 (b), the angle of view of another omnidirectional image in which can be seen can be specified.

<Operation procedure>
Of the operations of the browsing system 100 according to the present embodiment, the operation history may be recorded as in the first embodiment. FIG. 19 is an example of a sequence diagram of a procedure for determining a display area based on an operation history. In the description of FIG. 19, differences from FIG. 15 are mainly described. The processes in steps S10 to S22 are the same as those in FIG. However, it is assumed that the advertisement content is determined as “similar products” in step S20.

  S22-2: The display area determining unit 22 refers to the target and the target angle of view of the advertisement image DB 292 specified by the domain and the image ID included in the cookie, and the target angle of view closest to the display area determined in step S22. To decide. The distance between the center of the display area and the center of the target angle of view may be compared. And the object matched with this object view angle is specified.

S22-3: Since the advertisement content is “similar products”, the display area determination unit 22 reads the omnidirectional image specified by the domain and the image ID included in the cookie from the advertisement image DB 292. Then, an image similar to the omnidirectional image is specified from the advertisement image DB 292. There are several ways to identify similar images.
・ Identify similar celestial sphere images by image recognition processing such as pattern matching ・ Compare attributes of each celestial sphere image (property) (detached house, condominium, number of rooms, environment, distance to station, etc.) Identify similar omnidirectional images ・ Determine that omnidirectional images viewed by other viewers who viewed the same omnidirectional image viewed by the viewer are similar ・ Browsers Omnidirectional images viewed by viewers with similar attributes to the attributes (age, gender, occupation, family composition, annual income, location) of
S22-4: The display area determination unit 22 determines in which angle of view the target determined in step S22-2 is reflected in a similar omnidirectional image with reference to the target and target angle of view of the advertisement image DB 292. To do. This target angle of view is determined as a display area. The subsequent processing may be the same as in FIG.

  As described above, the browsing system 100 according to the present embodiment can determine the display area of the omnidirectional image based on the target estimated to be interested in the operation history. Therefore, the image when the operation history is recorded is not limited to the omnidirectional image. In addition, when displaying an omnidirectional image similar to the omnidirectional image displayed on the advertiser Web site 30 on the partner site 50, it can be displayed in a display area optimum for the viewer.

  In this embodiment, the target and the target angle of view are registered in the advertisement image DB 292. However, the display area determination unit 22 may determine the display area by image processing. When the display area is determined based on the points, the display area determination unit 22 identifies an object (for example, a kitchen) shown in the display area by pattern matching using machine learning or the like. By specifying a target angle of view in which the same target is captured with similar pattern matching for similar omnidirectional images, a display area in which the target is captured in similar omnidirectional images can be determined.

  In the present embodiment, a browsing system 100 that allows a viewer to switch the omnidirectional image displayed as an advertisement on the Web page of the partner site 50 will be described. The system configuration, hardware configuration, and functional block diagram are the same as those in the first embodiment, and the differences will be mainly described.

  FIG. 20 shows an example of the web page of the partner site 50 of the present embodiment. In FIG. 20, the advertising space is shown enlarged. This advertising space has an omnidirectional image display field 332 and a map field 331. In the map column 331, the imaging positions of the omnidirectional images are indicated by A to C. When the viewer presses the imaging positions A to C, the operation receiving unit 14 receives the advertisement, and the advertisement display unit 153 displays the omnidirectional image of the pressed imaging position in the omnidirectional image display field 332.

  In this way, the viewer can select and display a desired omnidirectional image from a plurality of omnidirectional images. In the example of FIG. 20, omnidirectional images of different imaging positions of the same property can be selected, but omnidirectional images of different properties may be selectable.

  A method for determining a plurality of omnidirectional images will be described. The display area determination unit 22 acquires the omnidirectional image of the image ID specified from the cookie in the operation history DB 291 from the advertisement image DB 292. At this time, since the omnidirectional images having the same part of the image ID are omnidirectional images of the same property, it is determined to be transmitted together. As for the initial display area, the display areas of the omnidirectional images having exactly the same image ID are matched.

  When the plurality of omnidirectional images similar to the omnidirectional image having the image ID specified from the cookie in the operation history DB 291 are transmitted to the information processing apparatus 10 as in the second embodiment, the display area determination unit 22 determines the similarity. The top three to four celestial sphere images with the highest are identified. In this case, as described in the second embodiment, an initial display area can be determined for each similar omnidirectional image.

  Although the operation history recording unit 154 of the information processing apparatus 10 has been described as determining the display area based on the viewing time, the display area may be determined by a technique called a heat map.

  FIG. 21 is an example of a diagram illustrating a heat map. The operation history recording unit 154 records the displayed viewing angle of view for a predetermined time or more. When a plurality of viewing angle of view overlap, an average center point 352 is calculated from each center point 351b. Then, a value for weighting the center point 352 is determined so that the larger the number of viewing angle of view overlaps, the higher the value becomes. In the weighting values of the points 351a and 352 in FIG. 21A, the point 352 is larger.

  FIG. 21 (b) shows an omnidirectional image visually representing a viewing angle of view that is often seen. Lattice points are set at equal intervals in the vicinity of the center point, and a weight value that decreases as the distance from the center point decreases. By drawing in a darker color as the weighting value is larger, a heat map-like visual effect as shown in FIG. 21B is obtained.

  The display area determination unit 22 of the history management server 20 determines a predetermined angle of view including the center point having the highest weight value as the display area.

  By using a heat map, even if the viewer continuously changes the viewing angle of view and the viewing time is not sufficient, the viewing angle at which the viewer has shown interest can be determined as the display area.

  In the present embodiment, a browsing system 100 that improves the visibility of information related to the omnidirectional image displayed in the advertising space will be described.

  FIG. 22 shows a display example of information related to the omnidirectional image displayed in the advertisement frame 402. The information about the omnidirectional image is a sentence that prompts the user to click (or tap) the omnidirectional image. Contents, link destination URLs, and the like. Hereinafter, information regarding the omnidirectional image is referred to as a display object 51.

  The problem assumed by the present embodiment will be described with reference to FIG. 22A, the display object 51 is displayed below the advertisement frame 402, the display object 51 is displayed above the advertisement frame 402 in FIG. 22B, and the advertisement frame 402 in FIG. 22C. The display object 51 is displayed on the left side of FIG. 22, and the display object 51 is displayed on the right side of the advertising space 402 in FIG. Although the display object 51 in FIG. 22 is represented by a rectangular area, it should be understood that, for example, text (characters) is described here.

  When the viewer clicks (or taps) the display object 51, the information processing apparatus 10 connects to the link destination URL embedded in the display object 51. Since the image part of the omnidirectional image can be rotated, enlarged / reduced, etc., by operations other than clicking (or tapping), information is available even if the viewer accidentally clicks (or taps) the omnidirectional image. The processing device 10 is not connected to the link destination.

  When the display object 51 is displayed so as to overlap the omnidirectional image in this way, the following inconvenience occurs. This will be described with reference to FIG.

  FIG. 23 is an example of a diagram for explaining the display object 51 displayed so as to overlap the omnidirectional image. FIG. 23 shows the whole celestial sphere image displayed by the equirectangular projection. Accordingly, a part of the angle of view is displayed in the advertisement frame 402. When the angle of view A in FIG. 23 is displayed in the advertising space 402, the display object 51 “go to site” is displayed with the sky in the background, so that the visibility is good. However, the viewer can arbitrarily change the angle of view of the omnidirectional image in the advertisement frame 402 by scrolling the celestial sphere image up and down, left and right, or by enlarging or reducing it. In addition, in order to browse the entire omnidirectional image, the browser may rotate the omnidirectional image of the advertisement frame 402 automatically (for example, in the horizontal direction).

  On the other hand, the position of the display object 51 in a certain advertisement frame 402 is constant. For example, when the viewer displays the angle of view B in the advertisement frame 402, the display target object 51 and the subject (building) overlap, and the visibility of the display target object 51 decreases. Generally, when the color of the omnidirectional image is close to the color of the display object 51, the visibility is lowered. For example, a reduction in visibility can occur when the color of the display object 51 is similar to the sky even at the angle of view A. On the other hand, even when the angle of view is B, if the color of the subject (building) and the display object 51 are significantly different, the visibility is unlikely to deteriorate.

  When a planar image is displayed as an advertisement, the advertiser may determine the position of the display object 51 and display the display object 51 in a color different from the color of the planar image at that position. However, in the case of the omnidirectional image, the position of the display object 51 in the advertisement frame 402 is constant, but the angle of view displayed in the advertisement frame 402 is arbitrarily changed. A situation may occur in which the color and the color of the omnidirectional image at the position of the display object 51 are similar.

Therefore, in the present embodiment, a browsing system 100 that performs a process of suppressing a reduction in the visibility of the display object 51 will be described. Specifically, the following processing is performed.
1. Changing the color of the display object 51 according to the color of the omnidirectional image at the position of the display object 51;
2. 2. Display of background object The display position of the display object 51 is changed.

[1. (Change the color of the display object)
<Outline of operation>
FIG. 24 is an example of a diagram for explaining the outline of the display method of the display object 51 of the present embodiment.
(1) As shown in FIG. 24A, the advertisement display unit 153 calculates the color of the omnidirectional image at the position 601 of the display object 51.
(2) As shown in FIG. 24B, the current color of the display object 51 is calculated.
(3) As shown in FIG. 24C, when the color difference between the color of the omnidirectional image and the color of the display object 51 is smaller than the threshold value, the color of the display object 51 is changed to a color whose color difference is equal to or greater than the threshold value. To do.

  By doing so, since the color difference between the color of the omnidirectional image and the color of the display object 51 is always equal to or greater than the threshold value, it is possible to suppress the visibility of the display object 51 from being lowered.

<About functions>
FIG. 25 is an example of a functional block diagram illustrating the functions of the information processing apparatus 10 according to the present embodiment in a block shape. In the description of FIG. 25, differences from FIG. 10 are mainly described. In the information processing apparatus 10 of FIG. 25, the advertisement display unit 153 includes a color difference calculation unit 153a, a color difference determination unit 153b, and a color determination unit 153c. Each of these functional blocks is a function or means realized by HTML, JavaScript (registered trademark), etc. included in the advertisement information.

  The color difference calculation unit 153a calculates the color difference between the color of the omnidirectional image at the position 601 of the display target object 51 (for example, the average of the circumscribed rectangular pixels of the display target object 51) and the color of the display target object 51. Since the LCD 206 displays the screen in the RGB color space, the color difference is taken as an example of the distance between the color of the omnidirectional image at the position 601 of the display object 51 and the color of the display object 51 in the RGB color space.

  The color difference determination unit 153b determines whether this color difference is smaller than a threshold value. The threshold value is experimentally determined in advance as a color difference that maintains visibility. When the color difference is smaller than the threshold value, the color difference determination unit 153b requests the color determination unit 153c to determine the color of the display object 51.

  The color determination unit 153c determines a color in which the color difference of the color of the display target 51 with respect to the color of the omnidirectional image at the position 601 of the display target 51 is greater than or equal to a threshold value in the RGB color space. For example, a color whose distance is equal to or greater than a threshold value is determined as the color of the display object 51 with respect to the color of the omnidirectional image at the position 601 of the display object 51 in the RGB color space. In this embodiment, the color is determined using the complementary color relationship in consideration of practicality, but the color difference is equal to or greater than the threshold value.

<Function of history management server>
Although there is no major change in the function of the history management server 20, information related to the display object 51 is stored in the advertisement image DB 292 of the history management server 20.

Table 6 schematically shows information stored in the advertisement image DB 292. In the description of Table 6, differences from Table 3 will be mainly described. The advertisement image DB 292 of the present embodiment has an item of display object 51. In the display object 51 item, the display object 51 is registered in association with the display position. The display position designates the position of the display object 51 using coordinates with the upper left corner of the advertisement frame 402 as the origin. The display object 51 is, for example, text (* .txt) and / or an image (* .jpg). As shown in the figure, one advertisement image may have a plurality of display objects 51. The advertisement display unit 153 of the information processing apparatus 10 displays the display object 51 at this display position. Since the color of the display object 51 is changed on the information processing apparatus 10 side, the color held by the history management server 20 as an initial value may be a predetermined color such as black. The image as the display object 51 is an icon indicating, for example, a reproduction button, an enlargement button, a reduction button, or the like. Text may be provided as an image.

<Example of color difference>
FIG. 26A is a diagram schematically showing the RGB color space. The color 501 of the omnidirectional image at the position 601 of the display object 51 and the color 502 of the display object 51 are uniquely determined in the RGB color space. Therefore, the color difference calculation unit 153a may calculate the distance L between the two colors. However, if the distance L is smaller than the threshold value, it is difficult to determine in which direction the color of the display object 51 should be changed with respect to the color 501 of the omnidirectional image at the position 601 of the display object 51.

  Therefore, it is effective that the color determination unit 153c determines the color of the display object 51 by changing to the HSV color space as shown in FIG. FIG. 26B shows a hue ring 602 in the HSV color space. In the hue ring 602, colors are arranged so as to follow red, orange, yellow, green, blue, indigo, and purple in a clockwise direction. In FIG. 26 (b), there are only 12 colors, but in actuality, the colors change continuously and form a circle of 360 degrees. The hue ring 602 is characterized in that colors different by 180 degrees have a complementary color relationship. Complementary colors are combinations of colors that make each other's colors stand out most.

  Therefore, the visibility can be maintained if the color determining unit 153c determines the color so that the color 502 of the display object 51 is complementary to the color 501 of the omnidirectional image at the position 601 of the display object 51. it is conceivable that. Further, the color of the display object 51 determined in this way has a distance equal to or greater than a threshold with respect to the color 501 of the omnidirectional image at the position 601 of the display object 51 in the RGB color space.

Therefore, the color determination unit 153c may convert the color in the RGB color space to the HSV color space, determine a color that has a complementary color relationship, and return it to the RGB color space again. Conversion between the RGB color space and the HSV color space is performed by the following equation.
Conversion from RGB to HSV S = 255 × (max−min) / max
V = max
H is obtained separately for each case.
When R is max: H = 60 × (B−G) / (max−min)
When G is max: H = 60 × {2+ (R−B)} / (max−min)}
When B is max: H = 60 × {4+ (G−R)} / (max−min)}
However, max is the maximum value of RGB, and min is the minimum value of RGB.
・ Conversion from HSV to RGB Hi = H ÷ 60 integer part (takes 0 to 5)
F = H / 60-Hi
M = V × (1−S / 255)
N = V × (1− (S / 255) × F)
K = V × {1-S / 255 × (1-F))
Hi = 0 → R = V, G = K, B = M
Hi = 1 → R = N, G = V, B = M
Hi = 2 → R = M, G = V, B = K
Hi = 3 → R = M, G = N, B = V
Hi = 4 → R = K, G = M, B = V
Hi = 5 → R = V, G = M, B = N
Note that conversion from the RGB color space to the HSV color space is an example, and the color determination unit 153c may determine a color having a color difference equal to or greater than a threshold value after conversion to the Lab color space.

<Operation procedure>
FIG. 27 is an example of a flowchart illustrating a procedure in which the information processing apparatus 10 determines and displays the color of the display object 51. This will be described with reference to FIG. 28 as appropriate.

  The advertisement acquisition unit 151 of the information processing apparatus 10 receives the advertisement information as shown in step 23 of FIG. 15 (S10). This advertisement information includes the display area determined by the history management server 20 and information (display position, text, and image) related to the display object 51 stored in the advertisement image DB 292.

  Next, the advertisement display unit 153 acquires advertisement information, and the color difference calculation unit 153a calculates the color of the omnidirectional image at the position 601 of the display object 51 (S20). As shown in FIG. 28, a certain angle of view of the omnidirectional image specified in the display area determined by the history management server 20 is displayed as a planar image in the advertisement frame 402 by perspective projection conversion. The position of the display object 51 is a two-dimensional coordinate. For example, the coordinates of the upper left corner of the circumscribed rectangle of the display object 51 are designated. The character size is specified in advance. Therefore, the position 601 of the display object 51 (the range occupied by the display object 51) becomes clear. Since it is the color of the background part of the display target object 51 that affects the visibility of the display target object 51, for example, the average value of the color of the entire circumscribed rectangle of the display target object 51 is calculated.

  The color determination unit 153c determines a color whose distance from the color calculated in step S20 is equal to or greater than a threshold as the color of the display object 51 (S30). As described above, the color (RGB) of the omnidirectional image at the position of the display object 51 is converted into the HSV color space to determine a complementary color, and the complementary color is changed to a color in the RGB color space.

  The advertisement display unit 153 displays the display object 51 so as to overlap the front side of the advertisement image (S40).

  In the process of FIG. 27, the color of the display object 51 is always determined when the information processing apparatus 10 receives the advertisement image. However, the initial color of the display object 51 and the position of the display object 51 are determined. The color difference between the colors of the celestial sphere image 601 may be calculated, and the display object 51 may be displayed with the initial color if the color difference is equal to or greater than a threshold value.

  FIG. 29 is an example of a flowchart illustrating a procedure in which the information processing apparatus 10 determines and displays the color of the display object 51 when the position 601 of the display object 51 with respect to the omnidirectional image changes.

  Since the viewer can rotate, enlarge, or reduce the omnidirectional image of the advertisement frame 402, the position of the display object 51 with respect to the omnidirectional image can change. For this reason, the advertisement display unit 153 acquires the presence / absence of an operation from the operation reception unit 14 and determines whether or not the omnidirectional image has been operated (S10).

  When the determination in step S10 is Yes, the advertisement display unit 153 displays the omnidirectional image at the angle of view operated by the viewer (S20).

  Next, the advertisement display unit 153 determines whether or not a certain time has elapsed (S30). In some cases, the omnidirectional image is automatically rotated (the advertisement display unit 153 rotates), and this predetermined time is a predetermined time that has elapsed since step S30 was last determined.

  Subsequent to step S20 or when the determination in step S30 is Yes, the position 601 of the display object 51 with respect to the omnidirectional image has changed, so the color difference calculation unit 153a has the omnidirectional position of the display object 51 at the position 601. The color of the sphere image is calculated (S40). The calculation method may be the same as in FIG.

  Then, the color difference determination unit 153b determines whether or not the color difference between the color of the omnidirectional image at the position 601 of the display object 51 and the color of the display object 51 is smaller than the threshold (S50).

  When the determination in step S50 is No, it is determined that the visibility is not lowered, so the color of the display object 51 is not changed.

  When the determination in step S50 is Yes, the color determination unit 153c determines a color whose distance from the color of the display target object 51 is equal to or greater than a threshold as the color of the display target object 51 (S60). The determination method may be the same as in FIG.

  The advertisement display unit 153 displays the display object 51 so as to overlap the front side of the advertisement image (S70).

  The advertisement display unit 153 can always ensure the visibility of the display object 51 by repeating the processing of FIG.

[2. (Display of background object)
Instead of changing the color of the display object 51, a background object may be superimposed on the display object 51. This will be described with reference to FIG.

  FIG. 30 is an example of a diagram for explaining the background object 52 displayed so as to be superimposed on the display object 51. In FIG. 30A, since the color of the position 601 of the display target object 51 is similar to the color of the display target object 51, the visibility of the display target object 51 is lowered. In FIG. 30B, a background object 52 is displayed over the display object 51. Since the color of the background object 52 is different from the color of the display object 51 (for example, complementary color), the visibility is unlikely to deteriorate.

  The background object 52 is a uniform color area that is equal to or larger than the display object 51. For example, a circumscribed rectangle or a rectangular area slightly larger than that. It may be the same as the range in which the color of the display object 51 in the omnidirectional image is calculated. A circle, an ellipse, a polygon, a balloon, an indeterminate shape, or the like may be used as long as it is equal to or larger than the display object 51. Further, the background object 52 may overlap only a part of the display object 51. Moreover, the color of the background object 52 should just be a different color so that the viewer can visually recognize the display target object 51.

  In addition, it is effective that the background object 52 has transparency set. For example, the transparency is specified as 0 to 100%. The transparency of 0% is a normal image that does not transmit the back image, and the transparency of 100% is an image that completely transmits the back image. The transmittance value may be appropriately determined between 10% and 90%, for example. The background object 52 in which the transparency is set improves the visibility of the display object 51 and enables browsing of the omnidirectional image that overlaps the background object 52.

  FIG. 31A is an example of a flowchart illustrating a procedure in which the information processing apparatus 10 displays a background object. In the description of FIG. 31A, differences from FIG. 27 will be described.

  The advertisement acquisition unit 151 of the information processing apparatus 10 receives the advertisement information as shown in step 23 of FIG. 15 (S10).

  Next, the advertisement display unit 153 acquires advertisement information, and the color difference calculation unit 153a acquires the color of the display object 51 (S20). The color difference calculation unit 153a acquires an initial color set as an attribute of the display object 51.

  The color determining unit 153c determines a color whose distance from the color of the display target object 51 is equal to or greater than a threshold as the color of the background object (S30). That is, the color (RGB) of the display object 51 is converted into the HSV color space to determine a complementary color, and this complementary color is changed to a color in the RGB color space.

  The advertisement display unit 153 displays the background object 52 and the display object 51 so as to overlap the advertisement image (S40).

  In the process of FIG. 31A, the color of the background object 52 is determined. However, the information processing apparatus 10 previously selects the color of the background object 52 that has a complementary color relationship with the color of the display object 51 of the initial value. It may be determined.

  In FIG. 31A, the background object 52 is always displayed when the information processing apparatus 10 receives the advertisement image, but the color of the position 601 of the display target 51 in the omnidirectional image and the display target are displayed. The information processing apparatus 10 calculates the color difference of 51 colors, and the background object may not be displayed as long as it is equal to or greater than the threshold value.

  FIG. 31B is an example of a flowchart illustrating a procedure in which the information processing apparatus 10 determines and displays the color of the background object when the position 601 of the display object 51 with respect to the omnidirectional image changes. In the description of FIG. 31B, differences from FIG. 29 will be described. First, the processing in steps S10 to S50 is the same as that in FIG.

  When the determination in step S50 is Yes, the color determination unit 153c determines a color whose distance from the display object 51 is equal to or greater than a threshold as the color of the background object 52 (S60). The determination method may be the same as in FIG.

  If the determination in step S50 is No, it is determined that the visibility is not lowered, so the color determination unit 153c deletes the background object 52 (S62).

  The advertisement display unit 153 displays the background image 52 and the display object 51 so as to overlap the advertisement image (S70).

  In the case of the background object 52, the display object 51 can be made more conspicuous than the color of the display object 51 is changed. By setting the transparency, it is possible to browse the omnidirectional image that overlaps the background object 52.

  Even when the background object 52 is not displayed (when the color of the text is changed as shown in FIG. 28), the text may be displayed translucently. Although the visibility of the text is slightly reduced, it is possible to view the omnidirectional image that overlaps the text.

[3. Change of display position of display object 51)
The display position of the display object 51 may be changed instead of changing the color of the display object 51.
FIG. 32 is an example of a diagram for explaining the change of the display position. For example, as shown in FIGS. 32A to 32C, display positions A to C that are candidates for display of the display object 51 are determined in advance. The color difference calculation unit 153a determines the display position having the largest color difference among the display positions A to C, and the advertisement display unit 153 displays the display object 51 at the display position. By doing so, the display object 51 can be displayed at the display position where the color difference from the display object 51 is the largest.

  In FIG. 32, there are only three display positions A to C, but there may be four or more display position candidates. In addition, the display positions may overlap, and it is easier to determine a display position with a larger color difference as the number of candidates increases within a range where the processing load can be tolerated. 32 are shapes suitable for the display of the horizontally written display object 51, but display position candidates suitable for the display of the vertically written display object 51 are also prepared. A candidate for a display position to be used is instructed from 20 as appropriate.

  FIG. 33A is an example of a flowchart illustrating a procedure in which the information processing apparatus 10 displays a background object. In the description of FIG. 33A, differences from FIG. 31A will be described.

  Steps S10 and S20 may be the same as in FIG.

  Next, the color difference calculation unit 153a calculates the average value of the colors of the omnidirectional images at the display positions for every display position candidate (S30).

  The color determination unit 153c determines a display position candidate having the largest color difference from the color of the display object 51 (S40).

  The advertisement display unit 153 displays the display object 51 at the determined display position (S50).

  FIG. 33B is an example of a flowchart illustrating a procedure in which the information processing apparatus 10 determines and displays the color of the background object 52 when the position of the display object 51 with respect to the omnidirectional image changes. In the description of FIG. 33B, differences from FIG. 31B will be described. First, the processing in steps S10 to S50 is the same as that in FIG.

  When the determination in step S50 is No, the display position of the display object 51 is not changed.

  When the determination in step S50 is Yes, the color difference calculation unit 153a calculates the average value of the colors at the display positions for all display position candidates (S60).

  The color determination unit 153c determines the display position having the largest color difference with respect to the color of the display object 51 (S70).

  The advertisement display unit 153 displays the display object 51 at the determined display position (S80).

  When the display position of the display object 51 is changed, the advertisement display unit 153 may move the display object 51 by linearly interpolating the display positions before and after the change. Thereby, it can suppress that the display target object 51 moves suddenly and a viewer loses sight.

  By doing so, the display object 51 is automatically moved, so that a reduction in visibility can be suppressed, and the movement of the viewer is easy to draw.

<Example of display object>
FIG. 34 is an example for explaining an example of the display object 51. 34A shows an example of the stop button 53 as the display object 51, and FIG. 34B shows an example of the playback button 54 as the display object 51. The stop button 53 and the playback button 54 are alternately displayed according to the operation of the viewer. Such a display object 51 is transmitted from the history management server 20 as an image, but the advertisement display unit 153 can change the color. Therefore, the color can be changed so as to suppress a decrease in visibility as in the text. Note that the stop button 53 and the play button 54 are merely examples, and an enlarge button, a reduce button, a right button (a predetermined amount of rotation to the right), a left button, an up button, a down button, and a move button (all other images having different imaging positions). Button for displaying a celestial sphere image) and an automatic playback button. Such an image may be called an icon.

  Even when an image is displayed as the display object 51, the image can be displayed translucently. Accordingly, the advertisement display unit 153 changes the color of the display object 51 to a color in which the visibility is difficult to be lowered and displays it in a translucent manner, so that the omnidirectional image overlapping the image can be viewed.

  Further, as shown in FIG. 34C, a URL such as an advertiser's home page may be the display object 51. Such a URL 55 may be called a link. This URL 55 is associated (embedded or linked) with the advertisement image even if it is not clearly shown as shown in FIG. This URL 55 is also translucent and can be displayed.

  In addition, the advertisement display unit 153 may blink the display object 51. In FIG. 34D, the display object 51 “Go to site” is displayed, but in FIG. The display object 51 “go” is not displayed. The advertisement display unit 153 switches between display and non-display of the display object 51 at predetermined time intervals with the same display position. Since the viewer's attention can be drawn by blinking the display target object 51, a decrease in visibility can be suppressed. In addition, it is preferable to blink after changing the color of the display target object 51 in consideration of the color difference.

  In addition, as shown in FIG. 35A, a plurality of display objects 51 may be displayed in one advertisement frame 402. In FIG. 35A, a display object 51 “Go to site” and a display object 51 “Campaign!” Are displayed. As shown in Table 6, since the display position of the display object 51 is determined for each display object 51, the advertisement display unit 153 can be changed to a color in which the visibility is difficult to decrease for each display object 51.

  As shown in FIG. 35B, when a plurality of display objects 51 are displayed, the advertisement display unit 153 mixes the display object 51 that changes the color of the text and the display object 51 that overlaps the background object 52. Good. For example, whether the color is changed or the background object is overlaid is determined in advance according to the type of information. Alternatively, the advertisement display unit 153 may switch every predetermined time. Alternatively, the advertisement display unit 153 may determine the content of the display target 51 by analyzing whether the text includes a predetermined word or whether the display target 51 is a text or an image.

  Further, the advertisement display unit 153 may change the color of the text display object 51 made up of a plurality of symbols (letters or numbers) for each symbol. In this case, the color of the range corresponding to the circumscribed rectangle of each symbol in the omnidirectional image is calculated, and the color whose distance is equal to or greater than the threshold is determined for each symbol. Further, the advertisement display unit 153 may display a background object having a color determined for each symbol. The color of the omnidirectional image at the position of the display object 51 is more similar to the overall color of the omnidirectional image as the area occupied by the display object 51 is larger, and it may be difficult to determine a color having a large color difference. On the other hand, when changing the color for each symbol, it is easy to determine a color whose visibility is less likely to deteriorate with respect to the color of the local omnidirectional image where the symbols overlap. Of course, the color may be determined for each of a plurality of characters such as every two characters or three characters.

<Summary>
As described above, the viewing system 100 of the present embodiment changes the color of the display object 51 or the background object 52 so that there is a color difference equal to or greater than the threshold with respect to the color of the display object 51 in the omnidirectional image. Even in a situation where the relative position of the display object 51 with respect to the omnidirectional image is changed, a reduction in the visibility of the display object 51 can be suppressed.

  In the present embodiment, it has been described that the omnidirectional image of the display area determined by the history management server 20 is displayed in the advertisement frame 402. However, the color change of the display object 51 of the present embodiment can be changed to an arbitrary display area. Is applicable. Therefore, the present invention is not limited to retargeting advertisements and can be applied to omnidirectional images of arbitrary Web servers.

<Other application examples>
The best mode for carrying out the present invention has been described above with reference to the embodiments. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the scope of the present invention. And substitutions can be added.

  For example, in this embodiment, the information processing apparatus 10 acquires a web page from the advertiser website 30 and the partner site 50 and displays an advertisement. However, the omnidirectional image is displayed not on the web page but on general application software. It may be applied to the case.

  Further, in this embodiment, the example in which the omnidirectional image is used for the advertisement has been described, but the omnidirectional image need not be used for the advertisement.

  Further, in this embodiment, an example in which the omnidirectional image is used for the preview of the real estate has been described, but what kind of content is reflected in the omnidirectional image or what kind of advertisement is used is limited. There is no. For example, it can be used in many fields, such as PR of tourist spots, department stores, inns, explanations in cars, road guidance, explanation of machine structure, explanation of places where people are difficult to enter.

  In this embodiment, the omnidirectional image displayed on the advertiser website is a still image, but the omnidirectional image may be a moving image. The information processing apparatus 10 records the browsing history of the reproduced moving image together with the reproduction time. When the partner site 50 displays the omnidirectional image, the display area is determined for each reproduction time. Thereby, the information processing apparatus 10 can reproduce the angle of view that the viewer was viewing at the time of advertisement.

  Further, in the present embodiment, it has been described that the omnidirectional image capable of capturing 360 degrees around the omnidirectional camera is arranged on the web page, but the image data of the web page may be a wide-angle image. The wide-angle image of the present embodiment refers to an image having a portion that cannot be displayed in the advertisement frame. For example, a panoramic image having an angle of view of about 180 degrees only in the horizontal direction may be used.

  In addition, the omnidirectional image is not limited to the case where the two images are created by a method in which the two images are pasted together. . That is, the method for creating the omnidirectional image is not limited.

  Further, the configuration example of FIG. 10 and the like has main functions in order to facilitate understanding of processing by the information processing apparatus 10, the advertiser website 30, the partner site 50, the advertisement distribution server 40, and the history management server 20. It is divided accordingly. The present invention is not limited by the way of dividing the processing unit or the name. The processing of the information processing apparatus 10, the advertiser website 30, the partner site 50, the advertisement distribution server 40, and the history management server 20 can be divided into more processing units according to the processing contents. Moreover, it can also divide | segment so that one process unit may contain many processes.

  Moreover, the advertisement delivery server 40 and the history management server 20 exist as separate servers, and the functions may be integrated as one server. Further, the function of the advertisement distribution server 40 may be distributed to a plurality of servers, or the function of the history management server 20 may be distributed to a plurality of servers.

  The history management server 20 may be called a DSP (Demand-Side Platform). In the DSP, the advertiser side is the name of the advertising platform, and the bid amount is determined in consideration of the attributes of the viewer, the type of medium, the time zone, etc., and the bid is made on the SSP (Supply-Side Platform). In response to the notification that the advertising space is displayed from the information processing apparatus 10, the SSP notifies the DSP of the attributes of the viewer, the type of medium, and the like. The DSP and the SSP operate in pairs to determine the omnidirectional image (successful bidder) to be displayed in the advertising space.

  Further, since the history management server 20 transmits an omnidirectional image (wide-angle image) to the information processing apparatus 10, it may be a server that distributes an image (advertisement).

  The advertiser website 30 is an example of a first screen information providing device, the partner site 50 is an example of a second screen information providing device, and the history management server 20 is an example of an image distribution device. The operation reception unit 14 is an example of an operation reception unit, the operation history transmission unit 155 is an example of an operation history transmission unit, the advertisement acquisition unit 151 is an example of a wide-angle image acquisition unit, and the advertisement display unit 153 is a display unit. The operation history DB 291 is an example of an operation history storage unit, the advertisement image DB 292 is an example of a wide-angle image storage unit, the display area determination unit 22 is an example of a display area determination unit, and the advertisement image communication unit 21 is an example. Is an example of an image transmission unit, and the cookie acquisition unit 152 is an example of an identification information acquisition unit. The advertisement request tag is an example of image request information, the advertisement distribution server 40 is an example of an external device, and the color difference determination unit 153b is an example of a color difference determination unit.

8 Browser 9 Spherical Camera 10 Information Processing Device 20 History Management Server 30 Advertiser Website 40 Advertising Distribution Server 50 Partner Site 100 Browsing System

JP 2008-19211 A

Claims (20)

A browsing system having an information processing device and an image distribution device,
The information processing apparatus includes:
Operation accepting means for accepting an operation on the image of the screen information acquired from the first screen information providing device;
An operation history transmitting means for transmitting the operation history for the image received by the operation receiving means, the identification information of the image, and the identification information of the information processing apparatus to the image distribution apparatus;
When acquiring screen information from the second screen information providing device, the identification information of the information processing device is transmitted to the image distribution device, and the wide angle image determined based on the identification information of the information processing device and the wide angle Wide-angle image acquisition means for acquiring a display area of the image;
Display means for displaying the display area of the wide-angle image acquired by the wide-angle image acquisition means on a display device based on the screen information acquired from the second screen information providing device;
The image distribution device includes:
Operation history storage means for storing the operation history in association with the identification information of the image and the identification information of the information processing apparatus;
Referring to wide-angle image storage means for storing the wide-angle image in association with the identification information of the image,
The wide-angle image is determined based on the identification information of the information processing apparatus acquired from the information processing apparatus, and the display area of the wide-angle image is determined based on the operation history associated with the identification information of the information processing apparatus. Display area determining means;
Image transmitting means for transmitting the wide-angle image and the display area to the information processing apparatus;
A browsing system. The image of the screen information acquired from the first screen information providing device is the wide-angle image;
The operation history transmission means transmits an operation history for the wide-angle image, identification information of the wide-angle image, and identification information of the information processing device to the image distribution device,
In the operation history storage means, the operation history is stored in association with the identification information of the wide-angle image and the identification information of the information processing device,
The wide angle image storage means stores the wide angle image in association with the identification information of the wide angle image,
The display area determining means specifies the operation history and the wide-angle image identification information associated with the identification information of the information processing apparatus acquired from the information processing apparatus from the operation history storage means,
The browsing system according to claim 1, wherein the display area of the wide-angle image of the wide-angle image storage unit associated with the identified identification information of the wide-angle image is determined based on the operation history. The display area determining means specifies the operation history and the wide-angle image identification information associated with the identification information of the information processing apparatus acquired from the information processing apparatus from the operation history storage means,
Detecting the wide-angle image similar to the wide-angle image associated with the identification information of the information processing device from the wide-angle image storage means;
Determining the display area of a wide-angle image similar to the image based on the operation history;
The browsing system according to claim 2, wherein the image transmission unit transmits the similar wide-angle image and the display area to the information processing apparatus. The wide-angle image storage means stores the angle of view of the wide-angle image and the object reflected in the angle of view in association with the identification information of the wide-angle image,
The display area determining means acquires from the wide-angle image storage means the object shown in the angle of view of the wide-angle image corresponding to the display area determined based on the operation history,
The angle of view in which the object is reflected in the wide-angle image similar to the wide-angle image associated with the identification information of the information processing device is specified from the wide-angle image storage unit, and the angle of view is determined to be the similar wide-angle. The browsing system according to claim 3, wherein the viewing area is determined as the display area of the image. In the operation history storage means, an operation history for a flat image, identification information of the flat image, and identification information of the information processing device are stored in association with each other,
The wide-angle image storage means stores information indicating the subject in the plane image in association with the identification information of the plane image.
The display area determining unit specifies the operation history and the planar image identification information associated with the identification information of the information processing device acquired from the information processing device from the operation history storage unit,
The browsing system according to any one of claims 1 to 4, wherein the display area of the wide-angle image is determined based on information indicating a subject of the planar image associated with identification information of the information processing apparatus. The operation history storage means stores a viewing angle of view viewed in the wide-angle image and a browsing time of the viewing angle.
The browsing system according to any one of claims 2 to 4, wherein the display area determining means determines the viewing angle of view with the longest browsing time as the display area of the wide-angle image. The operation history storage means further stores the presence or absence of an enlargement or reduction operation for each viewing angle of view,
The display area determining unit converts the browsing time, the enlargement, and the reduction operations into points, and determines the viewing angle determined according to the points as the display area of the wide-angle image. 6. The browsing system according to 6. The information processing apparatus includes:
When acquiring screen information from the first screen information providing device, acquire a script that communicates with an external device and the image distribution device;
Executing the script, requesting identification information of the information processing apparatus from the external apparatus, and having identification information acquisition means for acquiring the identification information of the information processing apparatus from the external apparatus,
The said script is executed and the said operation history transmission means transmits the said operation history, the identification information of the said image, and the identification information of the said information processing apparatus to the said image delivery apparatus in any one of Claims 1-7. The browsing system described. The information processing apparatus includes:
When acquiring the screen information from the second screen information providing device, obtain a script to communicate with the external device,
The wide-angle image acquisition means executes the script, transmits identification information of the information processing device to the external device, and requests image information from the external device to the image distribution device for the wide-angle image. Get
The browsing system according to claim 8, wherein the wide-angle image acquisition unit acquires the wide-angle image and the display area by transmitting identification information of the information processing apparatus to the image distribution apparatus using the image request information.   The browsing system according to claim 1, wherein the wide-angle image is an omnidirectional image. The wide-angle image acquisition means acquires the wide-angle image and the display area, acquires a display object that is information related to the wide-angle image and is displayed on the wide-angle image,
The display means performs a process of suppressing a reduction in visibility of the display object when the display object is displayed on the wide-angle image.
The browsing system of any one of Claims 1-10.   The display means changes a color of the display object as a process for suppressing a decrease in visibility of the display object, or a background object having the same width or more as the display object is displayed on the display object. The browsing system according to claim 11, wherein the browsing system displays the images in a superimposed manner. Color difference determining means for determining whether a color difference between the color of the wide-angle image at the position of the display object in the wide-angle image and the color of the display object is smaller than a threshold value;
When the color difference determining means determines that the color difference between the color of the wide-angle image at the position of the display object in the wide-angle image and the color of the display object is smaller than a threshold value,
The browsing system according to claim 12, wherein the display unit changes a color of the display object as a process of suppressing a decrease in visibility of the display object. Color difference determining means for determining whether a color difference between the color of the wide-angle image at the position of the display object in the wide-angle image and the color of the display object is smaller than a threshold value;
When the color difference determining means determines that the color difference between the color of the wide-angle image at the position of the display object in the wide-angle image and the color of the display object is smaller than a threshold value,
The display means displays a background object having the same or larger area as the display object as a process for suppressing a decrease in the visibility of the display object,
When the color difference determining means determines that the color difference between the color of the wide-angle image at the position of the display object in the wide-angle image and the color of the display object is greater than or equal to a threshold value,
The browsing system according to claim 12, wherein the display unit erases the background object. The display means determines the color of the display object so that a color difference between the color of the wide-angle image and the color of the display object at the position of the display object is equal to or greater than a threshold; or
The browsing system according to any one of claims 12 to 14, wherein a color of the background object in which a color difference with respect to a color of the display object is equal to or greater than a threshold value is determined.   The display unit calculates a color difference between a plurality of predetermined positions of the display object and a color of the display object as a process for suppressing a decrease in visibility of the display object, The browsing system according to claim 11, wherein the display object is displayed at a position of the display object having a large size.   The browsing system according to any one of claims 12 to 15, wherein the display means displays the display object or the background object in a translucent manner.   The browsing system according to claim 11, wherein the display object is a text or an image. Obtain screen information from the first screen information providing device and the second screen information providing device,
Operation accepting means for accepting an operation on the image of the screen information acquired from the first screen information providing device;
An operation history transmitting means for transmitting the operation history for the image received by the operation receiving means, the identification information of the image, and the identification information of the information processing apparatus to an image distribution apparatus;
When acquiring screen information from the second screen information providing device, the identification information of the information processing device is transmitted to the image distribution device, and the wide-angle image determined based on the identification information of the information processing device and the information Wide-angle image acquisition means for acquiring a display area of the wide-angle image;
An operation history of an information processing apparatus comprising: display means for displaying the display area of the wide-angle image acquired by the wide-angle image acquisition means on a display device based on screen information acquired from the second screen information providing device Manage the image distribution device
Operation history storage means for storing the operation history in association with the identification information of the image and the identification information of the information processing apparatus;
Referring to wide-angle image storage means for storing the wide-angle image in association with the identification information of the image,
The wide-angle image is determined based on the identification information of the information processing apparatus acquired from the information processing apparatus, and the display area of the wide-angle image is determined based on the operation history associated with the identification information of the information processing apparatus. Display area determining means;
Image transmission means for transmitting the wide-angle image and the display area to the information processing apparatus;
Program to function as. A browsing system having an information processing device and a server,
The information processing apparatus includes:
Accepting means for accepting an operation on an image included in screen information acquired from outside via a network;
A transmission unit that transmits a history of operations on the image received by the reception unit and identification information of the information processing apparatus to the server;
When other screen information different from the screen information is acquired, identification information of the information processing device is transmitted to the server based on information included in the other screen information, and the screen information different from the screen information is transmitted from the server. An acquisition means for acquiring a wide-angle image and a display area of the wide-angle image included in
Display means for displaying, on a display device, the display area of the acquired wide-angle image based on the other screen information;
The server
Storage means for storing the identification information of the image, the history of the operation on the image, and the identification information of the information processing apparatus in association with each other;
The wide-angle image based on the identification information of the image stored in association with the identification information of the information processing device and the history of the operation according to the identification information of the information processing device transmitted from the information processing device Determining means for determining a display area of the wide-angle image;
Transmitting means for transmitting the wide-angle image and the display area of the wide-angle image to the information processing apparatus;
The browsing system characterized by having.