JP2022008888A - Server, terminal, distribution system, distribution method, and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide contents based on positional information.

SOLUTION: Provided is a contents distribution system for distributing contents, comprising a server 1 for distributing the contents and a user terminal 3 for viewing the contents. The server 1 includes: an accumulation section 15 for associating and retaining the content to a pair of a marker and positional information; a reception section 11 for receiving marker information and the positional information; a movie obtainment section 13 for searching the pair of the marker indicated by the marker information and the positional information from the accumulation section 15, and obtaining the content associated with the pair of the marker and the positional information; and a transmission section 14 for transmitting the content associated with the pair of the marker and the positional information. The user terminal 3 includes: a photographing section 31 for photographing images; a positional detection section 32 for obtaining the positional information of the user terminal 3; a transmission section 33 for transmitting the marker information and the positional information that are obtainable from the images; a reception section 35 for receiving the content associated with the pair of the marker indicated by the marker information and the positional information; and a display section 36 for displaying the content.

SELECTED DRAWING: Figure 5

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a server, a terminal, a distribution system, a distribution method, and an information processing method.

Due to the high performance of mobile terminals, applications that use location information and applications that use image recognition are becoming widespread.

In the technology of Patent Document 1, the server transmits virtual display information corresponding to the real space captured by the mobile terminal based on the position of the mobile terminal, and the mobile terminal superimposes the virtual display information on the captured real space. To display.

In the technique of Patent Document 2, the server identifies a product from the result of image recognition processing for an image from a terminal, generates a display object representing a person who has taken a predetermined action on the product, and transmits the display object to the terminal. , The terminal superimposes a display object on the image and displays it.

Japanese Unexamined Patent Publication No. 2011-242816 International Publication No. 2012/096013

In Patent Document 1, since the virtual display information is managed by associating it only with the position, there is a problem that various information is mixed and the AR space becomes chaotic.

Patent Document 2 can detect a product by image recognition processing and superimpose and display information about the product on the image, but does not use the position information at which the product is photographed.

The present invention has been made in view of the above, and an object of the present invention is to provide content based on location information.

The distribution system of one aspect of the present invention is a distribution system for distributing content, which includes a server for distributing the content and a terminal for viewing the content, and the server distributes the content in a set of a marker and a position information. A storage unit that is associated and held, a reception unit that receives marker information and position information, and a set of markers and position information indicated by the marker information are searched from the storage unit and associated with the marker and position information set. The terminal has an acquisition unit for acquiring content and a transmission unit for transmitting content associated with the marker and the set of position information, and the terminal has a photographing unit for capturing an image and a position for acquiring the position information of the terminal. The detection unit, the transmission unit that transmits the marker information obtained from the image and the position information, the reception unit that receives the content associated with the marker indicated by the marker information and the position information set, and the content are displayed. It has a display unit to be used.

According to the present invention, it is possible to provide content based on location information.

It is a figure which shows an example of the structure of the content distribution system of this embodiment. It is a figure which shows an example of the image which a marker appears. It is a figure which shows an example of the display screen which superposed the content on the marker. It is a figure which shows an example of the map which shows the position where a marker is registered. It is a functional block diagram which shows an example of a server configuration. It is a functional block diagram which shows an example of the structure of a user terminal. It is a flowchart which shows the process flow of a server. It is a flowchart which shows the processing flow of a user terminal. It is a figure which shows the example which used the landmark as a marker. It is a flowchart which shows the flow of the process which a user registers a set of a marker and a position information.

[Content distribution system]
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

As shown in FIG. 1, the content distribution system of the present embodiment includes a server 1 and a user terminal 3. The server 1 and the user terminal 3 are connected via a network. Although only one server 1 is shown in FIG. 1, a plurality of servers may be provided for each function, or the functions of the server 1 may be realized on the cloud. The number of user terminals 3 is not limited to two, and a large number of user terminals 3 connect to the server 1.

The user terminal 3 transmits the marker information obtained by taking a picture with the camera included in the user terminal 3 and the position information of the user terminal 3 to the server 1. FIG. 2 shows an example of an image 200 showing the marker 210 taken by the user terminal 3. The marker 210 attached to the door is shown in the image 200 shown in the figure. The user terminal 3 transmits the captured image 200 and the position information of the capture position to the server 1. The marker 210 in FIG. 2 is an example, and a subject that can be identified by image recognition can be used as a marker. For example, a figure, poster, person, monument, or landmark can be used as a marker.

The server 1 returns the content associated with the set of the marker and the position information to the user terminal 3. For example, if the server 1 recognizes the marker 210 from the received image 200 by image recognition and a set of the recognized marker 210 and the received position information is registered, the server 1 has the content associated with the set. Is returned to the user terminal 3. When a comment is added to the content, the server 1 returns the comment together with the content to the user terminal 3.

The user terminal 3 displays the content received from the server 1. FIG. 3 shows an example of a display screen 300 displaying a moving image 310 which is a content in the user terminal 3. In the display screen 300 shown in the figure, the moving image 310 received from the server 1 is superimposed and displayed in the marker of the image taken by the user terminal 3 by using the AR technology. The mode for displaying the moving image 310 is not limited to this. In front of the photographed marker, the moving image 310 may be superimposed and displayed in a size larger than that of the marker. The display mode of the moving image 310 may be changed according to the shape of the structure to which the marker is attached. The moving image 310 may be displayed on the entire surface of the display screen 300 of the user terminal 3 without superimposing the moving image 310 on the captured image.

In the present embodiment, a moving image will be used as an example of the content, but the present embodiment is not limited to this. For example, the content may be a still image or text, or may be music or audio. The content may be a three-dimensional object by computer graphics. Further, the content may be a composite of these. Multiple types of content may be associated with a set of markers and location information. Further, a plurality of or a plurality of types of contents may be associated with the set of the marker and the position information. When a plurality of contents are associated with a set of a marker and a position information, the plurality of contents may be delivered in parallel or may be delivered in a predetermined order. A predetermined number of contents selected from a plurality of contents may be delivered.

When the user terminal 3 receives the comment given to the content, the user terminal 3 also displays the comment. A comment is a message posted in the past at that location. That is, the user terminal 3 displays a comment posted by a user who has taken the same marker 210 in the past and watched the moving image 310. The comment is displayed so as to appear from one end of the moving image, move toward the other end, and disappear at the other end, for example, in the moving image 310 displayed on the user terminal 3. The user terminal 3 may display a comment at an arbitrary position in the image with respect to the marker by using AR technology. For example, when a marker attached to a utility pole is photographed, the comment is displayed so as to rotate around the utility pole.

The user can post a comment to the content received from the server 1. When the user inputs a comment, the user terminal 3 transmits the input comment to the server 1. The server 1 stores a comment for the content in association with the set of the marker and the position information.

The user terminal 3 may display a map showing the position where the marker is present as shown in FIG. In the example of FIG. 4, it is shown that the markers are present at the points A and B. When the user shoots the marker at point A, he / she can watch the video associated with the pair of marker and point A, and when he / she shoots the marker at point B, he / she can watch another video associated with the pair of marker and point B. You can watch it. The markers at points A and B may be the same figure.

The same moving image may be associated with each of the set of the marker and the point A and the set of the marker and the point B. Even if the moving images that can be viewed at each of the points A and B are the same, the server 1 manages the comments input at each of the points A and B separately. The user can see the comment entered at point A when shooting the marker at point A and watching the video, and the comment entered at point B when shooting the marker at point B and watching the video. You can see it.

The posted comment itself posted by the user may be used as the content. When the server 1 receives the marker and the position information from the user terminal 3, the server 1 returns the comment stored in association with the set of the marker and the position information to the user terminal 3. The user can view the comment displayed superimposed on the captured marker image. When the user posts a comment, the server 1 associates the received comment with a marker and a set of location information and stores it.

[Server and user terminal configuration]
Next, the configuration of the server 1 and the user terminal 3 of the present embodiment will be described.

The server 1 shown in FIG. 5 includes a receiving unit 11, an image recognition unit 12, a moving image acquisition unit 13, a transmitting unit 14, and a storage unit 15.

The receiving unit 11 receives the marker information, the position information, and the comment from the user terminal 3. The marker information is, for example, an image obtained by taking a marker. The position information is information indicating the position of the user terminal 3, and is, for example, the position information obtained by the GPS (Global Positioning System) function provided in the user terminal 3. The location information can be of arbitrary accuracy depending on the application to be used. For example, location information based on the base station to which the user terminal 3 is connected may be used. The receiving unit 11 passes the received marker information and position information to the image recognition unit 12.

The receiving unit 11 records the received comment in the storage unit 15 in association with the moving image or the marker and the set of the position information being viewed by the user.

The image recognition unit 12 recognizes the subject in the marker information (image received from the user terminal). If the recognized subject is registered as a marker, the image recognition unit 12 passes the set of the recognized marker and the position information received by the reception unit 11 to the moving image acquisition unit 13. The business operator registers a figure, a poster, a person, a monument, a landmark, or the like as a marker in the server 1 in advance. A plurality of types of markers may be registered in the server 1. For example, an ID is assigned to each marker, and the image recognition unit 12 outputs the ID of the recognized marker.

The moving image acquisition unit 13 determines whether or not a set of the marker and the position information is registered. If the set of the marker and the position information is registered, the moving image acquisition unit 13 acquires the moving image associated with the set of the marker and the position information from the storage unit 15 and passes it to the transmission unit 14. For example, the business operator registers a set of a marker and a position information in the storage unit 15, and stores a moving image in the storage unit 15 in association with the set.

The transmission unit 14 distributes the moving image acquired from the storage unit 15 to the user terminal 3. Further, the transmission unit 14 acquires the comment recorded in association with the moving image or the set of the marker and the position information from the storage unit 15 and distributes it to the user terminal 3.

The storage unit 15 stores information necessary for processing of the server 1. For example, the storage unit 15 holds information for the image recognition unit 12 to detect a marker by image recognition. Further, the storage unit 15 holds a set of the marker and the position information, and holds a moving image associated with the set. You may keep the link to the video instead of the video itself. The storage unit 15 manages the posted comments in association with a moving image or a set of markers and location information.

The user terminal 3 shown in FIG. 6 includes a photographing unit 31, a position detection unit 32, a transmitting unit 33, an input unit 34, a receiving unit 35, and a display unit 36. As the user terminal 3, a mobile terminal such as a smartphone can be used.

The photographing unit 31 photographs the marker. The shooting unit 31 can use, for example, a camera of a mobile terminal.

The position detection unit 32 detects the position of the user terminal 3. The position detection unit 32 can use, for example, the GPS function of the mobile terminal.

The transmission unit 33 transmits the image captured by the imaging unit 31 and the position information detected by the position detection unit 32 to the server 1. Further, the transmission unit 33 transmits the comment input by the user to the server 1.

The input unit 34 inputs a comment from the user.

The receiving unit 35 receives the moving image and the comment from the server 1.

The display unit 36 displays the moving image and the comment received from the server 1.

Each part included in the server 1 and the user terminal 3 may be configured by a computer provided with an arithmetic processing unit, a storage device, and the like, and the processing of each part may be executed by a program. This program is stored in a storage device included in the server 1 and the user terminal 3, and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or can be provided through a network.

In the above, the server 1 performs the image recognition process to detect the marker from the image, but the user terminal 3 may perform the image recognition process to detect the marker. In this case, the user terminal 3 transmits the ID of the detected marker together with the position information. Further, the user terminal 3 may transmit a processed image as marker information. For example, the user terminal 3 may transmit a marker information obtained by cutting out a portion of the image in which the marker is present or reducing the amount of information in the image.

In addition, a plurality of markers may be attached to the place indicated by one position information, or a marker may be attached to each of a plurality of adjacent position information. For example, when a plurality of markers can be detected in the image taken by the user terminal 3, the user terminal 3 accepts the user to select a marker for displaying the content, and receives the distribution of the content associated with the selected marker. It may be displayed, or it may be displayed after receiving the delivery of all the contents associated with each of the plurality of markers. Alternatively, the server 1 or the user terminal 3 may select a marker that the user has browsed in the past, or may select a marker based on the user's attribute or history information.

[Operation of server and user terminal]
Next, the operations of the server 1 and the user terminal 3 of the present embodiment will be described.

The operation of the server 1 will be described with reference to the flowchart of FIG.

In step S11, the server 1 receives the image and the position information.

In step S12, the server 1 performs the recognition processing of the received image.

In step S13, the server 1 determines whether or not the set of the marker and the position information obtained by the image recognition is registered. If the set of marker and position information is not registered, the process ends. Also, if the marker cannot be obtained by image recognition, the process is terminated. At the end of the process, the server 1 may reply that there is no marker or location information registered.

In step S14, the server 1 transmits a moving image and a comment associated with the marker and the set of location information.

When the server 1 receives a comment for the transmitted moving image in step S15, the server 1 stores the received comment in association with the moving image in step S16.

The operation of the user terminal 3 will be described with reference to the flowchart of FIG.

In step S21, the user terminal 3 transmits an image and position information.

In step S22, the user terminal 3 receives the moving image and the comment. If the image transmitted in step S21 does not contain a marker, or if the marker and position information detected from the image are not registered, the user terminal 3 receives a notification to that effect and ends the process. ..

In step S23, the user terminal 3 displays a moving image.

When the user terminal 3 inputs a comment from the user in step S24, the user terminal 3 transmits the input comment in step S25.

[Usage example]
Next, an example of using the content distribution system of the present embodiment will be described.

An example of using the content distribution system for street live distribution will be described. For example, a live broadcast program (live distribution) is prepared as content, and the live broadcast program is associated with a set of a marker and location information. Place markers in several places (or even one) in the city. A poster announcing a live program can be used as a marker. An existing subject may be registered as a marker. When the marker is photographed by the user terminal 3 and transmitted to the server 1 during the distribution of the live broadcast program, the live broadcast program is distributed to the user terminal 3 and the user can browse the live broadcast program in a street live style. Before the live distribution, the preview video may be distributed as the content, and after the live distribution, the time-shifted video recorded with the live distribution may be distributed as the content.

Users can post comments while viewing live programs. The server 1 manages comments for each set of marker and location information, that is, for each location. In addition to the method of shooting a marker and viewing a live program locally, if you make it possible to view a live program anywhere, that is, if you make it possible to view a live program at home via the Internet, the server 1 manages comments regardless of the posting location so that comments posted locally can be distinguished. The server 1 may distribute the comment posted locally to only the user terminal 3 browsing locally, or may distribute the comment posted locally to all users in a manner that can be distinguished as a comment posted locally. When a live broadcast program is associated with a plurality of markers and a set of location information, that is, when markers are installed in a plurality of places, comments are managed for each place and comments are made only to the user terminal 3 in the same place. May be delivered, or may be delivered to all user terminals 3 in a manner in which comments for each location can be distinguished.

An example of using the content distribution system for directions will be described. For example, a poster announcing the holding of an event is used as a marker. The server 1 registers the information of the route from the position to the event venue in association with the set of the poster and the position information. When the user shoots a poster affixed to the city, the image of the poster and the location information are sent to the server 1, and the route from the position where the poster was shot to the event venue is returned from the server 1 and displayed on the user terminal 3. To. For example, the user terminal 3 superimposes an arrow object on the captured image to indicate the direction of the event venue. Even if you do not change the poster for each place to put it, you can know the current position of the user from the set of poster and location information. Similarly, the hospital mark of the telephone pole advertisement may be used as a marker, and information indicating directions from the position may be associated with each of the advertisement and the location information set as the content. The user can know the directions just by taking a picture of the hospital mark. Also, if you can post comments, you can know the reputation of the hospital and post it.

An example of using the content distribution system for store introduction and reputation will be explained. Put a sticker that will be a marker at the entrance of the store, and register the store information in the set of marker and location information. The sticker may be the same for all shops. When the marker is photographed by the user terminal 3, the server 1 returns and displays the introduction of the shop corresponding to the marker and the location information. The user terminal 3 may display a comment posted by the user. The posted comments are at least the comments of the users who came to the site, so they are more reliable comments.

An example of using the content distribution system for an electronic stamp rally will be described. For example, the railway company mark is registered as a marker. A stamp rally poster may be registered as a marker. Associate the stamp of each station with the set of the marker and the location information of each station. The user can obtain the stamp of the station by taking a picture of the railway company's mark or poster at each station. For example, the information of the stamp of the station is transmitted from the server 1 to the user terminal 3, and the stamp of the station is recorded in the stamp notebook in the application on the user terminal 3.

As shown in FIG. 9, when the landmark is used as a marker, the position information may be specified in the direction with respect to the landmark or may be specified by the distance from the landmark. The server 1 manages comments at each of the positions separated by direction or distance.

[User registers marker]
In the above, the example described in which the operator registered the marker and the marker and the position information. Hereinafter, an example in which the user registers the marker and the position information will be described with reference to the flowchart of FIG.

The user shoots the marker at the position to be registered. The user terminal 3 transmits a registration request for a set of a photographed image of a marker and position information to the server 1. The user terminal 3 may transmit the content associated with the set of the marker and the position information.

In step S31, the server 1 receives the registration request.

In step S32, the server 1 detects the marker from the image by the image recognition process and registers the set of the marker and the position information. When the marker cannot be detected from the image, the server 1 may register the subject in the image as a new marker, or may return an error in which the marker cannot be detected to the user terminal 3.

In step S33, the server 1 manages the content by associating it with the set of the marker and the position information registered in step S32.

For example, when a person is registered as a marker, the user takes a picture of a signboard installed in the city, and if the signboard shows the person registered as a marker, the user registers the position and the set of people, and the set. You can post a comment to. The newly registered marker and position information set may be displayed on the map shown in FIG. After that, when another user shoots the signboard, the posted comment can be viewed and a new comment can be posted.

As described above, the content distribution system of the present embodiment includes a server 1 for distributing the content and a user terminal 3 for viewing the content. The server 1 stores the storage unit 15 that holds the content associated with the marker and the position information set, the reception unit 11 that receives the marker information and the position information, and the marker and the position information set indicated by the marker information from the storage unit 15. It has a moving image acquisition unit 13 that searches and acquires the content associated with the marker and the position information set, and a transmission unit 14 that transmits the content associated with the marker and the position information set. The user terminal 3 includes a shooting unit 31 that captures an image, a position detection unit 32 that acquires the position information of the user terminal 3, a transmission unit 33 that transmits marker information and position information obtained from the image, and marker information. It has a receiving unit 35 for receiving the content associated with the set of the marker and the position information, and a display unit 36 for displaying the content. As a result, when a predetermined marker is photographed at a predetermined position, the content can be used and various contents based on the position information can be provided.

1 ... Server 11 ... Reception unit 12 ... Image recognition unit 13 ... Video acquisition unit 14 ... Transmission unit 15 ... Storage unit 3 ... User terminal 31 ... Imaging unit 32 ... Position detection unit 33 ... Transmission unit 34 ... Input unit 35 ... Reception unit 36 ... Display unit

Claims (10)

A server that delivers content
A storage unit that associates and holds content with a set of markers and location information,
A receiver that receives marker information and location information,
An acquisition unit that searches the storage unit for a set of a marker and a position information indicated by the marker information and acquires content associated with the set of the marker and the position information.
A server having a transmitter that transmits content associated with the marker and location pair. The server according to claim 1.
The marker information is an image including a subject registered as a marker, and is a server including an image recognition unit that performs image recognition processing on the image and detects the marker. The server according to claim 1 or 2.
The storage unit holds comments posted for the content for each set of the marker and location information.
The transmission unit is a server that transmits the content and comments associated with the marker and location information set. The server according to claim 1 or 2.
The content is a comment posted by the user and is a comment posted by the user.
The transmitter is a server that transmits comments associated with the marker and location information pair. A terminal for viewing content,
The shooting department that shoots images and
A position detection unit that acquires the position information of the terminal, and
A transmission unit that transmits the marker information obtained from the image and the position information,
A receiving unit that receives the content associated with the marker indicated by the marker information and the set of the position information, and the receiver.
A terminal having a display unit for displaying the content. The terminal according to claim 5.
The display unit is a terminal that superimposes and displays the content on the image captured by the imaging unit based on the marker information obtained from the image. The terminal according to claim 5 or 6.
The content is a video
The receiver receives the comment posted for the video and receives the comment.
The display unit is a terminal that moves from one end to the other end of the moving image and displays the comment so as to disappear at the other end. It is a distribution system that distributes content.
It is equipped with a server that distributes content and a terminal for viewing the content.
The server
A storage unit that associates and holds content with a set of markers and location information,
A receiver that receives marker information and location information,
An acquisition unit that searches the storage unit for a set of a marker and a position information indicated by the marker information and acquires content associated with the set of the marker and the position information.
It has a transmitter that transmits content associated with the marker and location pair.
The terminal is
The shooting department that shoots images and
A position detection unit that acquires the position information of the terminal, and
A transmission unit that transmits the marker information obtained from the image and the position information,
A receiving unit that receives the content associated with the marker indicated by the marker information and the set of the position information, and the receiver.
A distribution system having a display unit for displaying the content. It is a delivery method executed by the server that delivers the content.
Steps to receive marker information and location information,
A step of searching the storage unit that retains the marker and position information set indicated by the marker information by associating the content with the marker and position information set, and acquiring the content associated with the marker and position information set.
A delivery method comprising a step of transmitting content associated with the marker and location pair. It is an information processing method executed by the terminal for viewing content.
The steps to take an image and
Steps to acquire the location information of the terminal and
The step of transmitting the image and the position information, and
A step of receiving content associated with a marker detected from the image and the location information set, and
An information processing method having a step of displaying the content.

Images