KR20210142258A - Electronic commerce integrated meta-media generating method, distributing system and method for the electronic commerce integrated meta-media - Google Patents

Abstract

The present invention relates to an EC-integrated meta-media production method and a transmission system control method. According to the present invention, not only a video content-related product (article or service) but also various resources invested for video content production can be sold in direct EC connection. The present invention provides a control method for a system related to a novel EC model with regard to technology integrating EC with video content (e.g., movies, TV show, animation, music video). According to the present invention, a video content viewer can directly purchase from the video content viewing screen (site) not only a product (article or service) related to the video content but also various resources (e.g., person, space, tool, equipment, technology) invested for the production of the video content.

Description

ELECTRONIC COMMERCE INTEGRATED META-MEDIA GENERATING METHOD, DISTRIBUTING SYSTEM AND METHOD FOR THE ELECTRONIC COMMERCE INTEGRATED META-MEDIA

The present invention relates to a technology for integrating EC into video content, and provides a UI (Electronic Commerce) for a video content viewer to trade production resources for the video content, including the display of the video content, as a commodity. User Interface), a method for producing an EC-integrated metamedia including a function, a transmission system, and a transmission method.

In addition, regarding the term "metamedia", for example, in the "Multimedia Internet Dictionary" (https://www.jiten.com/dicmi/docs/k34/23195.htm searched on September 10, 2019) . Media that integrates media”.

[The best background technology]

The first background art in the present invention relates to a new model (form, method, and related technology) related to EC.

First, an EC model called “Live Commerce” has begun to spread, where talents and influencers transmit live video, and viewers can purchase products while asking questions and comments in real time.

Such "Live Commerce" is a mail order model in which EC is multiplied by the video of live delivery, and you can purchase goods while watching the live video, but as an image, it can be said that it is an interactive version of TV shopping, and viewers are exhibitors in real time.・You can shop while asking questions or comments about the seller.

In addition, an EC model called "Drama Commerce" is also beginning to spread, in which original dramas placed on the EC site are transmitted over the Internet and products appearing can be purchased.

Such "Drama Commerce" provides viewers with a sense of texture, silhouette, etc. that could not be conveyed on the EC site only for product descriptions with respect to products displayed in dramas (for example, products worn by talents or used by talents). It is said that it is possible to communicate, and it is attracting attention as a new measure that can overcome the weakness of the EC site.

The above-mentioned [first background art] is introduced in detail in Patent Document 1, Patent Document 2, Patent Document 3 and the like. First, in Patent Document 1, commercial information related to the movie while the movie is being displayed. Disclosed is a system and method for providing users with on-demand/access to a user and enabling the establishment of a link between a user and a seller who is the source of the commercial information. Various information devices, billing methods, and programs that enable video-related information to be obtained are disclosed. Further, Patent Document 3 discloses a method in which a viewer of a video program can access information about a product displayed in the video program. Systems and methods for providing an interactive viewing experience are disclosed.

[Second Background Art]

The second background technology in the present invention is a technology related to "image recognition AI", and image recognition by machine learning or deep learning utilizing AI (Artificial Intelligence) (Image Recognition) is related to the technology applied to the EC field.

When a user uploads a product image taken with a smartphone, etc. to the search function of the EC site, category identification, subject recognition, feature detection, etc. are performed by image recognition. A structure in which the same product or similar product is extracted from an image and displayed as a recommended product, and a moving picture editing system that can easily produce a moving picture or image content exclusively for the EC site by means of an image recognition function, etc. are widespread.

As such a structure or system requires a technique for object detection, in recent years, "image recognition AI" for object detection using deep learning identifies the types of a plurality of objects reflected in an image (dog, cat, car, etc.). etc.) data, as well as information such as position data of a moving object in a moving image, can be acquired at high speed and with high precision.

The above-mentioned [Second Background Art] has been described in detail in Non-Patent Document 1 and Non-Patent Document 2, and in recent years, such advanced technology regarding “image recognition AI” has been developed by cloud services. Companies that provide them are also starting to appear.

For example, in Non-Patent Document 3, a service capable of adding an image or video analysis function to an application using an API (Application Program Interface) is introduced. By identifying a plurality of displayed objects, data on the object can be acquired, and annotation (tagging) on the data on the object can be easily performed.

[Third background art]

The third background technique in the present invention relates to a technique related to "Audio Watermark", and it relates to a technique for storing, for example, text information obtained by encrypting an audio signal.

By using the technology related to "audio (sound) watermark", text information is inserted into the sound emitted from TV, radio, advertisement signage, and content, and this text information is acquired through the microphone of the smartphone. It becomes possible to build applications that link actions based on .

The above-mentioned [Third Background Art] is described in detail in Patent Document 4 and Patent Document 5, and in recent years, such a technology related to “audio (sound) watermark” has been developed in SDK (Software Development Kit). Companies that provide them are also starting to appear.

For example, in Non-Patent Document 4, a service capable of incorporating "audio (sound) watermark" technology into various applications using the SDK is introduced. Handling of contents linked with commercial contents (second screen and double screen measures, T2O2O <TV to Online to Offline>) can be easily performed.

[4th background technology]

The fourth background technology in the present invention is a technology related to "DAM" (Digital Asset Management), and is a technology for centrally managing digital contents such as text, video, photo, and catalog data using a platform or the like. is about

The system using the technology related to DAM is (1) aggregating various digital data, giving meta data, and facilitating access to necessary information, (2) transmitting media (Web site, EC site, catalog, SNS, etc.) ), data format conversion or data size change, (3) management of expiration dates related to copyrighted data, and (4) facilitation of the production process that converts creative production of digital content into a workflow; It provides functions to realize etc.

Video contents such as movies, dramas, animations, and music videos are not only funded, but also various resources (investor, director, original author, playwright, casting director, actor, voice actor, music producer, performer, interpreter/translator, shooting staff, editing staff) Persons or corporations such as lighting staff, decoration designers, costume designers, stylists, hair designers, makeup artists, brawl scene staff, stunts, extras, etc. Also, locations such as locations or studios where scenes are shot, moreover, animals or Small tools such as vehicles, clothes, ornaments, interiors, and furthermore, cameras, lighting, CG, equipment and technology used for recording, editing, etc.) When information on these resources is aggregated and metadata is given using related technologies, access to necessary information, conversion of data format, and change of data size are facilitated, and UI functions according to the EC model are automatically built. work becomes possible.

Regarding the above-mentioned [Fourth Background Art], many introductions have been made on the Internet. For example, in Non-Patent Document 5, browser-based file management and video management, video scene management/search, copyright information management, and download Functions related to control, application for use workflow, etc. are described.

Incidentally, the contents of all prior art documents raised above are incorporated herein by reference.

[Patent Document 1] Japanese Patent Laid-Open No. Hei 8-287107 [Patent Document 2] Japanese Patent Laid-Open No. 2002-334092 [Patent Document 3] Japanese Patent Application Laid-Open No. 2013-511210 [Patent Document 4] Japanese Patent Laid-Open No. 2008-058953 [Patent Document 5] Japanese Patent Laid-Open No. 2009-268036

[Non-patent document 1] “Machine learning from scratch” (machine learning overview) [Search September 10, 2019], Internet <https://qiita.com/taki_tflare/items/42a40119d3d8e622edd2> [Non-Patent Document 2] "Image Recognition by Deap Learning", the Journal of the Japanese Robotics Society [Vol. 35 No. 3 pp. 180~185, 2017] April 2017 [Non-Patent Document 3] Amazon "Amazon Rekognition" [Searched on September 10, 2019], Internet <https://aws.amazon.com/jp/rekognition/> [Non-Patent Document 4] Evixar 「ACR (Automatic Content Recognition), Sound Sensing」[Search September 10, 2019], Internet <https://www.evixar.com/evixaracr> [Non-Patent Document 5] Visual Processing Japan “Digital Asset Management” [Search September 10, 2019], Internet <http://www.cierto-ccc.com/cierto/function.html#dam>

“Live Commerce” and “Drama Commerce” described in the above [First Background Art] are ECs linked with video content, and are mainly produced and transmitted for the purpose of selling goods (goods or services), The audience target is focused on those who wish to purchase the product, but the original video content (movies, dramas, animations, music videos, etc.) is produced and distributed along with its story, entertainment, and artistry, so that audience target is focused on general viewers of all levels of the public by the theme of the video content, of course, since the above-mentioned various resources are put into the production of such video content, for example, research and trading of these resources If it can be easily carried out through content viewing, industry stakeholders involved in advertising, marketing, video content production, etc. can also focus as a target audience. [First background art] and [Second background art] In the prior art literature presented by , there is no specific method or technique for the investigation or transaction of such a resource.

In addition, the EC model by "drama commerce" described in the above-mentioned [first background art] is a so-called product placement (which is one of the advertising methods, and in the play of a movie or TV drama, an actor's small tool or background It is about a method that combines an EC site with a method of displaying an actual company name, product name, trademark, etc.) and, for example, while putting up a scene from a drama in the form of TV shopping, "In this scene, the actor 0000 was wearing Although there are models that sell while introducing products with "00000", models that induce purchases from links associated with scenes of dramas, etc. , people, space, props, equipment, technology, etc.), there is a problem with the response method to fluidity, which is intuitively conscious, [First background technology], [Second background technology], [Third background] Techniques] and [4th Background Art], in the prior art literature, specific techniques for solving such problems, for example, "image recognition AI", "audio (sound) watermark", "DAM", etc. The configuration of the UI utilizing the technology is not disclosed.

The present invention relates to a technology for integrating EC into video content, EC enabling viewers of video content to trade various resources (including displays of video content) invested in the production of the video content as a commodity The purpose of this is to provide a production method, a transmission system, and a transmission method of EC-integrated meta media including UI functions.

[Production method of EC integrated metamedia]

A method for producing EC integrated metamedia according to a non-limiting and exemplary kind of the present invention,

[a] A step of registering product information in a database for managing product data (product data management DB);

[b] creating a table (EC product table) for managing information on EC processing of the product;

[c] a step of creating a file (editing information sharing file) for sharing the editing information of the video content;

[d] creating a file (scene management file) for managing scene information from scene information in the edited information shared file and adding a product ID of the product data management DB;

[e] registering the scene data of the scene management file in a database (scene data management DB) for managing the scene data;

[f] registering the video data for public use of the video content in a database (video data management DB) for managing the video data;

[g] It characterized by including the step of generating the learned data for object detection from the scene of the said video data for publication, the scene data of the said scene data management DB, and the product data of the said product data management DB.

Further, it is characterized in that the step [e] includes a step of inserting an audio watermark (audio encoded (encoded) identification information) into each scene of the video content.

[EC Integrated Metamedia Transmission System]

An EC integrated metamedia transmission system according to a non-limiting and exemplary kind of the present invention,

It is an EC integrated metamedia delivery system that includes an EC UI function that allows viewers (users) of video content to trade the production resources of the video content as a commodity, and displays video content on the user's (viewer) client device. video content display means;

scene selection operation detecting means for detecting, with the client device, a selection operation by the user (viewer) for the scene displayed in the video content;

scene-related data acquisition means for acquiring the scene identification information and scene image data from the client device at the time of the detected selection operation;

object detection means for detecting an object included in the acquired scene image data;

product retrieval means for retrieving product information based on the acquired identification information;

object combining means for combining whether the object of the object detection result is included in the product information of the product search result;

UI-processed scene image data generating means for generating UI-processed scene image data in which a link element is attached to the range of the scene image data in which the object of the object combination result is displayed;

UI-processed scene image data call operation detecting means for detecting, with the client device, a call operation by the user (viewer) on the generated UI-processed scene image data;

By the detection, the UI-processed scene image data is transmitted to the client device, and a selection operation by the user (viewer) for the link element included in the UI-processed scene image data displayed is detected by the client device. selection link element acquisition means for acquiring a link element of the detected selection operation from the client device together with

link element-corresponding product information retrieval means for retrieving the acquired product information corresponding to the link element;

The retrieved product information is transmitted to the client device, and the selection operation of the EC processing type by the user for the product of the displayed product information is detected by the client device, and the EC processing type of the detected selection operation is detected. EC processing type acquisition means acquired from the client device;

EC processing calling means for calling EC processing for the product based on the acquired EC processing type is provided.

In addition, it is characterized in that the EC processing called by the EC processing calling means is provided with a transaction processing means based on a smart contract (contract).

[First method of transmission of EC-integrated metamedia]

A first method of transmitting EC integrated metamedia according to one non-limiting and exemplary type of the present invention,

An EC integrated metamedia transmission method including an EC UI function that enables a user (viewer) of video content to trade the production resource of the video content as a commodity,

[a] a step of displaying the video content on the user's client device;

[b] detecting, by the client device, a selection operation by the user for a scene displayed in the video content;

[c] acquiring, from the client device, identification information and scene image data of the scene at the time of the detected selection operation;

[d] a step of detecting an object included in the acquired scene image data;

[e] a step of searching for product information based on the acquired identification information;

[f] combining whether the object of the detection result is included in the product information of the search result;

[g] generating UI-processed scene image data in which a link element is attached to the range of the scene image data in which the object of the combination result is displayed;

[h] detecting, by the client device, a call operation by the user for the generated UI-processed scene image data;

[i] By the detection, the UI-processed scene image data is transmitted to the client device, and a selection operation by the user for the link element included in the UI-processed scene image data displayed is detected by the client device and acquiring, from the client device, a link element of the detected selection operation;

[j] a step of retrieving product information corresponding to the acquired link element;

[k] When the retrieved product information is transmitted to the client device and displayed, an EC processing type selection operation by the user for the product of the displayed product information is detected by the client device, and the EC of the detected selection operation is detected. acquiring the processing type from the client device;

[1] It is characterized by including the step of calling the EC process with respect to the said product based on the said acquired EC process type.

Further, it is characterized in that the step [g] includes a step of saving the UI-processed image data displayed on the client device.

In addition, when the above steps [b], [c], [d], [e], and [f] are performed multiple times while the video content in step [a] is being displayed on the client device, the step [ g], and saving the UI-processed scene image data generated in the process each time it is executed.

When the UI-processed image data transmitted to the client device in the step [i] is all the saved UI-processed image data, and the UI-processed scene image data is plural, the plurality of images displayed on the client device UI-processed image data is displayed in a thumbnail format.

It is characterized in that the EC processing for the product called based on the EC processing type in the step [1] includes a smart contract between the user and a supply source of the product.

[Second transmission method of EC integrated metamedia]

A second transmission method of EC integrated metamedia according to a non-limiting and exemplary aspect of the present invention,

[a] a step of inserting an audio watermark (audio-encoded identification information) into each scene of the video content;

[b] a step of broadcasting the video content on a general-purpose viewing device;

[c] detecting, by the client device, that the user (viewer) selects a scene displayed in the video content;

[d] acquiring, from the client device, identification information of a scene at the time of the selection operation by detection of the selection operation;

[e] a step of retrieving product information based on the acquired identification information;

[f] transmitting the product information to the client device;

[g] displaying the product information on the client device by the transmission;

[h] receiving the EC processing by the user for the product of the product information displayed on the client device;

[i] A step of referring to the EC processing type of the product information upon receipt of the EC processing;

[j] It is characterized by including the step of calling the structure of the EC process corresponding to the said EC process type by the said reference.

a step of retrieving video data based on the identification information obtained in step [d] and obtaining scene image data corresponding to the identification information from the searched video data;

a step of generating UI-processed display image data in which a link element is attached to the range of the scene image data in which the product of the product information searched in step [e] is displayed;

a step of replacing the configuration of the step [f] with a configuration of transmitting the UI-processed scene image data to the client device;

a step of replacing the configuration of the step [g] with a configuration of displaying the UI-processed scene image data on the client device;

Before executing the step [h], the client device detects that the user selected the link element included in the UI-processed scene image data displayed on the client device, and retrieves the detected link element from the client device. steps to obtain,

and retrieving product information based on the acquired link element, and transmitting product information of the search result to the client device for display.

According to the present invention, in addition to goods (goods or services) related to video content, it is related to the production of video content, such as funds, people (including corporations), space, props, equipment and technology invested in the production of video content. Resources (resources) can also be sold in direct connection with EC, making it easier to procure resources related to video content production, and distribution of EC sales directly related to video content to these procurement sources, especially for drama production. Accordingly, the authority scheme led by the sponsor or broadcasting station is liberated, and it is possible to produce a work that flexibly reflects the intention of the production director (eg, director or director, etc.).

[FIG. 1] A schematic diagram of Example 1 is shown. (representative)
[Fig. 2] Fig. 2 shows a flowchart related to the scene management data creation process.
[Fig. 3] Fig. 3 shows a flowchart related to the process of creating an object detection model.
[FIG. 4] shows a flowchart of the EC integrated metamedia transmission process.
[FIG. 5] shows a schematic diagram of the EC integrated metamedia transmission process.
[Fig. 6] Fig. 6 shows a flowchart of an EC processing process related to a product.
[Fig. 7] A flowchart of a voice (sound) watermark control process is shown.
[Fig. 8] An example of the format of the edit information shared file is shown.
[Fig. 9] A code example of a machine learning model related to image recognition is shown.
[Fig. 10] A code example of a machine learning model related to object detection is shown.
[Fig. 11] A configuration example of a main database is shown.

[Summary of the invention]

The present invention relates to a technology for integrating EC into video content such as movies, dramas, animations, and music videos. Various resources (person, space, props, equipment, technology, etc.) invested in the production of to provide.

In order to achieve the above object, the present invention utilizes technologies related to "image recognition AI", "voice (sound) watermark", "DAM", etc., and automatically generates a UI for integrating EC into video content. configuration is disclosed.

Specifically, if the techniques of the above-mentioned [First Background Art], [Second Background Art], and [Fourth Background Art] are combined, DAM technology is used first, and various functions related to drama production are used. By consolidating information about resources and providing metadata, access to necessary information is facilitated, data format conversion and data size are changed, etc., EC sites configured with an optimal UI for sales target It can be built, for example, an object (eg, a “person” such as an actor or a model) in which a user (general viewer = consumer) is intuitively interested in video content (eg, a drama), for example , a scene in which a "space" such as a tourist destination or a restaurant where the location was performed, for example, a "property" such as clothes or accessories) appears, is saved by the UI, and later saved using the UI is called, the resource (including the object) existing in the scene is recognized by the image recognition AI and a frame is created, and the user selects an object of interest to obtain information about the object or It becomes possible to provide an EC site integrated into the video content that the subject can be purchased.

In addition, when [3rd Background Art] is combined with the above combination, an ID (identifier) for identifying each scene in the post-production stage of the drama production is set using the "Audio (sound) watermark" description for the drama. Since it can be installed as an inaudible sound for the sound of A scene (smartphone) that is transmitted to the center server while the ID is acquired from the "audio (sound) watermark" installed in the sound of the drama through the dedicated application installed in the phone through the microphone (smartphone) The EC site displaying the scene from the point of view, i.e., corresponding to the scene in which the object of which the user is intuitively interested appears) can be called, and the user is present in the scene framed by the image recognition AI It is also possible to select a subject from the above-mentioned resources, and to obtain information about the subject or to purchase the subject.

According to the present invention, resources (resources) related to the production of video contents, such as funds invested in the production of video contents such as movies, dramas, animations, and music videos, people (including corporations), space, props, equipment and technology, etc. ) can also be sold in direct connection with the EC, making it easy to procure resources related to the production of video content.

In addition, according to the present invention, since EC sales directly linked to video content such as movies, dramas, animations, and music videos can be distributed to these procurement sources, the authority scheme led by sponsors and broadcasting stations is liberated, particularly in drama production, , it becomes possible to produce works that flexibly reflect the intentions of the production director (eg, director or director, etc.).

Furthermore, according to the present invention, a viewer (consumer) of video content such as a movie, drama, animation, or music video, for example, preserves a scene in which an object of intuitive interest appears in a drama by the UI, EC site integrated into video content (drama), where the saved scene is called up later, and an object of interest is selected from the image of the scene to obtain information about the subject or purchase the subject. It becomes possible to provide

Incidentally, the first embodiment of the present invention is characterized in that, by means of a dedicated viewing system, metamedia integrated with EC functions that can easily and directly purchase objects (products) in the video content being viewed from the viewing scene are transmitted. In the second embodiment of the present invention, it is characterized in that it is an EC function that can easily and directly purchase goods sold in video content from the scene without requiring a dedicated viewing system.

[Example 1]

Hereinafter, Embodiment 1 of the present invention will be described in detail.

Embodiment 1 of the present invention consists of [scene management data generation process], [object detection model creation process], [EC integrated metamedia transmission process], and [EC processing process for goods].

First, in the [Scene Management Data Generation Process], various resources (people, spaces, props, equipment, technology, etc.) invested in the production of video content such as movies, dramas, animations, and music videos are commercialized, and viewers In order to facilitate purchase by Electric Commerce), all resources that can sell the resource are identified as a product, and information about the product and EC processing for the product are classified (purchase or contract, etc. of product transaction). It is the process of creating and recording the configuration of EC processing depending on the form).

Incidentally, since the video content as described above is carefully planned what kind of resources to throw at the stage where production is decided, it is also performed based on clear information about the method of procurement of the resource, so the EC processing procedure is It is determined on the basis of information on procurement methods.

Specifically, (1) information on products (resources that can be sold) (detailed information of products and information on the classification of EC processing for products, and consists of digital data such as images, sounds, characters, and symbols) Along with the creation of an EC product table including A third process for generating an XML file from (recorded) video (recording a scene ID), (4) a fourth process for generating a scene management file from the XML file by a format conversion program (our company), (5) 5th process of adding a node related to basic information (example) to the scene management file, (6) Dynamic information (example) and product for each scene of the video edited by the information/product registration program (our company) 6th process of registering ID (addition of a node), (7) Scene data (basic information, dynamic information, product ID, etc.) of each scene from the registered scene management file by the scene data generation program (company) A seventh process of registering an ID in the scene data management DB in order to attach an ID and convert it into a DB as scene data, (8) attach a video ID to the final edited video by a video data storage program, and convert it to a DB as video data for public use In order to do this, the [scene management data generation process] is constituted by the eighth process stored in the video management DB.

Incidentally, regarding (2) video shooting (recording), for example, if video data of video content recorded by a third party (third party of video content) can be obtained, the [scene management data creation process].

Next, in the [object detection model creation process], in various scenes of video content such as movies, dramas, animations, and music videos, objects (goods, services, people, spaces, props, equipment, It is a process to enhance the ability of "Image Recognition AI" to instantly determine whether or not the technology, etc.) belongs to the EC target product.

Specifically, (1) a first process of creating learning data from each scene, scene data, and product data of an edited video by a learning model creation program, (2) creating a learning model by machine learning based on the created learning data A second process for determining the detection accuracy of the learning model while increasing it; (3) a third process for outputting a learning model (including product IDs) whose detection accuracy has reached a certain level or more and saving it in a dedicated storage file is composed by

In addition, the [EC integrated metamedia transmission process] is an EC function that can commercialize various resources (people, spaces, props, equipment and technologies, etc.) that are invested in the production of video content such as movies, dramas, animations, and music videos. This is the process of transmitting meta media that is integrated into the video content itself.

Specifically, (1) when a user accesses the center system with a PC or smartphone (for example, clicking/touching the link for “EC integrated metamedia transmission service” introduced on the browser portal site) , the first process in which the "EC integrated metamedia transmission site" is redirected from the center system, (2) the video (movies, dramas, animations, music videos, etc.) introduced by the user on the "EC integrated metamedia transmission site" of video content), a second process in which a dedicated video player is downloaded to the user's PC or smartphone, etc. (3) When the user clicks/touches the play button of the dedicated video player, the selected video is played ( 3rd process to be viewed), (4) When a scene selection operation (for example, a screen is clicked/touched by the user) during video playback, scene identification information (video ID, time code, etc.) A fourth process in which image data is acquired by the center system, (5) a fifth process in which scene data (scene ID, basic information, dynamic information, product ID list, etc.) is acquired from the scene data management DB by scene identification information ; 6th processing in which a product ID list included in the scene image is detected by processing (inference by the learning model), (8) the verified product ID list of the 6th processing and the product ID list of the 7th processing are combined; The eighth process of UI processing (spherical shape and link attachment, etc.) to the product image of the matching product ID, (9) When the user clicks/touches the rectangle, the corresponding product is extracted and displayed. .

Incidentally, regarding the provision of the dedicated video player, there are a method of running a dedicated web application with a browser, and a method of allowing a user to install and run the dedicated application on a smartphone.

And, in the [EC processing process for products], the products included in each scene are purchased by the viewers of the EC integrated metamedia by the EC function, and the revenue from the purchase is not only the product supply source but also the video content. It is a process that is also allocated to the producers of

Specifically, (1) the type of EC processing (purchase, contract, bidding, etc.) for the product displayed by the ninth processing of the [EC integrated metamedia transmission process] described above is determined by the user (menu selection, etc.) First processing, (2) second processing in which product information is displayed and product purchase by the user is determined, (3) third processing in which order information is input by the user and payment processing is performed, (4) order receipt information is sent to the center A fourth process transmitted to the system, (5) a fifth process in which order information (shipping address, delivery conditions, payment conditions, and settlement information) is transmitted from the center to the product supply source; (6) the user from the product supply source based on the order information The sixth process, in which goods are delivered to , (7) consists of seventh process in which the product price is deposited to the product supplier, dividends are deposited to the producer, and margins are deposited to the center.

Incidentally, with respect to a server-only program or client (PC, smartphone, etc.) dedicated application for executing each process in each process described above, it can be developed by JAVA, C, C++, JavaScript, python, etc., but As software, for example, with respect to video editing, Blackmagic Design - DaVinch Resolve (AFF, XML), SONY - VEGAS Pro (AAF), etc., also, for example, with regard to audio (sound) watermark control. , it is okay to use the SDK of Evixar, which is introduced in Non-Patent Document 4.

Fig. 1 shows a schematic diagram of &quot;EC integrated metamedia creation method and transmission system&quot; realized by the present invention.

Incidentally, the original text of the abbreviations marked with (*) in Fig. 1 is as follows.

About [MAIN SYSTEM],

APP*= APPLICATION

DBM*= DB MANAGEMENT

About [APP* SERVER],

ATP* = Authentication Process

VED* = Video Edit

FMX* = Format Exchange

IGR* = Info & Goods Register

SDG* = Scene Data Generator

VDS* = Video Data Storage

LMC* = Learning Model Creator

VDP* = Video Player

WMC* = Watermark Control

About [DBM* SERVER],

USR* = User

PRD* = Producer

SPL* = Supplier

GDD* = Goods Data

SCD* =Scene Data

VDD* = Video Data"

Hereinafter, the main processing steps constituting the present invention indicated by the symbol [A] in FIG. 1 will be described with reference to FIG. 1 and detailed views of the respective processing steps.

[Scene management data generation process]

FIG. 2 is a flowchart of a [scene management data generation process] (SMDG: Scene Management Data Generation) indicated by the symbol [A] in FIG. 1 .

First of all, the [scene management data generation process] is based on various resources (investor, director, original author, playwright, casting director, actor, voice actor, music producer, performer) that are invested in the production of video content such as movies, dramas, animations, and music videos. , interpreters/translators, shooting staff, editing staff, lighting staff, decoration designers, costume designers, stylists, hair designers, makeup artists, brawl scene staff, stunts, extras, etc. By commercializing spaces such as studios and studios, small tools such as animals, vehicles, costumes, ornaments, interiors, and equipment and technologies used for cameras, lighting, CG, recording, editing, etc.), viewers can use EC functions In order to facilitate purchase by The process of creating and recording (information for invoking the implementation structure of EC processing appropriate to the transaction type of the product, such as a contract in the case of a transaction of this person, equipment, technology, etc., and a bid when the product is exhibited in an auction) , and is constituted by the processing of the main 8 steps.

In the first step shown in Fig. 2 (1), information (detailed product information and product information) about a product that can be sold (thing of the resource that can be sold) is provided by the EC integrated metamedia in the present invention. The EC product table 6230 is generated by the EC product data 7240 including information on the type of EC processing for , the EC product ID of the EC product table 6230 is registered in the product data management DB 5210 .

Incidentally, a configuration example of the product data management DB 5210 and the EC product table 6230 is shown in FIG. 10 .

In the second step shown in FIG. 2 (2), video contents such as movies, dramas, animations, and music videos using resources provided by rental, investment, or donation from the product (production resource) supply source 1400 shown in FIG. 1 . In addition to being photographed by the video content creator 1300 , the recorded video data 2400 is transmitted to the center 1100 (it may also be mailed).

The video data 2400 is, in the center 1100, included (embedded, embedded) in the EC integrated metamedia transmission.

In the third step shown by (3) in Fig. 2, the video data 2400 transmitted to the center 1100 in the second step is performed by the video editing software 3210 (VED*SW in Fig. 1). , an edit information sharing file 6210 (eg, in XML format) is output.

In addition, the edit information sharing file 6210 includes basic information (such as information about a video or a scene) necessary for generating EC-integrated metamedia, and an example of the format is shown in FIG. 8 .

In the fourth step shown by (4) of Fig. 2, the format conversion program 3220 (FMX*PG in Fig. 1) from the edited information shared file 6210 output in the third step, the scene is converted into a scene. A management file 6220 is created.

In addition, in the scene management file 6220, all information about a scene necessary for EC integrated metamedia transmission is recorded.

In the fifth step shown in FIG. 2 (5), basic information (eg, video description, drama or game information, event name and date and time) is added to the scene management file 6220 generated in the fourth step. A node (a tag in a document that hierarchically expresses data) is added.

In the sixth step shown by (6) in Fig. 2, the information/product registration program 3230 (IGR*PG in Fig. 1) for the scene management file 6220 to which the basic information is added in the fifth step. ), dynamic information for each scene of the edited video (for example, information that changes for each scene, such as shooting location, music, scene description, etc., a configuration example is shown in FIG. 10) and product data management DB 5210 The product ID of the product data 7210 is registered (addition of a node).

In the seventh step indicated by (7) of FIG. 2, the scene data generation program 3240 generates the data (basic information, basic information, Dynamic information, product ID, etc.) are extracted, a scene ID is attached, and is registered in the scene data management DB 5220 as scene data 7220 .

In the eighth step shown by (8) in FIG. 2, the video ID is attached to the final edited version of the video data 2410 edited in the third step process by the video storage program 3250, and the video data 7230 ) (video data for public use), stored in the video data management DB 5230

[Object detection model creation process]

Fig. 3 is a flow chart of a process relating to [object detection model creation process] (ODMC: Object Detection Model Creation) indicated by the symbol [A] in Fig. 1 .

First, in the [object detection model creation process], and in the second process in the object detection model creation process, the viewer of video content such as movies, dramas, animations, and music videos can access various scenes of the video content. A UI that can easily determine whether or not intuitively conscious objects (goods, services, people, spaces, props, equipment, technology, etc.) can purchase or purchase information for purchase as EC target products ( User Interface) to the viewer, adding an AI function related to object detection required for system construction, and enhancing this is a process of three main steps.

In the first step shown in Fig. 3 (1), video data is managed in the eighth step process of the [scene management data generation process] by the learning model creation program 3310 (LMC*PG in Fig. 1). Each scene of the video data 7230 saved for public use in the DB 5230, the scene data 7220 saved in the scene data management DB 5220, and product data 7210 saved in the product data management DB 5210 Learning data for machine learning is created from

In the 2nd step shown by (2) of FIG. 3, the detection precision of the said learning model is determined, raising the precision of a learning model by machine learning using the said learning data created in the 1st step.

In the third step shown in Fig. 3 (3), when the precision of the learning model becomes more than a certain level by machine learning in the second step, the learning model with the precision greater than or equal to the certain level is converted into the learned learning model data ( 7310 ), stored in a learning model preservation file 6310 .

Here, the library used for the learning model creation program as described above is widely introduced in books as well as the Internet. For example, Keras written in Python is a high-level neural network executable on TensorFlow, CNTK, or Theano. As a (neural network) library, it has been developed with an emphasis on enabling rapid experimentation, and is widely publicly available, <https://employment.en-japan.com/engineerhub/entry/2017 /04/28/ 110000#3-Image recognition using Inception-v3 model> In [Search on September 10, 2019], an execution script related to image recognition of animals (dogs and cats) is introduced, and the execution result is learning Models are also published, and there is, for example, PyTouch as a library for deep learning dedicated to Python, <https://github.com/amdegroot/ssd. In pytorch/blob/master/ssd.py> [Search on September 10, 2019], a code for deep object detection is also introduced. Fig. 10 shows a code example of machine learning related to the object detection.

[EC Integrated Metamedia Transmission Process]

FIG. 4 is a flowchart of an EC integrated meta-media transmission process (ECIMD: EC integrated Meta-media Distribution) described in symbol [A] in FIG. 1 .

The above-mentioned live commerce and drama commerce, etc., are transmitting video content to promote products that have already been sold to the general public or new products that will be sold in the future. It is separated and is limited to a simple link partnership, and it does not necessarily mean that you can only purchase products there.

The [EC integrated metamedia transmission process] in the present invention provides an EC function that enables viewers of video content such as movies, dramas, animations, and music videos to easily and directly purchase products from the viewing system. It is a system that transmits metamedia integrated into It is constituted by the process of 9 steps.

In the first step shown in Fig. 4 (1), the user 1200 in the client device such as the PC 2310 or the smartphone 2320, for example, "EC" introduced on the portal site of the browser When the link for "integrated metamedia delivery service" is clicked (touched) and the center system 2200 is accessed, the "EC integrated metamedia delivery site" is redirected.

In the second step shown in FIG. 4 (2), for example, a video (movie, drama, animation, movie, drama, animation, etc.) introduced in the "EC integrated metamedia transmission site" redirected by the user 1200 in the first step. When a favorite is selected from video content such as music video), a dedicated video player 2420 is downloaded from the center system 2200 to a client device such as a PC 2310 or a smartphone 2320 of the user 1200. .

Incidentally, regarding the provision of the dedicated video player 2420 described above, there are a method in which a dedicated web application is run by a browser, and a method in which a user installs and runs the dedicated application in a smartphone.

In the third step shown in Fig. 4 (3), when the user 1200 clicks (touches) the play button of the dedicated video player 2420, for example, the selected video is played back (by the user). viewing) is initiated.

In the fourth step shown in Fig. 4 (4), the video being reproduced by the dedicated video player 2420 whose reproduction is started by the user operation in the third step is a scene selection operation, for example, the video being reproduced. When the screen is clicked (touched) by the user 1200, scene identification information 7410 (including video ID, scene ID, time code, etc.) at the time of the click (touch) and a scene image of the scene Data 7420 is acquired by center system 2200 .

Incidentally, as for the acquisition method by the center system 2200, for example, the dedicated video player 2420 obtains the scene identification information 7410 and the scene image data 7420 by the scene selection operation. A method of acquiring and transmitting this to the center system 2200, for example, the center system 2200 monitors information regarding video reproduction in the dedicated video player 2420, and performs a scene selection operation At the time of , a method of directly acquiring the scene identification information 7410 and the scene image data 7420, for example, acquiring only the scene identification information 7410, and the above [scene management data creation process] In addition, video data corresponding to the video ID of the scene identification information 7410 stored in the video data management DB 5230 (Fig. 2(8)) is extracted from the extracted video data, and the scene identification information ( There is a method in which scene data that can be extracted based on the scene ID of 7410 is used as substitute data for the scene image data 7420 .

In the fifth step indicated by (5) in Fig. 4, based on the scene identification information 7410 (including video ID, scene ID, time code, etc.) acquired by the center system 2200 in the fourth step Thus, scene data retrieval processing is performed, and corresponding scene data (including video ID, scene ID, basic information, dynamic information, product ID list, and the like) is acquired from the scene data management DB 5220 .

In the sixth step shown in FIG. 4 (6), the product data inquiry process for the product ID list 7430 included in the scene data acquired in the fourth step, that is, the product ID list 7430 is Based on the product data in the product data management DB 5210, the product data is searched and whether the product data of the product ID list 7430 exists is inquired. there is).

In the seventh step shown by (7) of FIG. 4, object detection processing (inference processing by a learning model) based on the scene image data 7420 acquired in the fourth step (simultaneous with the processing in the fifth step) ), the product ID list 7440 included in the scene image data 7420 is detected.

In the eighth step shown by (8) of FIG. 4, the product ID list 7430 verified by the product data inquiry processing of the sixth step and the product ID list 7440 detected by the object verification processing of the seventh step is combined, and the existence of the product ID is confirmed by determination of the existence of the product ID (eg, it is determined whether the product ID included in the product ID list 7440 is present in the product ID list 7430 ). UI processing (spherical drawing processing for the product image, linking for product data acquisition by product ID, etc.) is performed on the product image (product image included in the scene image data 7420) of Processed scene image data 7460 is generated.

In the ninth step shown by (9) of FIG. 4, the UI-processed scene image data 7460 (there may exist a plurality) that has been UI-processed in the eighth step is transmitted to the PC 2310, the smartphone 2320, or the like. A product transmitted and displayed to the client device of , and included in the UI-processed scene image data 7460 by the user 1200 for the displayed UI-processed scene image data 7460, for example. When the rectangular range of the image is clicked (touched), product data corresponding to the product image is extracted from the product data management DB 5210 and displayed.

The above [EC integrated metamedia transmission process] is an important component in the present invention, and in addition to the description by the process flow diagram of FIG. 4, the symbol [B] in FIG. 1 (excluding [EC PROCESS]) The specific implementation image on the client side shown is demonstrated using the numbers (1) - (7) (the same number is attached to the affiliate process) described in the schematic diagram of FIG.

[EC Integrated Metamedia Transmission Process (Second Description)]

In the process of FIG. 5 ( 1 ), the user 1200 accesses the "EC integrated metamedia transmission site" from the client device 2300 such as the PC 2310 or the smartphone 2320, and performs a video (movie) or video content such as drama, animation, or music video) is selected, (1) the video is reproduced by the dedicated video player 2420 .

In the process of FIG. 5(2), when the user 1200 touches (clicks) a scene image of the video in a scene intuitively interested in the video being reproduced, for example, the touch (click) ), the scene identification information 7410 and the scene image data 7420 relating to the scene screen of the viewpoint are transmitted (acquired) to the main system 2200 of the center.

In addition, the operation (scene selection operation) by the user 1200 can be configured so that it can be performed any number of times during video playback without interrupting (pausing) or stopping the video playback. have.

Here, the scene identification information 7410 transmitted (acquired) to the main system 2200 of the center is transmitted by the main system 2200 from the fifth step in the [EC integrated metamedia transmission process] described above. Through the processing up to the eighth step, the UI-processed scene image data 7460 is generated.

Further, when the above-described scene selection operation has been performed several times, the UI-processed scene image data 7460 transmitted by the operation may be configured to be stored in the scene identification information storage file 7450 .

In the process of FIG. 5(3), for example, when the user 1200 interrupts (pauses) or stops the video reproduction and, for example, performs a scene call operation, the scene identification information storage file ( 7450), the UI-processed scene image data 7460 saved by the scene saving operation is transmitted to the client device 2300, and when there are a plurality of the UI-processed scene image data 7460, for example, the first The UI-processed scene image data is displayed in the main area of the client device (display screen), and the remaining UI-processed scene image data is displayed in the side area in thumbnail form.

In the process of FIG. 5( 4 ), the user 1200 selects (touches or clicks) any one of the thumbnails of the UI-processed scene image data 7460 (plural) displayed on the client device 2300 . Then, the selected UI-processed scene image data 7460 is displayed in the main area.

In the process of FIG. 5 ( 5 ), the user 1200 selects any one from object images (plural existences) subjected to spherical processing (surrounded by a sphere) included in the selected UI-processed scene image data 7460 ( touch or click within the spherical range), link data (including product ID) corresponding to the selected object image is transmitted to the center system 2200 from the product data management DB based on the product ID of the link data When the corresponding product data is extracted, the extracted product data is transmitted to the client device 2300 .

In the processing process of FIG. 5( 6 ), product information of the product data transmitted to the client device 2300 is displayed on the client device 2300 , and the type of EC processing for the product (eg, product) When the user 1200 selects the detailed information on , inquiry about a product, purchase of a product, etc.), the selection EC process proceeds.

The above is a detailed description regarding each processing process which comprises this invention described by the symbol [A] of FIG.

Next, the [EC processing process related to a product] constituting the present invention represented by the symbol [B] ([EC PROCESS]) in FIG. 1 will be described below with reference to FIG. 1 and FIG. 6 .

[EC processing process for products]

FIG. 6 is a CLIENT Application 4100 described in symbol [B] in FIG. 1 , that is, a web application downloaded or installed exclusively to a PC 2310 or smartphone 2320 that is a client device of the user 1200 shown in FIG. 1 . It is a schematic diagram regarding the [EC processing process related to goods] performed by the application (hereinafter referred to as the client application 4100).

First, in the present invention, the [EC processing process related to goods] is performed by a viewer of the EC integrated meta media in video contents such as movies, dramas, animations, and music videos transmitted by the EC integrated meta media. The products included in each scene, that is, not only the general EC products, but also various resources (people, spaces, props, equipment, technologies, etc.) invested in the production of the video content, provide the EC function integrated into the video content. Purchases made through the company, and the revenue related to the purchase is distributed not only to the product supply source (those who sell the resource or those who make money through rental, investment, donation, etc.) of the resource, but also to the creator or center of the video content It is a process that is performed, and is constituted by the processing of the main 7 steps.

In the first step shown in FIG. 6 ( 1 ), the selection is made by a spherical touch by the user 1200 in the ninth step of the [EC integrated metamedia transmission process] described above, and the PC 2310 or the smartphone ( 2320) type of EC processing for the products indicated in (e.g., purchase of marine products, contracts when marine products are transactions of people, equipment, technology, etc., bidding when marine products are exhibited in auction, etc. Information for invoking the implementation configuration of the EC processing suitable for the transaction form of .

In the second step shown by (2) in Fig. 6, when the purchase type is selected in the EC processing of the first step, detailed information about the product is displayed and the user 1200 decides to purchase the marine product. do.

In the third step shown in FIG. 6 ( 3 ), as order information of the product for which purchase was decided by the user 1200 in the second step, for example, order (purchasing) person information (here) of the product. In , information of the user 1200) and delivery information such as a shipping address and contact information of the product are input, and payment processing related to the purchase of the product is performed, for example, a payment agency 1500 The purchase price of the product is withdrawn from the bank account of the user 1200 .

Incidentally, regarding the input of order information in the third step, in the configuration of the first step or the second step, a login operation by the user 1200 (user authentication processing, in the present invention, FIG. 1) can be omitted by adding a configuration in which the center system 2200 accepts the ATP* PG [3110], and such a configuration is a configuration related to member registration employed in an existing EC site or the like. can be easily built using

In addition, with respect to the timing at which the purchase price is transferred from the bank account of the user 1200 by the payment processing described above, depending on the type of the product purchased by the user 1200, cooling-off is performed. ), even if the system is applied.

In the fourth step indicated by (3) of FIG. 6 , the order information input by the user 1200 in the third step and payment information by the user 1200 processed by the payment company 1500 The order receipt information based on ' is transmitted to the center system 2200 .

In the fifth step shown by (5) of FIG. 6 , the order information based on the order receipt information transmitted to the center system 2200 in the fourth step is transmitted from the center system 2200 to the information terminal of the product supply source 1400 . In addition to being transmitted (for example, by E-mail), a notification regarding the order information is also transmitted to the information terminal of the manufacturer 1300 or the like.

In the sixth step shown in FIG. 6 (6), based on the order information transmitted to the product supply source 1400 in the fifth step, the user 1200 (order information delivery destination) from the product supply source 1400 is the same as the address of the user 1200), the product is delivered.

In the seventh step shown by (7) of FIG. 6, the product price is transferred to the product supply source 1400 from the purchase price (sales amount) recovered by the settlement agency 1500 in the third step, and the dividend is transferred to the producer At 1300 , further, a margin is credited to center 1100 .

In the above example, viewers of EC-integrated metamedia and users who purchase products are spoken as general consumers. There are people in the industry, developers of new products or new services, etc., and such products are purchased (transacted) by such viewers, for example, recruitment of models or stuntmen, or hotels and restaurants on location sites. Since use, special shooting, and introduction of technology related to CG are also expected, it is necessary to construct an EC function that can respond to such transactions. Thus, for example, products accompanying contracts such as travel, insurance, securities, education, etc., can be transacted with a new concept such as smart contracts, and the EC structure corresponding to such transaction types has also been established. , if such a transaction structure is incorporated into the [product-related EC processing process] in the present invention, for example, a contract in the case where the product is a transaction related to human employment, use of equipment, transfer of technology, etc. It is possible to easily configure a function that can call the implementation configuration of EC processing suitable for the transaction form of the product, such as bidding when the product is exhibited in an auction.

As described above, with the configuration shown in Example 1, not only products (goods or services) related to video content but also various resources invested in the production of video content can be sold directly with EC. It becomes possible to provide a manufacturing method and a control method of a transmission system.

[Example 2

Hereinafter, Embodiment 2 of the present invention will be described in detail.

Embodiment 2 of the present invention, in addition to the [scene management data generation process], [object detection model creation process] and [product EC processing process] in Embodiment 1, [audio (sound) watermark control process] ].

Incidentally, the above [scene management data generation process], [object detection model creation process] and [product EC processing process] have already been described in the detailed description regarding Example 1 of the present invention, In the following, description is abbreviate|omitted.

First, in the [audio (sound) watermark control process], [audio encoding processing] for inserting the audio (sound) watermark into the EC integrated metamedia, and the audio (sound) installed in the EC integrated meta media It is constituted by two processes of [audio decoding process] for detecting a watermark.

First, in the [audio encoding process], in the [scene management data creation process] described in the first embodiment, from the scene management file edited with the information/approval registration program, the scene data generation program registers in the scene data management DB. Using the scene data and the edited video registered in the video data management DB by the video data preservation program from the scene data and the last edited video data, (1) generating scene identification information from the video ID and scene ID of the scene data; A first process, (2) a second process of encoding the generated scene identification information into audio (sound) watermark data using dedicated audio (sound) watermark control software, (3) video editing software, It is constituted by the third process of re-editing while inserting the audio (sound) watermark data into each scene of the edited video.

Next, [audio decoding processing] is, for example, a dedicated application having an audio (sound) watermark control function for video content by EC integrated metamedia that is transmitted (broadcast or reproduced) by the user to the TV. When this installed smartphone (supplied microphone) is pointed, (1) the audio of EC integrated metamedia output from the TV is recorded, and audio (sound) watermark data is obtained from the recorded audio by a dedicated application. A first process and (2) a second process in which the acquired audio (sound) watermark data is decoded and scene identification information (including video ID and scene ID) is detected.

For example, here, when the detected scene identification information is transmitted from the user's smartphone to the center system, product list data based on the scene identification information is generated from the product data management DB, and the generated product list data is When transmitted from the center system to the user's smartphone, the transmitted product list data is displayed on the smartphone, and it is possible to shift to EC processing selection for purchasing a favorite product from the product list.

Fig. 7 shows a flow of processing related to [audio (sound) watermark control process] in Example 2 of the present invention, and the flow of symbols [A1] to [A4] is [audio (sound) watermark control process/ audio encoding process], and the flow of symbols [B1] to [B4] is [audio (sound) watermark control process/audio decode process].

[Voice (sound) watermark control process/Voice encoding process]

First, in the [audio (sound) watermark control process/audio encoding process] in the present invention, text data of scene identification information is synthesized into inaudible sound and stored in each scene of a video for public use (for editing of video audio) ) process, and is constituted by the process of three main steps.

In the first step indicated by [A1] in Fig. 7, scene data 7220 is obtained from the scene data management DB 5220 based on the video ID of the video in which the audio (sound) watermark is inserted by the scene identification information generation process. is extracted, scene identification information 7410 including the video ID and the scene ID of the scene data 7220 is generated.

In the second step indicated by [A2] in Fig. 7, the scene identification information 7410 generated in the first step is converted to audio (sound) watermark data by the dedicated audio (sound) watermark control software 3510. (7520) is encoded.

In the third step shown in Fig. 7 [A3], the audio watermark data 7520 encoded in the second step is a video video into which the audio (sound) watermark is inserted by the video editing software 3210. Based on the ID, the video data 7230 extracted from the video data management DB 5250 is re-edited while being installed in each scene.

[Voice (sound) watermark control process/voice decode process]

First, the [audio (sound) watermark control process/audio decode process] in the present invention is performed from the audio watermark (text data of scene identification information synthesized with inaudible sound) stored in each scene of the public video. It is a process of extracting scene identification information, and is constituted by the process of 4 main steps.

In the first step indicated by [B1] in Fig. 7, the user 1200 transmits (broadcasts or plays) the video content ([audio (sound) watermark control process/audio input) by the EC integrated meta media to the TV. code processing], when a scene selection operation, i.e., a smartphone 2320 in which a dedicated application 4510 having an audio (sound) watermark control function is installed, is directed to the smartphone 2320 ), the audio (sound wave) of the EC integrated metamedia transmitted (broadcast or reproduced) to the TV is input to the microphone unit of the ), and recorded data 7510 based on the audio (sound wave) is generated by the dedicated application 4510. At the same time as being generated, audio watermark data relating to the scene of the video content at the time of recording of the recorded data is acquired.

In the second step indicated by [B2] in Fig. 7, scene identification information 7410 (video ID, scene) about a scene at the time of recording from the acquired audio watermark data by the dedicated application 4510 ID) is detected by the audio decoding function, and the detected scene identification information 7410 is transmitted to the center system 2200 .

In the third step shown in FIG. 7 [B3], the product list data 7530 is retrieved from the product data management DB 5210 by product data search processing based on the scene identification information 7410 transmitted to the center system 2200. ) is generated, the generated product list data 7530 is transmitted to the smartphone 2320 .

In the fourth step shown in FIG. 6 [B4], the transmitted product list data 7530 is displayed on the smartphone 2320 by the dedicated application 4510, and the user 1200 EC Accepts treatment choices.

In addition, instead of the product list data 7530 transmitted to the smartphone 2320, the UI-processed scene image data described in the above [Second description of the EC integrated metamedia transmission process] is transmitted to the smartphone ( 2320), when the scene selection operation is performed several times, the UI-processed scene image data (plural) is displayed as thumbnails as described above, so that a more convenient product selection method can be provided to the user. can

In this case, with respect to the scene image data constituting the UI-processed scene image data (plural) transmitted to the smartphone 2320, the scene image data of the video image being transmitted to the TV cannot be directly acquired. , by the scene data (including video ID and scene ID) retrieved by the scene data retrieval process based on the scene identification information 7410 transmitted to the center system 2200; In the eighth step, a scene in which video data transmitted to the TV is referenced (searched) from the video data management DB 5230 shown in FIG. Since the image data can be acquired, as described in step 8 of the [EC integrated metamedia transmission process] described above, the product corresponding to the product list data 7530 included in the acquired scene image data is displayed. The UI-processed scene image data (pluralities) can be configured by UI processing for a range.

As described above, with the configuration shown in the second embodiment, it is possible to provide an EC function that can easily and directly purchase goods sold in video content from the scene without requiring a dedicated viewing system. .

According to such a configuration, the EC integrated video content can be transmitted through TV broadcasting, and for example, a scene image of the EC integrated video content can be captured by simply pointing a smartphone at the EC integrated video content broadcast on the street. Since it can be acquired as such and, moreover, it can be provided as UI-processed scene image data, the user's impulsive interest in the product is not lost.

1100 Center
1200 users
1300 Producer
1400 commodity supply
1500 payment companies
1900 internet
2100 Communication Control Unit (Router, etc.)
2200 main system (EC integrated metamedia transmission system)
2210 WEB Server Device
2220 application server device
2230 database management server unit
2300 client devices
2310 pcs
2320 Smartphone
2410 original video
2420 dedicated video player
2500 subsystem (for EC systems)
3110 ATP* PG>Authentic Process (authentication processing program)
3210 VED* SW>Video Editing (video editing software)
3220 FMX* PG>Format Exchange (format converter)
3230 IGR* PG>Info & Goods Register (information and product registration program)
3240 SDG* PG>Scene Data Generator
3250 VDS* PG>Video Data Storage (video data storage program)
3310 LMC* PG> Learning Model Creator
3510 WMC* SW>Watermark Control (voice (sound) watermark control software)
4100 client application
4210 VDP* AP>Video Player (dedicated video player application)
4510 WMC* AP>Watermark Control (voice (sound) watermark control application)
5110 USR* MNG DB>User (User Management DB)
5120 PRD* MNG DB>Producer (Producer Management DB)
5130 SPL* MNG DB>Supplier (source management DB)
5210 GDD* MNG DB>Goods Data (product data management DB)
5220 SCD* MNG DB>Scene Data (Scene Data Management DB)
5230 VDD* MNG DB>Video Data (video data management DB)
6210 Edit Information Share File
6220 Scene Management File
6240 EC Commodity Table
6310 Training Model Preservation File
7210 product data
7220 scene data
7230 video data
7240 EC product data
7310 training model data
7410 Scene Identification Information
7420 Scene image data
7430 Product ID list (obtained by product inquiry process)
7440 Product ID list (detected by object verification processing)
7450 Scene Selection Information Preservation File
7460 UI processed scene image data
7510 Recorded data
7520 Voice Watermark Data
7530 product list data

Claims (11)

A method for producing EC-integrated metamedia including an EC UI function that enables a user who becomes a viewer of video content to trade the production resource of the video content as a commodity, the method comprising:
[a] A step of registering product information in a product data management database for managing product data;
[b] creating an EC product table for managing information on EC processing of the product;
[c] A step of creating an editing information sharing file for sharing the editing information of the video content;
[d] creating a scene management file for managing scene information from the scene information in the edited information sharing file and adding a product ID of the product data management database;
[e] registering the scene data of the scene management file in a scene data management database for managing the scene data;
[f] registering the video data for public use of the video content in a video data management database for managing the video data;
[g] A production method comprising a step of generating learned data for object detection from scenes in the video data for publication, scene data in the scene data management database, and product data in the product data management database. According to claim 1,
The production method, wherein the step [e] includes a step of inserting an audio watermark into each scene of the video content. An EC integrated metamedia delivery system including an EC UI function that enables a user who becomes a viewer of video content to trade production resources for the video content as a commodity, the system comprising:
video content display means for displaying video content on a user's client device;
scene selection operation detecting means for detecting, with the client device, a selection operation by the user on the scene displayed on the video content;
scene-related data acquisition means for acquiring the scene identification information and scene image data from the client device at the time of the detected selection operation;
object detection means for detecting an object included in the acquired scene image data;
product retrieval means for retrieving product information based on the acquired identification information;
object combining means for combining whether the object of the object detection result is included in the product information of the product search result;
UI-processed scene image data generating means for generating UI-processed scene image data in which a link element is assigned to a range of the scene image data in which the object of the object combination result is displayed;
UI-processed scene image data call operation detecting means for detecting, to the client device, a call operation by the user on the generated UI-processed scene image data;
When the UI-processed scene image data is transmitted to the client device by the detection and the client device detects a selection operation by the user on the link element included in the UI-processed scene image data displayed selection link element acquisition means for acquiring, from the client device, a link element of the detected selection operation;
link element-corresponding product information retrieval means for retrieving the acquired product information corresponding to the link element;
The retrieved product information is transmitted to the client device, and the selection operation of the EC processing type by the user for the product of the displayed product information is detected by the client device, and the EC processing type of the detected selection operation is detected. EC processing type acquisition means acquired from the client device;
and EC processing calling means for calling the EC processing for the product based on the acquired EC processing type. 4. The method of claim 3,
A transmission system, characterized in that the EC processing invoked by the EC processing calling means includes transaction processing means based on a smart contract. It is a method of transmitting EC integrated metamedia including a UI function of EC that allows a user who becomes a viewer of video content to trade the production resource of the video content as a product,
[a] displaying the video content on the user's client device;
[b] detecting, by the client device, a selection operation by the user for a scene displayed in the video content;
[c] acquiring, from the client device, identification information and scene image data of the scene at the time of the detected selection operation;
[d] a step of detecting an object included in the acquired scene image data;
[e] a step of searching for product information based on the acquired identification information;
[f] combining whether the object of the detection result is included in the product information of the search result;
[g] generating UI-processed scene image data in which a link element is attached to the range of the scene image data in which the object of the combination result is displayed;
[h] detecting, by the client device, a call operation by the user for the generated UI-processed scene image data;
[i] By the detection, the UI-processed scene image data is transmitted to the client device, and a selection operation by the user for the link element included in the UI-processed scene image data displayed is detected by the client device and acquiring, from the client device, a link element of the detected selection operation;
[j] a step of retrieving product information corresponding to the acquired link element;
[k] When the searched product information is transmitted to the client device and displayed, an EC processing type selection operation by the user for the product of the displayed product information is detected by the client device, and the EC of the detected selection operation is detected. acquiring the processing type from the client device;
[l] A transmission method including a step of invoking an EC process for the product based on the acquired EC process type. 6. The method of claim 5,
The transmission method, wherein the step [g] further includes a step of saving the UI-processed scene image data displayed on the client device. 6. The method of claim 5,
When steps [b], [c], [d], [e], and [f] are performed multiple times while the video content of step [a] is being displayed on the client device, the step [g] The transmission method further comprising the step of storing the UI-processed scene image data generated in this each time the said execution is carried out. 6. The method of claim 5,
When the UI-processed scene image data transmitted to the client device in step [i] is all the saved UI-processed scene image data, and the UI-processed scene image data is plural, displayed on the client device A transmission method, characterized in that the plurality of UI-processed scene image data is displayed in a thumbnail format. 6. The method of claim 5,
The delivery method, characterized in that the EC processing for the product called based on the EC processing type in the step [l] includes a smart contract between the user and a supply source of the product. It is a method of transmitting EC integrated metamedia including a UI function of EC that allows a user who becomes a viewer of video content to trade the production resource of the video content as a product,
[a] a step of inserting an audio watermark (audio-encoded identification information) into each scene of the video content;
[b] A step in which the video content is broadcast on a general-purpose viewing device;
[c] detecting, by the client device, that the user has selected a scene displayed in the video content;
[d] acquiring, from the client device, identification information of a scene at the time of the selection operation by detection of the selection operation;
[e] a step of retrieving product information based on the acquired identification information;
[f] transmitting the product information to the client device;
[g] displaying the product information on the client device by the transmission;
[h] receiving the EC processing by the user for the product of the product information displayed on the client device;
[i] a step of referring to the EC processing type of the product information upon receipt of the EC processing;
[j] A transmission method including a step of calling a configuration of EC processing corresponding to each EC processing type by the reference. 11. The method of claim 10,
a step of retrieving video data based on the identification information obtained in step [d] and obtaining scene image data corresponding to the identification information from the searched video data;
a step of generating UI-processed scene image data in which a link element is attached to the range of the scene image data in which the product of the product information retrieved in the step [e] is displayed;
a step of replacing the configuration of the step [f] with a configuration of transmitting the UI-processed scene image data to the client device;
a step of replacing the configuration of the step [g] with a configuration of displaying the UI-processed scene image data on the client device;
Before executing the step [h], the client device detects that the user selected the link element included in the UI-processed scene image data displayed on the client device, and retrieves the detected link element from the client device. steps to obtain,
The transmission method further comprising the step of retrieving product information based on the acquired link element, and transmitting product information of the search result to the client device for display.

Images