WO2018036456A1

WO2018036456A1 - Method and device for tracking and recognizing commodity in video image and displaying commodity information

Info

Publication number: WO2018036456A1
Application number: PCT/CN2017/098325
Authority: WO
Inventors: 郑浩; 潘杰; 张怡; 丁航
Original assignee: 大辅科技（北京）有限公司
Priority date: 2016-08-22
Filing date: 2017-08-21
Publication date: 2018-03-01
Also published as: CN107633441A

Abstract

Disclosed in the present invention is a method for tracking and recognizing a commodity in a video image and displaying commodity information. The method comprises: obtain real images; recognizing a feature image in the real images; when the feature image matches a commodity feature in a database, using an area where the feature image in the real images is located, as a commodity image area to load commodity information; and displaying the commodity information. Also disclosed in the present invention is a device for tracking and recognizing a commodity in a video image and displaying commodity information.

Description

Method and apparatus for tracking and identifying merchandise in a video image and displaying merchandise information

Technical field

The present application relates to the field of image recognition, and in particular to a method and apparatus for tracking and identifying products in a video image and displaying product information.

Background technique

With the continuous development of hardware technologies for terminal devices such as personal computers and mobile phones, more and more people choose to use personal devices such as personal computers or mobile phones to watch various television programs provided by video websites. The so-called video website refers to a website that allows Internet users to publish, browse and share video works online with the support of related technology platforms. The well-known video websites include Youku, LeTV, iQiyi, etc.; usually video websites will also be launched. Its own video client application (also known as video client) is specially used to play video works provided by video websites on terminal devices such as mobile phones or personal computers, such as Youku video client and iQiyi video client.

In addition, with the development of panoramic camera equipment, a large number of self-media emerged, and various video works (such as panoramic video, panoramic live broadcast, etc.) were shared on small apps, attracting many fans to participate. Many sponsors (or advertisers) want to place their ads on these wide-ranging videos, but the traditional advertising model is difficult to attract viewers' interest because of the lack of fun and interactivity, resulting in low conversion rates.

Summary of the invention

In view of this, the present invention provides a new mode of loading advertisements on video in order to improve the conversion rate of advertisements.

According to a first aspect of the present invention, there is provided a method of tracking and identifying an item in a video image and displaying the item information, the method comprising:

Obtain a real-life image;

Identifying feature images in a live view image;

When the feature image matches the product feature in the database, the region of the feature image in the live image is loaded with the product information as the product image region;

The product information is displayed.

In some embodiments of the present invention, when the feature image matches the product feature identifier in the database, the live image is virtualized to form a first virtualized image, and the first virtualized image includes the live image a first virtual object corresponding to the medium feature image;

Adding an identifier to a preset location of the first virtual object in the first virtualized image to generate a second virtualized image, wherein the identifier is used to identify the first virtual object.

Preferably, the video is decomposed into a plurality of real-image images according to a frame, and in the plurality of first virtualized images formed by performing virtualization processing on the plurality of real-image images, the identifiers of the first virtual objects corresponding to the same object are consistent. .

Preferably, the first virtualized image is located in a merchandise image area.

Preferably, the second virtualized image is commodity information.

In some embodiments of the invention, the second virtualized image is displayed during video playback. Since the video can be viewed as a combination of multi-frame live images, it can be seen that the second virtualized image follows the positional display of the feature image in the video.

In still other embodiments of the invention, the second virtualized image is displayed while the video is paused. For some video viewers, displaying product information during video playback can interfere with their viewing of the video. In order to avoid causing dissatisfaction of such viewers, the second virtualized image may be set to be transparent when the video is played, and displayed when paused.

According to a second aspect of the present invention, there is provided an apparatus for tracking and identifying an item in a video image and displaying the item information, characterized by comprising

An image recognition unit, configured to acquire a real-life image and identify a feature image in the real-life image;

a display unit for displaying video and product information;

a feature database for storing product feature identifiers and corresponding product information; and

The commodity information loading unit is configured to load the region of the feature image in the live image as the product image region with the corresponding product information.

In some embodiments of the present invention, when the feature image matches the product feature identifier in the feature database, the product information loading unit virtualizes the real-life image acquired by the image recognition unit to form a product image region. a virtualized image, the first virtualized image includes a first virtual object corresponding to the feature image in the live view image; an identifier is added to the preset position of the first virtual object in the first virtualized image, and the generated a second virtualized image of the item information, wherein the identification is for identifying the first virtual object.

Preferably, the image recognition unit decomposes the video into a plurality of real-life images according to the frame, and in the plurality of first virtualized images formed by the virtualizing the plurality of real-image images, the first virtual object corresponding to the same object The logo is consistent.

Preferably, the item information includes one or more of a URL, a product introduction, and a brand introduction.

According to a fourth aspect of the present invention, a client of a panoramic video system includes: a data receiving module and a data display module, wherein: the data receiving module is configured to receive video data and send the data to the data display module, where The video data includes a panoramic video and a corresponding VR image or a specific image, wherein the VR image is accompanied by a hyperlink; the data display module is configured to display the panoramic video and the corresponding VR image or a specific image thereof, and A request signal generated by the user clicking the hyperlink is received and transmitted to the outside.

Preferably, the panoramic video is a spherical panoramic video or a cubic panoramic video, the specific image being displayed at the top or bottom of the spherical panoramic video or the cube panoramic video.

Preferably, the panoramic video is a cylindrical panoramic video, the particular image being displayed above or below the cylindrical panoramic video.

Preferably, the data display module is further configured to jump to display the specific image from the panoramic video.

According to a fifth aspect of the present invention, a server for a panoramic video system includes: a VR generating module, a request processing module, and a data sending module, wherein the VR generating module is configured to generate and send a corresponding VR image based on the panoramic video. a data sending module; the request processing module, configured to receive a request signal from the outside and perform a corresponding operation on the request signal and generate a specific image of the corresponding operation, and send the specific image to the data a sending module; the data sending module, configured to send video data to the outside, wherein the video data includes the panoramic video and the corresponding VR image or the specific image.

According to a sixth aspect of the present invention, a panoramic video system is provided, comprising a client as described above and a server as described above.

Preferably, it is a panoramic video map system, the panoramic video being a street panoramic video.

Preferably, the street panoramic video comprises a street real scene and a street map.

Preferably, the VR image is a VR image of a commodity.

Preferably, the specific image is an advertisement of the specific item or a website of the specific item.

According to a seventh aspect of the present invention, a panoramic video product introduction system is provided, including:

a display unit for displaying video and product information;

a feature database for storing product feature identifiers and corresponding product information;

a virtual image generating unit that generates a virtual image based on the feature image and superimposes on the real image; and

DRAWINGS

The above and other features and advantages of the present invention will become apparent to those skilled in the <RTIgt

1 is a flow diagram of a method of tracking an item in a video image and displaying item information, in accordance with some embodiments of the present invention.

2 is a schematic diagram of an apparatus for tracking an item in a video image and displaying item information, in accordance with some embodiments of the present invention.

3 is a panoramic video system in accordance with an embodiment of the present invention.

4 is a client of a panoramic video system in accordance with a particular embodiment of the present invention.

Figure 5 is a server of a panoramic video system in accordance with an embodiment of the present invention.

detailed description

In the following description, numerous specific details are set forth in the However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, some of the technical features well known in the art have not been described in order to avoid confusion with the present invention.

As shown in FIG. 1, step S110: acquiring a real-life image.

The virtual reality device or the augmented reality device can capture the current scene through its own image acquisition unit to generate a real-life image. In addition, the virtual reality device can also receive real-life images sent by other electronic devices.

Step S120: Identify the feature image in the live view image.

The video is broken down into frames into a continuous still image. Calculate the image for each frame of still image The sharpness of the static image is obtained by the edge point and/or the manner in which the image sharpness is calculated. The image is divided into a plurality of image regions by using an image segmentation technique; for each image region, a feature value for describing the property of the image region is obtained, and it is determined whether the feature value meets a preset feature of the product image region. If yes, it is determined that the image area is a product image area, and the position of the area in the still image is the position of the product contained therein in the still image.

Step S130: Determine whether the feature image matches the product feature in the database.

The image features in the product image area are extracted as reference features; the candidate product features matching the reference features are searched in a previously generated feature database, and the product information corresponding to the candidate product features that are successfully matched is obtained.

Step S140: When the feature image matches the product feature in the database, the region of the feature image in the live image is loaded with the product information as the product image region.

The acquired real-life image is virtualized to form a first virtualized image. The first virtualized image includes a virtual object corresponding to the feature image in the live image.

The virtual reality device virtualizes the acquired real-life image to form a first virtualized image, wherein the first virtualized image includes a virtual object corresponding to all or part of the objects in the real-life image.

A virtual object can present a three-dimensional state, or a planar state. The virtual object in the first virtualized image formed by the virtual reality device is a simulation of the corresponding object in the real image.

The augmented reality device performs virtualization processing on the acquired real-life image, specifically, adding a corresponding virtual object to all or part of the feature image in the real-life image, and the added virtual object constitutes the first virtualized image.

An identifier is added to the preset position of the first virtual object in the first virtualized image to form a second virtualized image. Wherein the identifier is used to identify the first virtual object.

The first virtual object is a virtual object corresponding to a specific image in the live image, and the first virtual object may be one or more. For example, the first virtual object may be a virtual object corresponding to an object in the real image, and may be a virtual object corresponding to a certain type of object in the real image, or may be a virtual corresponding to one or more objects selected by the user. Object.

Adding an identifier to the preset location of the first virtual object, the identifier is used to identify the first virtual object, and the user is convenient to distinguish the first virtual object from the other virtual object, so that the user can track the first virtual object.

Step S150: Display the item information (second virtualized image).

The display unit of the virtual reality device usually displays a closed display effect, that is, the user can hardly see the real environment through the display unit of the virtual reality device, and often only sees the second virtualization displayed by the display unit of the virtual reality device. image.

The display unit of the augmented reality device presents a non-closed display effect, that is, the user can view the real environment through the display unit of the augmented reality device while viewing the image displayed by the augmented reality device through the display unit, and the real scene is The second virtualized image is superimposed and displayed, which can produce a visual effect combining virtual and real.

The electronic device performs virtualization processing on the acquired real-life image to form a first virtualized image, and adds a logo to the preset position of the first virtual object to form a second virtualized image in the first virtualized image, and The second virtualized image is displayed by the display unit. By adding an identifier for identifying the first virtual object in a preset position of the first virtual object in the first virtualized image, the function of the electronic device is enriched, and at the same time, the user can intuitively distinguish the first virtual object from other virtual objects. It is convenient for the user to track the first virtual object and improve the user experience.

In an implementation, adding the identifier to the preset position of the first virtual object in the first virtualized image may be: adding a preset icon to the preset position of the first virtual object in the first virtualized image.

For the user, the first virtual object and other virtual objects can be visually distinguished by the icon added at the preset position of the first virtual object. When the position of the object in the live image changes, the position of the virtual object corresponding to the object is also changed correspondingly in the first virtualized image formed based on the real image, and the icon at the preset position of the virtual object is Visual objects can be visually tracked.

In a specific application, in a case where the number of the first virtual objects in the first virtualized image is plural, in order to further distinguish the plurality of first virtual objects, the method shown in FIG. 1 may be improved. Specifically, in the plurality of first virtualized images formed by performing virtualization processing on the plurality of real-life images, the identifiers of the first virtual objects corresponding to the same object are consistent.

The virtual reality device or the augmented reality device generally acquires the real-life image according to the preset time interval, and virtualizes the acquired real-life image to form a first virtualized image, and displays the first virtualized image through the display unit of the device, thereby The user presents a coherent image.

In the above-disclosed technical solution of the present invention, in the plurality of first virtualized images formed by performing virtualization processing on the plurality of live images, the identifiers added to the first virtual objects corresponding to the same object are consistent. That is to say, in the plurality of second virtualized images formed for the same scene, the identifiers of the virtual objects corresponding to the same object remain unchanged.

In the implementation, the identifiers of the virtual objects corresponding to different objects may be further set to be different. In this case, according to the identifier of each first virtual object, the user can not only distinguish each first virtual object, but also intuitively track the first virtual object according to the identifier of each first virtual object.

The image recognition unit 101 is configured to acquire a real-life image and identify a feature image in the real-life image. The real-life image may be captured by the image acquisition unit of the current scene to generate a real-life image, or may be a captured video image.

Decompose a video (or live image) into frames into a continuous still image. For each frame of still image, the sharpness of the still image is obtained by calculating the image edge points and/or calculating the image sharpness. The image is divided into a plurality of image regions by using an image segmentation technique; for each image region, Obtaining a feature value for describing a property of the image region, determining whether the feature value meets a feature of a preset product image region; if yes, determining that the image region is a product image region, and the region is in the static image The location is the location of the item contained therein in the still image.

The feature database 103 is configured to store the product feature identifier and the corresponding product information. The image recognition unit 101 extracts an image feature in the product image region as a reference feature; searches for a candidate product feature matching the reference feature in the feature database 103 generated in advance, and acquires product information corresponding to the candidate product feature that is successfully matched. .

The product information loading unit 104 is configured to load the region of the feature image in the live image as the product image region with the corresponding product information.

When the feature image matches the product feature identifier in the feature database, the product information loading unit 104 performs virtualization processing on the real image acquired by the image recognition unit, and forms a first virtualized image in the product image region. a virtualized image includes a first virtual object corresponding to the feature image in the live view image; an identifier is added to the preset position of the first virtual object in the first virtualized image, and a second virtualized image including the product information is generated Wherein the identifier is for identifying the first virtual object.

The item information includes one or more of a URL, a product introduction, and a brand introduction.

The image recognition unit 101 decomposes the video into a plurality of real-life images according to a frame. In the plurality of first virtualized images formed by performing virtualization processing on the plurality of real-image images, the identifier of the first virtual object corresponding to the same object remains. Consistent.

The display unit 102 is configured to display video and product information.

A display unit of a virtual reality device (such as a VR helmet, VR glasses, or a VR box) usually displays a closed display effect, which means that it is difficult for a user to see the real environment through the display unit of the virtual reality device, and often only see the virtual environment. The second virtualized image displayed by the display unit of the real device.

The display unit of the augmented reality device (such as a smart phone, a Pad, an AR helmet, etc.) presents a non-closed display effect, that is, the user can view the real environment through the display unit of the augmented reality device while viewing the augmented reality device through the display unit. The displayed image is superimposed and displayed for the user on the real scene and the second virtualized image, and can produce a visual effect combining virtual and real.

The panoramic video system is composed of panoramic video editing and browsing software and panoramic video capture and transmission equipment. It can provide people with rich panoramic images and combine modern network technology. It is an effective fusion of video capture and virtual reality technology. It can be easily used, and the collection device is relatively economical to use, and finally generates a high-information digital media solution, which can provide real-time for people. Video information.

Virtual Reality (VR) technology helps users interact with objects in the real world in a natural way, greatly expanding the ability of humans to understand the world, simulate the world, adapt to the world, and even transform the world.

However, in the current application environment, users often need more information and stronger interaction after focusing on specific targets in the panoramic video.

3 is a panoramic video system in accordance with an embodiment of the present invention. As shown in FIG. 3, a panoramic video system 100 is provided in accordance with an embodiment of the present invention, including a client 110 and a server 120.

4 is a client of a panoramic video system in accordance with an embodiment of the present invention. As shown in FIG. 4, a client 110 of a panoramic video system 100 is provided in accordance with an embodiment of the present invention. The client 110 of the panoramic video system includes a data receiving module 111 and a data display module 112. The data receiving module 111 is configured to receive video data and send the data to the data display module, where the video data includes a panoramic video and a corresponding VR image or a specific image, wherein the VR image is accompanied by a hyperlink; the data display module 112. For displaying the panoramic video and the corresponding VR image or a specific image thereof, and receiving a request signal generated by the user clicking the hyperlink and transmitting the request signal to the outside.

The data receiving module 111 can be implemented in various manners. Specifically, for example, the data receiving module 111 includes three sub-modules, namely, a network receiving feedback sub-module 114, a decoding video synchronization sub-module 115, and a real-time stitching sub-module 116. The network receiving feedback module is responsible for receiving video data transmitted from the server, and the decoding video synchronization module is responsible for converting the video data into multiple segments of video in the form of panoramic video, and synchronizing the video segments; and the real-time stitching module is to segment each segment. The video is stitched in real time under the guidance of certain parameters, and finally a complete panoramic video is synthesized. Alternatively, the data receiving module 111 directly receives the panoramic video from the outside, that is, directly transmits the panoramic video to the data display module 112 without performing any data processing.

At this point, the data display module 112 provides the user with a three-dimensional viewing interface to present the panoramic video to the people. At the same time, the data display module also displays the VR image generated according to the panoramic video. In addition to the real scene, the user can also see the VR image, and the VR image is accompanied by a hyperlink, that is, the user can click on the VR image to enable the panoramic video system to execute and super Corresponding specific operations are linked, and in this particular embodiment, for example, the data display module displays a particular image. In the process, the data display module is further configured to receive a request signal generated by the user clicking the VR image, and send the request signal to the outside.

Based on the client of the panoramic video system according to a specific embodiment of the present invention, the user can pass the point Clicking on the VR image for further content or interacting with the client in other forms enhances the richness and interactivity of the panoramic video.

There are three main display modes for panoramic video, cylindrical panoramic video, spherical panoramic video and cube panoramic video. Among the three display models, cylindrical panoramic video best meets the needs of VR images and roaming systems, and is relatively easy to create and convenient to use. In the process of generating cylindrical panoramic video, the design method of the panoramic photo is mainly used.

Thus, based on a client of a panoramic video system in accordance with an aspect of a particular embodiment of the present invention, wherein the panoramic video is a cylindrical panoramic video, the particular image is displayed above or below the cylindrical panoramic video. When a particular image is above or below the cylindrical panoramic video, the particular image does not affect the user's main view of the panoramic video.

Based on a client of a panoramic video system in accordance with another aspect of a particular embodiment of the present invention, the panoramic video is a spherical panoramic video or a cube panoramic video, the particular image being displayed at the top or bottom of the spherical panoramic video or cube panoramic video. When a particular image is at the top or bottom of a spherical panoramic video or cube panoramic video, the particular image does not affect the user's primary view of the panoramic video.

Based on a client of a panoramic video system in accordance with another aspect of the present invention, the data display module is further configured to jump from the panoramic video to display the particular image. When the panoramic video jumps to a specific image, the user can view the specific image more conveniently, and it is also easier to present a specific image more abundantly.

5 is a server of a panoramic video system in accordance with an embodiment of the present invention. As shown in FIG. 3, a server 120 of a panoramic video system 100 is provided in accordance with an embodiment of the present invention. A server 120 of a panoramic video system includes a VR generating module 121, a request processing module 122, and a data sending module 123. The VR generating module 121 is configured to generate a corresponding VR image based on the panoramic video and send it to the data sending module 123. The request processing module 122 is configured to receive a request signal from the outside and perform a corresponding operation on the request signal and generate a specific image of the corresponding operation, and the specific image is sent to the data sending module 123; the data sending module 123 is configured to send video data to the outside, wherein the video data includes the panoramic video and the The corresponding VR image or the specific image is described.

It can be seen that the server 120 of the panoramic video system according to the embodiment of the present invention receives the request signal from the external data, such as the data display module 112 of the client 110 of the panoramic video system, by requesting the processing module 122 and executes the request signal for the request signal. Correspondingly operating and generating a specific image of the corresponding operation, and transmitting the specific image to the data sending module 123; while the server 120 of the panoramic video system according to the specific embodiment of the present invention is based on the panoramic video by the VR generating module 121 Generate their corresponding The VR image is sent to the data sending module 123; the data sending module 123 sends video data to the external data receiving module 111 of the client 110, for example, the panoramic video system, wherein the video data includes the panoramic video and the Its corresponding VR image or the particular image.

The data sending module 123 can be implemented in various manners. Specifically, for example, the data sending module 123 includes three sub-modules, namely, a network receiving feedback sub-module, a decoding video synchronization sub-module, and a real-time stitching sub-module. The network receiving feedback module is responsible for receiving video data transmitted from the outside, and the decoding video synchronization module is responsible for converting the video data into multiple pieces of video in the form of panoramic video, and synchronizing each piece of video; and the real-time stitching module is to segment each segment. The video is stitched in real time under the guidance of certain parameters, and finally a complete panoramic video is synthesized. Alternatively, the data transmitting module 123 directly transmits the video data to the outside, that is, directly transmits the video data to the data receiving module 111 without performing any data processing.

As described above, FIG. 3 is a panoramic video system according to a specific embodiment of the present invention. As shown in FIG. 3, a panoramic video system 100 including a client 110 and a server 120 is provided according to an embodiment of the present invention. .

In particular, a panoramic video system in accordance with a particular embodiment of the present invention is a panoramic video map system such that the panoramic video is a street panoramic video. Specifically, the street panoramic video includes a street real scene and a street map. The existing electronic map only includes panoramic photos, and the invention is combined with an electronic map to provide a panoramic video for the user, thereby providing more accurate information.

Specifically, the VR image is a VR image of a commodity. The VR image of the product may be a VR image of the product or a VR image of the merchant. in particular,

Specifically, the specific image in the client 110 or the server 120 of a panoramic video system according to a specific embodiment of the present invention is an advertisement of the specific item or a website of the specific item.

Example 1

Panoramic video map system

A VR-based panoramic video map or an AR-based panoramic video map can be developed.

Taking a panoramic video map of VR as an example, a panoramic video in a predetermined area is pre-photographed. Set some of the scenes as feature images to match the product features in the database. For example, the bakery matches the products sold in the bakery; the sports store matches the products sold in the store; the museum matches the information in the museum.

The first virtualized image is set as needed, for example, a product image is displayed. Considering the user experience, the content of the first virtualized image is as concise as possible. It is also possible to select a push image based on the result of the big data calculation.

When the user clicks on the scene set as the feature image through the handle or the line of sight, the first virtualized image is presented.

In a preferred embodiment, if the user pauses the video, when the video is paused, the second virtualized image is played, and the second virtualized image is the product information.

Example 2

Video map system

Taking the video map of the AR as an example, the scene in the predetermined area is set as a feature image to match the product features in the database. For example, the bakery matches the products sold in the bakery; the sports store matches the products sold in the store; the museum matches the information in the museum.

In the real scene, the scene set as the feature image is recognized in real time, and the first virtualized image is presented after the scene appears in the line of sight for more than a preset time (for example, 2 seconds).

In a preferred embodiment, if the user clicks or directly clicks the first virtualized image on the touch screen through the handle or the line of sight, the second virtualized image is played, and the second virtualized image is the commodity information.

The embodiment of the present invention is not limited to a panoramic video map, and may be any video product or a system using video. When the product information needs to be added or the interaction is improved, the technical solution described in the present invention may be adopted. The presentation content of the first virtualized image and the second virtualized image may also be designed according to customer needs, and is not limited to being exemplified in the embodiments of the present specification.

It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should understand that the present invention is not limited by the described action sequence. Because certain steps may be performed in other sequences or concurrently in accordance with the present invention. In addition, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

In the above embodiments, the descriptions of the various embodiments are different, and the details that are not detailed in a certain embodiment can be referred to the related descriptions of other embodiments. In the several embodiments provided herein, it should be understood that the disclosed apparatus may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be electrical or otherwise.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

It should be pointed out that the various steps/components described in the present application can be split into more steps/components according to the needs of the implementation, and two or more steps/components or partial operations of the steps/components can be combined into new ones. Steps/components to achieve the objectives of the present invention.

The above method according to the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or can be downloaded through a network. The computer code originally stored in a remote recording medium or non-transitory machine readable medium and to be stored in a local recording medium, whereby the methods described herein can be stored using a general purpose computer, a dedicated processor, or programmable or dedicated Such software processing on a recording medium of hardware such as an ASIC or an FPGA. It will be understood that a computer, processor, microprocessor controller or programmable hardware includes storage components (eg, RAM, ROM, flash memory, etc.) that can store or receive software or computer code, when the software or computer code is The processing methods described herein are implemented when the processor or hardware is accessed and executed. Moreover, when a general purpose computer accesses code for implementing the processing shown herein, the execution of the code converts the general purpose computer into a special purpose computer for performing the processing shown herein.

The present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit and scope of the invention. The present invention has been described by the above-described embodiments, but it should be understood that the above-described embodiments are only for the purpose of illustration and description. Further, those skilled in the art can understand that the present invention is not limited to the above embodiments, and various modifications and changes can be made according to the teachings of the present invention. These modifications and modifications fall within the scope of the claimed invention. Within. The scope of the invention is defined by the appended claims and their equivalents.

Claims

A method of tracking and identifying merchandise in a video image and displaying merchandise information, the method comprising:

Obtain a real-life image;

Identifying feature images in a live view image;

When the feature image matches the product feature in the database, the region of the feature image in the live image is loaded with the product information as the product image region;

The product information is displayed.
The method according to claim 1, wherein when the feature image matches the product feature identifier in the feature database, the live image is virtualized to form a first virtualized image, and the first virtualized image includes a first virtual object corresponding to the feature image in the real-life image;

Adding an identifier to a preset location of the first virtual object in the first virtualized image to generate a second virtualized image, wherein the identifier is used to identify the first virtual object.
The method according to claim 2, wherein the video is decomposed into a plurality of real-life images according to a frame, and the plurality of first virtualized images formed by virtualizing the plurality of real-image images are corresponding to the same object. The identifiers of the first virtual object remain consistent.
The method of claim 2 or 3 wherein the first virtualized image is located in a merchandise image area.
The method of claim 4 wherein said second virtualized image is item information.
The method of claim 2 or 3, wherein the second virtualized image is displayed during video playback or when the video is paused.
A device for tracking and identifying items in a video image and displaying product information, including

An image recognition unit, configured to acquire a real-life image and identify a feature image in the real-life image;

a display unit for displaying video and product information;

a feature database for storing product feature identifiers and corresponding product information; and

The commodity information loading unit is configured to load the region of the feature image in the live image as the product image region with the corresponding product information.
The apparatus according to claim 7, wherein when the feature image matches the product feature identifier in the feature database, the real information map acquired by the article information loading unit on the image recognition unit Forming a first virtualized image in the product image area, the first virtualized image includes a first virtual object corresponding to the feature image in the real-life image; and in the first virtualized image Adding an identifier to a preset location of a virtual object, generating a second virtualized image containing merchandise information, wherein the identifier is used to identify the first virtual object.
The apparatus of claim 7, wherein the item information comprises one or more of a URL, a product introduction, and a brand introduction.
The device according to claim 8, wherein the video is decomposed into a plurality of real-life images according to a frame, and the plurality of first virtualized images formed by performing virtualization processing on the plurality of real-image images correspond to the same object. The identifiers of the first virtual object remain consistent.
A client for a panoramic video system, comprising: a data receiving module and a data display module, wherein:

The data receiving module is configured to receive video data and send the data to the data display module, where the video data includes a panoramic video and a corresponding VR image or a specific image, wherein the VR image is accompanied by a hyperlink;

The data display module is configured to display the panoramic video and the corresponding VR image or a specific image, and receive a request signal generated by the user clicking the hyperlink and send the request signal to the outside.
The client of claim 11 wherein the panoramic video is a spherical panoramic video or a cube panoramic video, the particular image being displayed at the top or bottom of the spherical panoramic video or the cube panoramic video.
The client of claim 11 wherein said panoramic video is a cylindrical panoramic video, said particular image being displayed above or below said cylindrical panoramic video.
The client according to claim 11, wherein the data display module is further configured to jump to display the specific image from the panoramic video.
A server for a panoramic video system, comprising: a VR generating module, a request processing module, and a data sending module, wherein

The VR generating module is configured to generate a corresponding VR image based on the panoramic video and send the data to the data sending module;

The request processing module is configured to receive a request signal from the outside and perform a corresponding operation on the request signal and generate a specific image of the corresponding operation, and send the specific image to the data sending module;

The data sending module is configured to send video data to the outside, wherein the video data includes the panoramic video and the corresponding VR image or the specific image.
A panoramic video system comprising the client of claim 11 and the server of claim 15.
A panoramic video system according to claim 16 which is a panoramic video map system, said panoramic video being a street panoramic video.
The panoramic video system of claim 17 wherein said street panoramic video comprises street real and street maps.
The panoramic video system of claim 16 wherein said VR image is a VR image of an item.
A panoramic video system according to claim 18, wherein said specific image is an advertisement of said specific item or a website of said specific item.