CN111263097A

CN111263097A - Media data transmission method and related equipment

Info

Publication number: CN111263097A
Application number: CN202010051951.7A
Authority: CN
Inventors: 刘俊; 杨胜凯
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-09-19
Filing date: 2020-01-16
Publication date: 2020-06-09
Anticipated expiration: 2040-01-16
Also published as: WO2021051912A1; CN111263097B

Abstract

The application discloses a media data transmission method and related equipment, wherein the media data transmission method comprises the following steps: a camera generates a plurality of original video frames; a camera generates a video using a plurality of original video frames; the method comprises the steps that a camera acquires position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video; the camera sends the video and the position information to the storage device, so that the position information of the video and the target image can be transmitted, and the consumption of transmission bandwidth is effectively reduced.

Description

Media data transmission method and related equipment

Technical Field

The present application relates to the field of data processing, and in particular, to a media data transmission method and related device.

Background

The camera captures a large amount of video data and then transfers the video to a storage device for storage.

With the push of camera intellectualization, the camera needs to transmit pictures to the storage device in addition to videos, so that the storage device can perform operations such as image recognition. The additional picture transfers increase the network bandwidth usage and the storage device space usage.

Disclosure of Invention

The embodiment of the application provides a media data transmission method and related equipment, which can be used for transmitting position information of videos and images and effectively reducing the consumption of transmission bandwidth.

In a first aspect, an embodiment of the present application provides a media data transmission method, where the method includes:

a camera generates a plurality of original video frames;

a camera generates a video using a plurality of original video frames;

the method comprises the steps that a camera acquires position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

the camera sends the video and the position information to the storage device.

In this example, by transmitting the position information of the video and the target image, since the size of the position information is much smaller than that of the image, the transmission bandwidth occupied by the position information is also much smaller than that occupied by the image, and thus the consumption of bandwidth resources is reduced.

In combination with the first aspect, in one possible embodiment of the first aspect,

in the case where the category of the target image is a large map, the position information includes a first absolute position and/or a first relative position, where the first absolute position includes one or more of a frame number and a timestamp of the target image in the video, and the first relative position includes an offset of the target image with respect to a particular video frame.

in the case that the category of the target image is a small image, the position information includes a second absolute position and/or a second relative position, wherein the second absolute position includes one or more of a frame number and a timestamp of a video frame corresponding to the target image in the video, and the second absolute position also includes a position of the target image in the corresponding video frame; the second relative position includes an offset of the target image from the particular video frame, the position of the target image in the corresponding video frame.

In this example, the position information includes an absolute position and a relative position, and the absolute position and the relative position can be transmitted at the same time to check each other, so as to avoid errors caused by loss of the video frame in the transmission process.

With reference to the first aspect, in a possible embodiment of the first aspect, the camera does not generate the target image and does not send the target image to the storage device.

In the example, the camera does not generate the target image and does not send the target image to the storage device, so that the target image does not need to be transmitted separately, and the consumption of bandwidth resources is reduced.

With reference to the first aspect, in a possible embodiment of the first aspect, the method further includes:

the camera selects a target frame where a target image is located from the video;

and the camera acquires the position information of the target image in the video according to the target frame and the video.

the target picture is located in an I-frame in a group of pictures GOP of the video.

the camera acquires a target image from the plurality of original video frames, and the image quality of the target image is an image including target features in the plurality of original video frames.

With reference to the first aspect, in a possible embodiment of the first aspect, the method further includes: the storage equipment receives the video sent by the camera and the position information of the target image;

and the storage equipment acquires the target image from the corresponding video frame of the video according to the position information.

the storage equipment stores the video sent by the camera and the position information of the target image;

when the storage life cycle of the video is finished, the storage equipment acquires a target image from a corresponding video frame of the video according to the position information;

the storage device stores the target image;

the storage device deletes the video.

In this example, when the storage life cycle of the video is finished, the target image is obtained from the corresponding video frame of the video according to the position information and is stored, so that the target image can be stored after the subsequent video is deleted, and the target image can be provided when the subsequent image is searched.

the storage device receives a video stream sent by a camera and position information of a target image, wherein the target image is a video frame in the video stream or a part of the video frame in the video stream;

and the storage equipment acquires the target image from the corresponding video frame of the video stream according to the position information.

In a second aspect, an embodiment of the present application provides a media data transmission method, where the method includes:

the storage device receives a video sent by a camera and position information of a target image, wherein the target image is a video frame in the video or a part of the video frame in the video;

In combination with the second aspect, in one possible embodiment of the second aspect,

With reference to the second aspect, in a possible embodiment of the second aspect, during a storage life cycle of the video, when the storage device receives a request for reading a target image, the storage device obtains the target image from a corresponding video frame in the video according to the location information.

With reference to the second aspect, in one possible embodiment of the second aspect, the method further includes: when the storage life cycle of the video is finished, the storage equipment acquires a target image from a corresponding video frame of the video according to the position information; the storage device saves the target image and deletes the video.

In this example, in the storage life cycle of the video, the target image is obtained from the corresponding video frame in the video, so that in the storage life cycle of the video, the target image does not need to be stored, the storage device only needs to store the video and the position information, and the memory space occupied by the position information is far smaller than the memory space occupied by the target image, so that the storage position information can reduce the consumption of storage resources relative to the storage of the target image.

In a possible embodiment of the second aspect in combination with the second aspect, the storage device does not store the target image during a storage lifetime of the video; after the storage life cycle of the video is finished, the storage device stores the target image.

In this example, the target image does not need to be stored during the storage life cycle of the video, and the consumption of storage resources can be reduced.

In a third aspect, an embodiment of the present application provides a media data transmission apparatus, including:

a first generating unit for generating a plurality of original video frames;

a second generation unit for generating a video using the plurality of original video frames;

an acquisition unit configured to acquire position information of a target image in a video, wherein the target image is a video frame in the video or a portion of the video frame in the video;

and the sending unit is used for sending the video and the position information to the storage equipment.

In combination with the third aspect, in one possible embodiment of the third aspect,

With reference to the third aspect, in a possible embodiment of the third aspect, the media data transmission apparatus does not generate the target image, and does not send the target image to the storage device.

With reference to the third aspect, in a possible embodiment of the third aspect, the method further includes:

With reference to the third aspect, in a possible embodiment of the third aspect, the method is further configured to:

and acquiring a target image from the plurality of original video frames, wherein the image quality of the target image is the image of the plurality of original video frames including the target characteristic.

In a fourth aspect, an embodiment of the present application provides a media data transmission apparatus, including:

the device comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving a video sent by a camera and position information of a target image, and the target image is a video frame in the video or a part of the video frame in the video;

and the acquisition unit is used for acquiring the target image from the corresponding video frame of the video according to the position information.

In combination with the fourth aspect, in one possible embodiment of the fourth aspect,

With reference to the fourth aspect, in a possible embodiment of the fourth aspect, during a storage life cycle of the video, when the storage device receives a request for reading a target image, the storage device obtains the target image from a corresponding video frame in the video according to the location information.

In combination with the fourth aspect, in a possible embodiment of the fourth aspect, the method is further configured to: when the storage life cycle of the video is finished, the storage equipment acquires a target image from a corresponding video frame of the video according to the position information; the storage device saves the target image and deletes the video.

In combination with the fourth aspect, in a possible embodiment of the fourth aspect, the method is further configured to:

receiving a video stream sent by a camera and position information of a target image, wherein the target image is a video frame in the video stream or a part of the video frame in the video stream;

and acquiring a target image from a corresponding video frame of the video stream according to the position information.

In a fifth aspect, an embodiment of the present application provides a camera, including:

a processor for generating a plurality of original video frames, generating a video using the plurality of original video frames; acquiring position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

and the transceiving module is used for sending the video and the position information to the storage device.

In combination with the fifth aspect, in one possible embodiment of the fifth aspect,

With reference to the fifth aspect, in one possible embodiment of the fifth aspect, the camera does not generate the target image, and the transceiver module does not transmit the target image to the storage device.

With reference to the fifth aspect, in one possible embodiment of the fifth aspect, the method is further configured to:

selecting a target frame where a target image is located from a video;

and acquiring the position information of the target image in the video according to the target frame and the video.

In a sixth aspect, an embodiment of the present application provides a storage device, including:

the receiving and sending module is used for receiving the video sent by the camera and the position information of a target image, wherein the target image is a video frame in the video or a part of the video frame in the video;

and the processor is used for acquiring the target image from the corresponding video frame of the video according to the position information.

In combination with the sixth aspect, in one possible embodiment of the sixth aspect,

With reference to the sixth aspect, in a possible embodiment of the sixth aspect, during a storage life cycle of the video, when the storage device receives a request for reading a target image, the storage device obtains the target image from a corresponding video frame in the video according to the location information.

In combination with the sixth aspect, in one possible embodiment of the sixth aspect, the method is further configured to: when the storage life cycle of the video is finished, acquiring a target image from a corresponding video frame of the video according to the position information; save the target image, and delete the video.

In combination with the sixth aspect, in one possible embodiment of the sixth aspect, the method is further configured to:

In a seventh aspect, an embodiment of the present application provides a camera, including: a processor, a transceiver, and a memory, the processor executing code in the memory to perform a method as in the first aspect.

In an eighth aspect, an embodiment of the present application provides a storage device, where the storage device includes: a processor, a transceiver, and a memory, the processor executing code in the memory to perform a method as in the second aspect.

In a ninth aspect, embodiments of the present application provide a computer-readable storage medium, in which a computer program is stored, the computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method according to any one of the first and second aspects.

A tenth aspect provides a computer program product for performing a method according to any one of the first and second aspects when the computer program product is read and executed by a computer.

These and other aspects of the present application will be more readily apparent from the following description of the embodiments.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application.

Fig. 1 is a schematic diagram of a camera for capturing images according to an embodiment of the present application;

FIG. 2 provides a schematic illustration of a large graph and a small graph for an embodiment of the present application;

FIG. 3A is a diagram illustrating a video frame transmission according to an embodiment of the present application;

FIG. 3B is a schematic diagram of a camera transmitting video and position information according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a storage device storing video and location information according to an embodiment of the present application;

FIG. 5A is a diagram illustrating an index table of a large graph according to an embodiment of the present application;

FIG. 5B is a diagram illustrating an index table of a thumbnail image according to an embodiment of the present application;

FIG. 6A is a schematic diagram of a camera extracting a target image according to an embodiment of the present disclosure;

FIG. 6B is a diagram illustrating an index table stored in a thumbnail image reuse manner according to an embodiment of the present application;

FIG. 6C is a diagram of an index table stored in a thumbnail reuse storage according to an embodiment of the present application;

FIG. 6D is a diagram illustrating an index table of another large graph according to an embodiment of the present disclosure;

fig. 7 is an interaction diagram of a media data transmission method according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a media data transmission apparatus according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a camera according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of another camera provided in the embodiment of the present application;

fig. 11 is a schematic structural diagram of a media data transmission apparatus according to an embodiment of the present application;

FIG. 12 is a schematic structural diagram of a storage device according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below with reference to the drawings.

The video transmission process to which the present application relates will be described first.

As shown in FIG. 1, the camera will capture a video according to a time axis t, wherein the video comprises n frames of images I₁，I₂，…，I_nImage I₁Is the camera at t₁Image acquired at any moment, image I₂Is the camera at t₂Image acquired at time …, image I_nIs the camera at t_nImages acquired at a time. Here, the first and second liquid crystal display panels are,t₁，t₂，…，t_nthe time intervals between may or may not be equal, i.e. t_n-t_n-1，t_n-1-t_n-2，…，t₂-t₁May or may not be equal, and are not particularly limited herein.

In addition to sending the video to the storage device, the camera also needs to select a target image from the video to send to the storage device. The target image may be a large image or a small image, and the storage device may be a storage server, a video stream management platform, or the like. As shown in fig. 2, the large map may be a complete image of a certain video frame, or an image occupying an area of a certain video frame exceeding a preset threshold, or the like. In a particular embodiment, the large map may contain a plurality of target subjects (subjects may be understood as target features), for example, the large map may include a scene where a vehicle knocks over a pedestrian, and thus the large map may be used to analyze the relationship and behavior between different target subjects. A thumbnail may be a partial region of a certain video frame. In a particular embodiment, the thumbnail may include only a single target subject, or only a partial region of a single target subject, for example, the thumbnail may include a face portion of a pedestrian, and thus, the thumbnail may be used to analyze the details and structure of a single target subject. Here, the target subject may be a pedestrian, an animal, a vehicle, a license plate, a road sign, a traffic light, etc., and is not particularly limited herein. The small image may be obtained by extracting a region image block having a target subject from the large image, and the extraction method may be an image feature extraction algorithm, specifically, may be HOG (histogram of Oriented Gradient), SIFT (Scale-invariant features transform), and the like, and is not specifically limited herein.

In order to reduce the bandwidth occupied by the camera for transmitting the video and the target image to the storage device, the camera may further compress the video using a video compression algorithm and compress the target image using a picture compression algorithm, where the video compression algorithm may be h.264, h.265, h.266, and the like, and is not limited herein. The picture compression algorithm may be JPEG, HEIF, etc., and is not particularly limited herein.

When the bandwidth occupied by the camera for sending data to the storage device is reduced, the camera compresses the video by adopting a video compression algorithm and compresses the target image by adopting a picture compression algorithm, and the occupied bandwidth is reduced after compression. However, transmission resources are still consumed by bandwidth during transmission, especially when a large number of target images are transmitted, the consumption of bandwidth resources is still huge, and after the target images are transmitted to the storage device, the storage device needs to consume memory resources to store the target images, which results in large consumption of storage resources. Therefore, how to reduce the transmission of the target image is a problem to be solved.

The embodiment of the application aims to solve the problems of large bandwidth consumption when a target image is transmitted and large consumption of memory resources and hard disk resources when storage equipment stores the target image, the video (the video can be transmitted in a streaming mode and is called as video stream) and the position information of the target image are transmitted in a transmission mode, and the size of the position information is far smaller than that of the image, so that the bandwidth occupied by the position information is far smaller than that of the image, the consumption of bandwidth resources is reduced, after the storage equipment receives the position information of the video and the target image, the storage equipment stores the video and the position information, and the storage position information can also reduce the consumption of the storage resources relative to the storage of the target image.

The application provides a media data transmission method and related equipment, which can effectively reduce consumption of bandwidth resources and storage resources.

Sending video, including image I, to a storage device at a camera₁，I₂，…，I_nIn addition to transmitting the video, the camera also needs to transmit position information of the target image, which is used for marking the position of the target image in the video. Here, the video and the position information may be transmitted simultaneously or may be transmitted separately; can be transmitted by the same channel or by different channels (e.g. all using a data channel, or a video stream using dataA channel, a location information use management channel), which is not particularly limited herein. The following describes in detail the position information of the target image in the case of a large image and a small image, respectively.

When the category of the target image is a large image, the position information of the target image may be a first absolute position or a first relative position, or may include both the first absolute position and the first relative position. Wherein the first absolute position may include one or more of a frame number and a time stamp of the target image in the video, and the like. For example, the video comprises n frames of images I₁，I₂，…，I_nThe target image may be a 5 th frame image, and the first absolute position may be a frame number 5 of the target image in the video. The first relative position may be an offset with respect to a particular video frame, etc. For example, the video comprises n frames of images I₁，I₂，…，I_nThe target image may be a 5 th frame image. If the first relative position is 4, the offset of the target image from the first frame image may be 4, or if the specific video frame is, for example, an I frame obtained by compressing a video frame, the offset may be an offset of the target image from the I frame, or the like. Of course, when the video frame corresponding to the target image is an I frame, the position information may be described by using a first absolute position, specifically, the position information may be a frame number of the I frame, and the like, and when the target image frame is a non-I frame, the position information may be described by using a first relative position, specifically, an offset of the target image with respect to the I frame, and the like. Supplementary explanation: when the large map is not a complete frame image, the position information of the target image may further include a position of the target image in the video frame, the position including coordinates and a size (see the following description in detail).

When the category of the target image is the thumbnail, the position information of the target image may be the second absolute position or the second relative position, or include both the second absolute position and the second relative position. The second absolute position may include one or more of a frame number, a timestamp, and the like of a video frame corresponding to the target image in the video, and a position of the target image in the video frame, where the position includes a coordinate and a size, the coordinate of the target image in the video frame may be represented as (x, y), x is a horizontal coordinate, y is a vertical coordinate, the size of the target image in the video frame may be represented as mXn, m is a horizontal size, and n is a vertical size. The second relative position comprises the absolute position of the video frame corresponding to the target image and the relative position of the target image in the corresponding video frame, the relative position of the video frame corresponding to the target image and the absolute position of the target image in the corresponding video frame, and the relative position of the video frame corresponding to the target image and the relative position of the target image in the corresponding video frame. The relative position of the target image in the corresponding video frame may be an offset from a position of a specific identifier, for example, the video frame is an image of a target pedestrian visiting the Tiananmen, and the target image is a target pedestrian, then the relative position of the target image in the video frame may be the direction and distance of the target pedestrian relative to the Tiananmen, for example, the target pedestrian is at the east side of the Tiananmen, and the position of the target pedestrian is 100 meters away, which is denoted as (east, 100).

The position information of the target image can also comprise the frame type of the video frame in which the target image is positioned when the video frame is transmitted after being compressed, wherein the frame type comprises an I frame and a P frame. Sending video to storage means at camera, e.g. including image I₁，I₂，…，I_nAs shown in fig. 3A, fig. 3A shows a schematic diagram of the transmission of video frames. After the video frame is encoded, an I frame and a P frame are obtained, the I frame may be a complete video frame, the I frame may be understood as a key frame, and the P frame is a difference between the I frame and a previous key frame. The target image may be an I frame or a P frame.

When the camera sends the video and the position information to the storage device, the camera can also send the associated information of the target image, wherein the associated information comprises the acquisition time of the video, the acquisition time of a video frame corresponding to the target image, the camera identification, the category of the target image, the sequence number of the target image, the frame number of the video frame corresponding to the target image and the offset of the video frame corresponding to the target image. The category of the target image includes a large image and a small image, and the time for acquiring the video may be the starting time of the video, and the like.

It can be understood that in practical applications, the absolute position and the relative position can be transmitted simultaneously to check each other, so as to avoid errors caused by loss of video frames during transmission.

In a specific embodiment, the position information transmitted between the camera and the storage device is compressed, as shown in FIG. 3B. Specifically, the camera may compress the original location information and send the compressed location information to the storage device. The original position information may be part or all of the position information described above. When the original position information is partial position information in the position information, the original position information at least includes position information of the target image and the like.

As shown in fig. 4, fig. 4 shows a specific example of when the storage device stores the received video and the location information, and after receiving the video and the location information, the storage device stores the video in a storage space corresponding to the video, and stores the location information. For storage convenience, the position information may be further processed to obtain an index table of the target image, and the index table may be stored. The position information (or the index table) may represent the position information of the target picture in the video, the index table includes a large-graph index table and a small-graph index table, and the large-graph index table and the small-graph index table may be stored separately, that is, all the large-graph index tables are stored in one memory space, and all the small-graph index tables are stored in another memory space.

When processing the position information, the index table for obtaining the target image may specifically be: and extracting the position information of the target image, generating a template according to a preset index table, and generating the index table of the target image. When extracting the position information of the target image, the position information may be extracted from the cache or may be extracted from the memory. The preset index table generation template may be a preset template. Of course, the index table of the target image may be acquired in other manners.

In a specific embodiment, when the index table is generated according to the index table template, fig. 5A shows a schematic diagram of an index table of a large graph, and fig. 5B shows a schematic diagram of an index table of a small graph. As shown in fig. 5A, the index table of the large graph includes a camera identifier, a capturing time of a video frame corresponding to the large graph, a frame number of the video frame, a picture type, a picture sequence number, a video frame type, a video frame offset, and the like, and the content of the index table may be directly extracted from the received position information. As shown in fig. 5B, the index table of the thumbnail includes the camera identifier, the capture time of the video frame where the thumbnail is located, the frame number of the video frame, the picture type, the picture sequence number, the video frame type, the video frame offset, the offset of the thumbnail in the video frame, the size of the thumbnail in the video frame, and the like. The offset of the thumbnail in the video frame is represented by coordinates, the size of the thumbnail in the video frame is represented by a horizontal size and a vertical size, for example, 80X80, the horizontal size is 80, and the vertical size is 80, and the contents of the index table can be directly extracted from the received position information.

The storage device may not store the target image separately. And after the storage equipment receives a request for reading the target image from the host, acquiring the corresponding target image from the video by using the index table, and sending the target image to the host.

After the storage device stores the video, a storage life cycle is set for the video, after the storage life cycle is finished, the video is deleted, and the storage life cycle can be specifically understood as a fixed duration. In order to avoid that the target image cannot be read any more after the video is deleted, the storage device may extract the target image from the video at the end of the storage life cycle of the video, store the target image in the corresponding storage space, and update the picture index table (the updated index table is used to describe the storage location of the target image in the storage device). The moment when the storage life cycle ends is a condition for triggering the step of extracting the target image from the video, and after the extraction of the target image is completed, the storage device deletes the video. At a specific point in time, the end of the life cycle includes: the life cycle will reach the end point in time or shortly after the end point in time of the life cycle. Of course, before the storage life cycle of the video is finished, the storage device may extract the target image from the video, such as: the extraction operation is completed within 10 minutes before the end of the storage life cycle and the storage is performed, and the storage device can delete the video immediately after the storage life cycle is ended.

When the target image is extracted from the video according to the index table of the target image, the following steps may be specifically performed: the method comprises the steps of obtaining at least one video corresponding to a camera identifier according to the camera identifier, then determining a target video from the at least one video according to the video acquisition time, wherein the target video comprises a target image, extracting a video frame from the target video according to the time of the video frame in an index table, the type of the video frame, the frame number of the video frame in the video and the video frame offset, obtaining the target image from the video frame according to the position information of the target image, determining the video frame as the target image if the target image is a large image (taking the large image as an example for explanation), and obtaining the target image according to the offset of the picture in the video frame in the index table and the size of the picture in the video frame if the target image is a small image. In a specific example, as shown in fig. 6A, fig. 6A shows a schematic diagram of extracting a target image. The storage device extracts n videos, namely video 1, video 2, …, video n-1 and video n, from a storage space corresponding to the videos according to the camera identification, determines m videos, namely video k, … and video j from the n videos according to the camera identification, determines a target video from the m videos according to the acquisition time of the videos, and determines a target image from n video frames of the target video according to other information in an index table, wherein the other information comprises the time of the video frame, the type of the video frame, the frame number of the video frame, the offset of the video frame, the position information (a first absolute position and/or a first relative position, a second absolute position and/or a second relative position) of the target image and the like.

When the target image is extracted from the video, whether the target image exists in the video is judged firstly, and if the target image exists, the target image is extracted from the video according to the index table of the target image and is stored in the image storage space. Judging whether a target image exists in the video or not, and judging according to the index table, specifically, judging whether the video frame exists in the video or not according to the frame number of the video frame in the index table and the acquisition time of the video frame, if so, determining that the target image exists in the video, and if not, determining that the target image does not exist in the video.

When the target image is stored, the large image and the small image can be stored separately or in a multiplexing manner. Multiplexing storage can be understood as that when the target image is a small image, the large image or the video frame where the target image is located is stored, and the position information of the small image in the large image or the video frame is stored, so that the effect that both the small image and the large image/video frame are stored is realized. When the target image is stored, the target image may be encoded by using different encoding formats and then stored, and the encoding format may be an HEIF format.

After deleting the video, since the specific indexed position in the index table of the current target image is the position in the video, the current index table cannot satisfy the condition for indicating the position of the target image, and therefore the index table needs to be updated, and the updated index table is used for describing the storage position of the target image, the picture type of the target image, the encoding format of the target image during storage, and the like.

In different storage modes, the updated picture index table is also different, and the updated picture index table adds the storage location of the target image and the file name of the target image, and deletes the original location information related to the video, such as the type of the video frame, the offset of the video frame, and the like. As shown in fig. 6B, fig. 6B is a schematic diagram of an index table of a thumbnail when the thumbnail is stored in a multiplexing manner, and in this case, a video frame corresponding to the thumbnail is multiplexed when the thumbnail is stored. Fig. 6C shows that the small graphs are stored in a multiplexing manner, and at this time, the large graphs corresponding to the small graphs are multiplexed when the small graphs are stored, if the large graphs are complete video frames, the offsets of the small graphs in the large graphs do not need to be obtained again, and if the large graphs are partial video frames, the offsets of the small graphs in the large graphs need to be obtained again. FIG. 6D is a diagram illustrating an index table of the updated big graph. The thumbnail storage types comprise 0 and 1, wherein 1 represents thumbnail multiplexing storage, and 0 represents thumbnail single storage.

The storage device may extract the target image from the video or from a storage space of the target image and feed back the target image to the requester if a request to read the target image is received after the video is stored. The method specifically comprises the following steps: extracting the target image from the video according to the index table in the storage life cycle of the video when the request for reading the target image is received; when the storage life cycle of the video is over, the target image is extracted from the storage space of the target image according to the index table, after the target image is extracted, the target image is fed back to the requester, and when the target image is fed back to the requester, the picture format of the target image can be converted into the picture format corresponding to the requester, for example, when the requester requests the JPEG format, the format of the target image is converted into the JPEG format. The manner of extracting the target image from the video may refer to the image extraction method shown in fig. 6A in the foregoing embodiment, and is not described herein again.

The storage device may further receive the position information of the video stream and the target image sent by the camera, and the storage device acquires the target image from the corresponding video frame in the video stream according to the position information.

As shown in fig. 7, fig. 7 is an interaction diagram of a media data transmission method according to an embodiment of the present application. The data transmission method of the present embodiment includes the steps of:

s101, the camera acquires position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video.

The target image comprises a large image and/or a small image, wherein the large image can be a complete image of a certain video frame, or an image occupying the area of the certain video frame and exceeding a preset threshold value, and the like; a thumbnail may be a partial region of a certain video frame. In a particular embodiment, the thumbnail may include only a single target subject, or only a partial region of a single target subject.

The position information includes an absolute position, which may be, for example, a frame number, a timestamp, etc. of a video frame, and a relative position, which may be, for example, an offset with respect to a particular video frame.

Before performing the acquisition of the position information of the target image in the video, the camera generates a plurality of original video frames, and the camera generates the video by using the plurality of original video frames.

And S102, the camera sends the video and the position information to a storage device.

When the video camera sends the video and the position information to the storage device, the video and the position information can be sent at the same time or at non-same time.

S103, the storage device receives the video sent by the camera and the position information of the target image, wherein the target image is a video frame in the video or a part of the video frame in the video.

And S104, the storage equipment acquires the target image from the corresponding video frame of the video according to the position information.

When the storage device acquires the target image according to the position information, the storage device may acquire the target image according to an index table carrying the position information, specifically, acquire the target image from a corresponding video frame in the video according to the index table, or acquire the target image from a storage space of the target image according to the index table.

In one possible implementation form of the method,

in the case that the category of the target image is a thumbnail, the position information includes a second absolute position and/or a second relative position, where the second absolute position includes one or more of a frame number and a timestamp of a video frame corresponding to the target image in the video, and the second absolute position also includes a position of the target image in the corresponding video frame, and the position includes a coordinate and a size; the second relative position includes an offset of the target image relative to the particular video frame, coordinates and size of the target image in the corresponding video frame.

In one possible embodiment, the camera does not generate the target image and does not send the target image to the storage device.

In one possible embodiment, the method further comprises:

The target frame may be understood as a video frame in which the target image is located in the foregoing embodiment.

In one possible embodiment of the method according to the invention,

The I-frame in the group of pictures GOP of the video is a key frame.

In one possible embodiment, the camera acquires a target image from a plurality of original video frames, the target image having an image quality that includes an image of the target feature in the plurality of original video frames.

Target features may be understood as specific features, e.g. behavior between multiple subjects, etc.

In one possible embodiment, the method further comprises: the storage equipment receives the video sent by the camera and the position information of the target image;

In one possible embodiment, the method further comprises:

the storage device stores the target image;

the storage device deletes the video.

In one possible embodiment, the method further comprises:

For the sake of convenience, the present embodiment does not expand the definition of the large map, the small map, the location information, the index table, etc., and please refer to fig. 2, fig. 3A, fig. 3B, fig. 5A, fig. 5B, etc., and the description of the related large map, the small map, the location information, the index table, the definition of the specific video frame, etc. In this embodiment, the capturing of the video, the transmission of the video, and the like by the camera are not described, and please refer to fig. 1, fig. 3A, fig. 3B, and the related description. For other terms and explanations, refer to the description in the foregoing embodiments.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a media data transmission device provided in the present application. The media data transmission apparatus 800 according to the embodiment of the present application includes:

a first generating unit 810 for generating a plurality of original video frames;

a second generating unit 820 for generating a video using a plurality of original video frames;

an obtaining unit 830, configured to obtain position information of a target image in a video, where the target image is a video frame in the video or a portion of the video frame in the video;

a sending unit 840, configured to send the video and the location information to the storage device.

In one possible embodiment of the method according to the invention,

in the case that the category of the target image is a large image, the position information includes a first absolute position and/or a first relative position, wherein the first absolute position includes one or more of a frame number and a timestamp of the target image in the video, and the first relative position includes an offset of the target image relative to a specific video frame;

in one possible embodiment of the method according to the invention,

In one possible embodiment, the media data transfer device does not generate the target image and does not send the target image to the storage device.

In one possible embodiment, the method is further configured to:

selecting a target frame where a target image is located from a video;

In one possible embodiment of the method according to the invention,

In one possible embodiment, the method is further configured to: and acquiring a target image from the plurality of original video frames, wherein the image quality of the target image is the image of the plurality of original video frames including the target characteristic.

Referring to fig. 9, fig. 9 is a schematic structural diagram of a camera provided in the present application. The camera 900 of the present embodiment includes a processor 910 and a transceiver module 920, wherein,

a processor 910 configured to generate a plurality of original video frames, and generate a video using the plurality of original video frames; acquiring position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

and the transceiver module 920 is configured to send the video and the location information to the storage device.

In one possible embodiment of the method according to the invention,

In one possible embodiment, the processor 910 does not generate the target image and the transceiver module 920 does not transmit the target image to the storage device.

In one possible embodiment, the method is further configured to:

selecting a target frame where a target image is located from a video;

In one possible embodiment of the method according to the invention,

As shown in fig. 10, an embodiment of the present application further provides a video camera 1000, where the video camera 1000 includes a processor 1010, a memory 1020 and a transceiver 1030, where the memory 1020 stores instructions or programs, and the processor 1010 is configured to execute the instructions or programs stored in the memory 1020. When the instructions or programs stored in the memory 1020 are executed, the processor 1010 is configured to perform the operations performed by the processor 920 in the above embodiments, and the transceiver 1030 is configured to perform the operations performed by the transceiver module 902 in the above embodiments.

Referring to fig. 11, fig. 11 is a schematic structural diagram of a media data transmission device provided in the present application. The media data transmission apparatus 1100 provided in the embodiment of the present application includes:

a receiving unit 1110, configured to receive a video sent by a camera and position information of a target image, where the target image is a video frame in the video or a portion of the video frame in the video;

an obtaining unit 1120, configured to obtain a target image from a corresponding video frame of the video according to the position information.

In one possible embodiment of the method according to the invention,

in one possible embodiment of the method according to the invention,

In one possible embodiment, in the storage life cycle of the video, when a request for reading the target image is received, the target image is acquired from the corresponding video frame in the video according to the position information.

In one possible embodiment, the method is further configured to: when the storage life cycle of the video is finished, acquiring a target image from a corresponding video frame of the video according to the position information; save the target image, and delete the video.

In one possible embodiment, the method is further configured to:

Referring to fig. 12, fig. 12 is a schematic structural diagram of a storage device provided in the present application. The storage device 1200 provided in the embodiment of the present application includes a transceiver module 1210 and a processor 1220:

a transceiver module 1210, configured to receive a video sent by a camera and position information of a target image, where the target image is a video frame in the video or a portion of the video frame in the video;

and a processor 1220 for obtaining the target image from the corresponding video frame of the video according to the position information.

In one possible embodiment of the method according to the invention,

In one possible embodiment, during the storage life cycle of the video, when the storage device 1200 receives a request for reading a target image, the storage device 1200 obtains the target image from a corresponding video frame in the video according to the location information.

In one possible embodiment, the method is further configured to:

As shown in fig. 13, the embodiment of the present application further provides a server 1300, where the server 1300 includes a processor 1310, a memory 1320 and a transceiver 1330, where the memory 1320 stores instructions or programs, and the processor 1310 is configured to execute the instructions or programs stored in the memory 1320. When the instructions or programs stored in the memory 1320 are executed, the processor 1310 is configured to perform the operations performed by the processor 1220 and the transceiver 1330 is configured to perform the operations performed by the transceiver module 1210 in the above embodiments.

The present application further provides a computer-readable storage medium, where the computer-readable storage medium may store a program, and when the program is executed, the program includes some or all of the steps of any one of the media data transmission methods described in the above method embodiments.

Embodiments of the present application further provide a program product, wherein when the computer program product is read and executed by a computer, part or all of the steps of any of the media data transmission methods described in the above method embodiments are executed.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the embodiments of the present application are all or partially generated when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, digital subscriber line) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy disk, memory disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in view of the above, the content of the present specification should not be construed as a limitation to the present application.

Claims

1. A method for media data transmission, the method comprising:

a camera generates a plurality of original video frames;

the camera generating a video using the plurality of raw video frames;

the camera acquires position information of a target image in a video, wherein the target image is a video frame in the video or a part of the video frame in the video;

and the camera sends the video and the position information to a storage device.

2. The method of claim 1,

in the case that the category of the target image is a large map, the position information includes a first absolute position and/or a first relative position, wherein the first absolute position includes one or more of a frame number and a timestamp of the target image in the video, and the first relative position includes an offset of the target image relative to a specific video frame.

3. The method of claim 1,

in the case that the category of the target image is a small image, the position information includes a second absolute position and/or a second relative position, where the second absolute position includes one or more of a frame number and a timestamp of a video frame corresponding to the target image in the video, and the second absolute position further includes a position of the target image in the corresponding video frame; the second relative position includes an offset of the target image relative to a particular video frame, a position of the target image in a corresponding video frame.

4. A method according to any one of claims 1 to 3, wherein the camera does not generate the target image and does not transmit the target image to the storage device.

5. The method of any one of claims 1 to 4, further comprising:

the camera selects a target frame where the target image is located from the video;

6. The method according to any one of claims 1 to 5,

the target image is located in an I frame in a group of pictures (GOP) of the video.

7. The method of any one of claims 1 to 5, further comprising:

and the camera acquires a target image from the plurality of original video frames, wherein the target image is an image comprising target characteristics in the plurality of original video frames.

8. The method of any one of claims 1 to 5, further comprising: the storage equipment receives the video sent by the camera and the position information of the target image;

9. The method of any one of claims 1 to 5, further comprising:

when the storage life cycle of the video is finished, the storage equipment acquires the target image from the corresponding video frame of the video according to the position information;

the storage device saves the target image;

the storage device deletes the video.

10. The method of any one of claims 1 to 3, further comprising:

the storage device receives a video stream sent by the camera and position information of a target image, wherein the target image is a video frame in the video stream or a part of the video frame in the video stream;

11. A method for media data transmission, the method comprising:

the method comprises the steps that a storage device receives video sent by a camera and position information of a target image, wherein the target image is a video frame in the video or a part of the video frame in the video;

12. The method of claim 11,

13. The method of claim 11,

14. The method according to claims 11 to 13, wherein during the storage life cycle of the video, when the storage device receives a request for reading the target image, the storage device obtains the target image from the corresponding video frame in the video according to the location information.

15. The method of claims 11-14, further comprising: when the storage life cycle of the video is finished, the storage equipment acquires the target image from the corresponding video frame of the video according to the position information; the storage device saves the target image and deletes the video.

16. An apparatus for media data transmission, the apparatus comprising:

a first generating unit for generating a plurality of original video frames;

a second generating unit configured to generate a video using the plurality of original video frames;

an acquisition unit, configured to acquire position information of a target image in a video, where the target image is a video frame in the video or a portion of a video frame in the video;

and the sending unit is used for sending the video and the position information to a storage device.

17. The apparatus of claim 16,

18. The apparatus of claim 16,

19. The apparatus of any of claims 16 to 18, wherein the media data transfer means does not generate the target image and does not send the target image to the storage device.

20. A camera, characterized in that the camera comprises:

and the transceiver module is used for sending the video and the position information to a storage device.

21. The camera of claim 20,

22. The camera of claim 20,

23. A camera according to any one of claims 20 to 22, wherein the camera does not generate the target image and does not transmit the target image to the storage device.

24. A storage device, the device comprising:

the receiving and sending module is used for receiving a video sent by a camera and position information of a target image, wherein the target image is a video frame in the video or a part of the video frame in the video;

25. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to carry out the method according to any one of claims 1-15.